Computing resource allocation techniques and neural network systems

Abstract

The application provides a computing resource allocation technology and a neural network system. The neural network system comprises a processor and a plurality of neural network chips connected to the processor, each neural network chip comprising a plurality of computing units with storage and computing functions. The processor configures each neural network layer with computing units performing the neural network operation of the layer according to the output data volume of the neural network layer in the neural network system, so that the computing capacity of the computing units performing the operation of adjacent neural network layers is matched. The neural network system provided by the application can be applied to the field of artificial intelligence and improve the data processing efficiency of the neural network system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a computing resource allocation technology and a neural network system. Background Technology

[0002] Deep learning (DL) is an important branch of artificial intelligence (AI). It's a type of neural network designed to mimic the structure of the human brain, achieving better recognition results than traditional shallow learning methods. Convolutional Neural Networks (CNNs) are one of the most common deep learning architectures and the most widely researched deep learning methods. A typical application of CNNs is image processing. Image processing involves recognizing and analyzing input images, ultimately outputting a set of categorized image content. For example, we can use CNN algorithms to extract and classify the body color, license plate number, and vehicle type of a car in an image.

[0003] Convolutional neural networks (CNNs) typically extract image features using a three-layer sequence: convolutional layers, pooling layers, and rectified linear units (ReLUs). The process of extracting image features is essentially a series of matrix operations (e.g., matrix multiplication and addition). Therefore, how to process images in parallel and quickly within the network is a key research area for CNNs. Summary of the Invention

[0004] This application provides a computing resource allocation technology and a neural network system that can improve the data processing speed in neural networks.

[0005] In a first aspect, embodiments of the present invention provide a method for allocating computing resources in a neural network system. This method can be executed by a host connected to a neural network chip. According to this method, after obtaining the data volume of the first output data of the first neural network layer and the data volume of the second output data of the second neural network layer in the neural network system, N first weights to be configured for the first neural network layer and M second weights to be configured for the second neural network layer are determined based on the deployment requirements of the neural network system. Further, according to the computing specifications of the computing units in the neural network system, the N first weights are deployed to P computing units, and the M second weights are deployed to Q computing units. The input data of the second neural network layer includes the first output data. N and M are both positive integers, and the ratio of N to M corresponds to the ratio of the data volume of the first output data to the data volume of the second output data. P and Q are both positive integers. The P computing units are used to execute the operations of the first neural network layer, and the Q computing units are used to execute the operations of the second neural network layer.

[0006] The computing resource allocation method provided in this embodiment of the invention considers the amount of data output by adjacent neural network layers when configuring computing units to execute each layer of neural network operations according to deployment requirements, so that the computing capabilities of computing nodes executing different neural network layer operations are matched, thereby making full use of the computing capabilities of computing nodes executing each layer of neural network operations and improving the efficiency of data processing.

[0007] In conjunction with the first aspect, in one possible implementation, the deployment requirement includes computational latency, and the first neural network layer is the starting layer for all neural network layers in the neural network system. Determining the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer includes: determining the value of N based on the amount of the first output data, the computational latency, and the computation frequency of the Resistive Random Access Memory (ReRAM) crossbar in the computing unit; and determining the value of M based on the ratio of the amount of the first output data to the amount of the second output data and the value of N.

[0008] Specifically, in one possible implementation, when the first neural network layer is the starting layer of all neural network layers in the neural network system, the value of N can be obtained according to the following formula:

[0009]

[0010] in, The number N used to indicate the number of weights N that need to be configured in the first layer of the neural network. This represents the number of rows in the output data of the first layer of the neural network. t is the number of columns of output data from the first layer of the neural network. t is the set computation delay, and f is the computation frequency of the CrossBar in the computation unit. The value of M can be calculated using the following formula: N / M = First output data volume / Second output data volume.

[0011] In conjunction with the first aspect, in another possible implementation, the neural network system includes multiple neural network chips, each neural network chip includes multiple computing units, and each computing unit includes at least one Resistive Random Access Memory (ReRAM) crossbar. The deployment requirement includes the number of chips in the neural network system. When the first neural network layer is the starting layer of the neural network system, determining the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer includes: determining the value of N based on the number of chips, the number of ReRAM crossbars in each chip, the number of ReRAM crossbars required to deploy one weight of each neural network layer, and the ratio of the output data volume of adjacent neural network layers; and determining the value of M based on the ratio of the data volume of the first output data to the data volume of the second output data and the value of N.

[0012] Specifically, in one possible implementation, when the deployment requirement is the number of chips required by the neural network system, and the first neural network layer is the starting layer of the neural network system, the N first weights to be configured in the first neural network layer and the M second weights to be configured in the second neural network layer can be obtained according to the following two formulas, where the value of N is... The value of M is... The value of .

[0013]

[0014]

[0015] Where xb1 represents the number of crossbars required to deploy one weight of the first (or initial) layer of the neural network. xb1 represents the number of weights required for the initial layer, and xb2 represents the number of crossbars required to deploy one weight in the second layer of the neural network. This represents the number of weights required for the second layer of the neural network. xb n This represents the number of crossbars required to deploy one set of weights in the nth layer of the neural network. The number of weights required for the nth layer of the neural network is represented by K, the number of chips required for the neural network system to be deployed, and L, the number of crossbars in each chip. Used to represent the number of weights required for the i-th layer; Used to represent the number of weights required for the (i-1)th layer. Used to represent the number of rows in the output data of the i-th layer. The number of columns used to represent the output data of the i-th layer. Used to represent the row number of the output data of the (i-1)th layer. The number of columns used to represent the output data of the (i-1)th layer, where the value of i can be from 2 to n, and n is the total number of layers in the neural network system.

[0016] In conjunction with the first aspect, in another possible implementation, the neural network system includes multiple neural network chips, each neural network chip includes multiple secondary computing nodes, and each secondary computing node includes multiple computing units. The method further includes mapping the P computing units and the Q computing units to multiple secondary computing nodes according to the number of computing units contained in the secondary computing nodes of the neural network system. Specifically, at least a portion of the P computing units and at least a portion of the Q computing units are mapped to the same secondary computing node. This approach allows as many computing units performing operations on adjacent neural network layers as possible to reside in the same secondary computing node, thereby reducing the amount of data transmitted between computing nodes and improving the data transmission speed between different neural network layers.

[0017] In another possible implementation, the method further includes mapping the plurality of secondary computing nodes mapped to the P computing units and the Q computing units to the plurality of neural network chips according to the number of secondary computing nodes contained in each neural network chip. Specifically, at least a portion of the secondary computing nodes belonging to the P computing units and at least a portion of the secondary computing nodes belonging to the Q computing units are mapped to the same neural network chip. This approach allows as many secondary computing nodes performing operations on adjacent neural network layers as possible to reside in the same neural network chip, further reducing the amount of data transmitted between computing nodes and improving the data transmission speed between different neural network layers.

[0018] In conjunction with the first aspect and one possible implementation of the first aspect, in another possible implementation, the ratio of N and M corresponding to the ratio of the amount of the first output data to the amount of the second output data includes: the ratio of N and M being the same as the ratio of the amount of the first output data to the amount of the second output data.

[0019] In a second aspect, this application provides a neural network system, including a host and multiple neural network chips, each neural network chip including multiple computing units, the host being connected to the multiple neural network chips and used to execute the computing resource allocation method described in the first aspect or any possible implementation of the first aspect.

[0020] Thirdly, this application provides a resource allocation apparatus, including a functional module capable of performing the computational resource allocation method described in the first aspect and any possible implementation thereof.

[0021] Fourthly, this application also provides a computer program product, including program code, wherein the instructions included in the program code are executed by a computer to implement the computing resource allocation method described in the first aspect and any possible implementation thereof.

[0022] Fifthly, this application also provides a computer-readable storage medium for storing program code, the program code including instructions that are executed by a computer to implement the computing resource allocation method described in the first aspect and any possible implementation thereof. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0024] Figure 1 A schematic diagram of the structure of a neural network system provided in an embodiment of the present invention;

[0025] Figure 1A This is a schematic diagram of the structure of another neural network system provided in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of a computing node in a neural network chip provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of the logical structure of a neural network layer in a neural network system provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of a set of computing nodes in a neural network system that processes data from different neural network layers, provided by an embodiment of the present invention.

[0029] Figure 5A flowchart of a computational resource allocation method in a neural network system provided by an embodiment of the present invention;

[0030] Figure 6 A flowchart illustrating another computing resource allocation method provided in an embodiment of the present invention;

[0031] Figure 6A A flowchart of a resource mapping method provided in an embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram of another computing resource allocation method provided in an embodiment of the present invention;

[0033] Figure 8 A flowchart of a data processing method provided in an embodiment of the present invention;

[0034] Figure 9 A weighting diagram provided for an embodiment of the present invention;

[0035] Figure 10 A schematic diagram of the structure of a ReRAM crossbar provided in an embodiment of the present invention;

[0036] Figure 11 This is a schematic diagram of a resource allocation device provided in an embodiment of the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments.

[0038] Deep learning (DL) is an important branch of artificial intelligence (AI). Deep learning aims to mimic the structure of neural networks in the human brain, achieving better recognition results than traditional shallow learning methods. Artificial neural networks (ANNs), also known as neural networks (NNs) or neural-like networks, are mathematical or computational models in machine learning and cognitive science that mimic the structure and function of biological neural networks (the central nervous system of animals, especially the brain) for estimating or approximating functions. Artificial neural networks can include convolutional neural networks (CNNs), deep neural networks (DNNs), and multilayer perceptrons (MLPs), among others. Figure 1 This is a schematic diagram of the structure of an artificial neural network system provided in an embodiment of the present invention. Figure 1 The following diagram illustrates a convolutional neural network as an example. Figure 1 As shown, the convolutional neural network system 100 may include a host 105 and a convolutional neural network circuit 110. The convolutional neural network circuit 110 may also be referred to as a neural network accelerator. The convolutional neural network circuit 110 is connected to the host 105 via a host interface. The host interface may include a standard host interface and a network interface. For example, the host interface may include a Peripheral Component Interconnect Express (PCIE) interface. Figure 1 As shown, the convolutional neural network circuit 110 can be connected to the host 105 via the PCIe bus 106. Therefore, data can be input to the convolutional neural network circuit 110 via the PCIe bus 106, and processed data can be received from the convolutional neural network circuit 110 via the PCIe bus 106. Furthermore, the host 105 can also monitor the operating status of the convolutional neural network circuit 110 through the host interface.

[0039] The host 105 may include a processor 1052 and memory 1054. It should be noted that, in addition to... Figure 1 In addition to the devices shown, the host 105 may also include a communication interface and other devices such as a disk as external storage, without limitation.

[0040] Processor 1052 is the processing and control unit of host 105. Processor 1052 may include multiple processor cores. Processor 1052 may be a very large-scale integrated circuit. An operating system and other software programs are installed on processor 1052, enabling processor 1052 to control memory 1054, cache, disk, and peripheral devices (such as…). Figure 1 Access to the neural network circuit in the processor 1052. It is understood that, in this embodiment of the invention, the Core in the processor 1052 may be, for example, a central processing unit (CPU) or other application-specific integrated circuit (ASIC).

[0041] Memory 1054 is the main memory of host 105. Memory 1054 is connected to processor 1052 via a double data rate (DDR) bus. Memory 1054 is typically used to store various running software, input and output data, and information exchanged with external storage in the operating system. To improve the access speed of processor 1052, memory 1054 needs to have a high access speed. In traditional computer system architectures, Dynamic Random Access Memory (DRAM) is typically used as memory 1054. Processor 1052 can access memory via the memory controller (…). Figure 1 (Not shown in the image) High-speed access memory 1054, performing read and write operations on any storage unit in memory 1054.

[0042] The Convolutional Neural Network (CNN) circuit 110 is a chip array composed of multiple neural network (NN) chips. For example, such as Figure 1As shown, the CNN circuit 110 includes multiple NN chips 115 and multiple routers 120. For ease of description, the NN chip 115 in this embodiment is simply referred to as chip 115. The multiple chips 115 are interconnected through routers 120. For example, one chip 115 can be connected to one or more routers 120. The multiple routers 120 can form one or more network topologies. Data can be transmitted between the chips 115 through the one or more network topologies. For example, the multiple routers 120 can form a first network 1106 and a second network 1108, wherein the first network 1106 is a ring network and the second network 1108 is a two-dimensional mesh network. Thus, data input from input port 1102 can be sent to the corresponding chip 115 through the network formed by the multiple routers 120, and data processed by any chip 115 can also be sent to other chips 115 for processing or sent out from output port 1104 through the network formed by the multiple routers 120.

[0043] Furthermore, Figure 1 A schematic diagram of the structure of chip 115 is also shown. (For example...) Figure 1 As shown, chip 115 may include multiple neural network processing units 125 and multiple routers 122. Figure 1 This description uses tiles as the processing unit in a neural network. Figure 1 In the architecture of the data processing chip 115 shown, one tile 125 can be connected to one or more routers 122. The multiple routers 122 in the chip 115 can form one or more network topologies. Data transmission between tiles 125 can occur through these various network topologies. For example, the multiple routers 122 can form a first network 1156 and a second network 1158, where the first network 1156 is a ring network and the second network 1158 is a two-dimensional mesh network. Thus, data input to the chip 115 from the input port 1152 can be sent to the corresponding tile 125 according to the network formed by the multiple routers 122, and data processed by any tile 125 can also be sent to other tiles 125 or sent out from the output port 1154 through the network formed by the multiple routers 122.

[0044] It should be noted that when chips 115 are interconnected via routers, the network topology formed by multiple routers 120 in the convolutional neural network circuit 110 and the network topology formed by multiple routers 122 in the data processing chip 115 can be the same or different, as long as data transmission can be performed between chips 115 or between tiles 125 through the network topology, and chips 115 or tiles 125 can receive or output data through the network topology. In this embodiment of the invention, the number and type of networks formed by multiple routers 120 and 122 are not limited. Furthermore, in this embodiment of the invention, routers 120 and 122 can be the same or different. For clarity, in... Figure 1 The router 120 connecting the chip and the router 122 connecting the tile are distinguished by their identifiers. For ease of description, in this embodiment of the invention, the chip 115 or tile 125 in the convolutional neural network system can also be referred to as a computing node.

[0045] In practical applications, under another scenario, chips 115 can also be interconnected via a high-speed interface (High Transport I / O) without going through router 120. For example... Figure 1A As shown, Figure 1A This is a schematic diagram of the structure of another neural network system provided in an embodiment of the present invention. Figure 1A In the neural network system shown, the host 105 is connected to multiple PCIe cards 109 via a PCIe interface 107. Each PCIe card 109 can include multiple neural network chips 115, which are connected to each other via a high-speed interconnect interface. The interconnection method between the chips is not limited here. It is understood that in practical applications, the tiles within a chip may also be connected without a router, using... Figure 1A The diagram illustrates a high-speed interconnection method between chips. In another scenario, it could also involve interconnecting tiles within the chip. Figure 1 The router connection shown is used, while the chips use... Figure 1A The high-speed interconnection method shown is not limited to the connection method between or within chips, as described in this embodiment of the invention.

[0046] Figure 2 This is a schematic diagram of the structure of a computing node in a neural network chip provided by an embodiment of the present invention. Figure 2 As shown, chip 115 includes multiple routers 120, each of which can connect to one tile 125. In practical applications, one router 120 can also connect to multiple tiles 125. Figure 2As shown, each tile 125 may include an input / output interface (TxRx) 1252, a switching device (TSW) 1254, and multiple processing elements (PEs) 1256. The TxRx 1252 is used to receive data input to tile 125 from router 120, or to output the calculation results of tile 125. Alternatively, the TxRx 1252 is used to implement data transmission between tile 125 and router 120. The switch (TSW) 1254 connects to the TxRx 1252 and is used to implement data transmission between the TxRx 1252 and the multiple PEs 1256. Each PE 1256 may include one or more computing engines 1258, which are used to perform neural network calculations on the data input to the PE 1256. For example, the data input to tile 125 can be multiplied and added with a preset convolution kernel in tile 125. The computation results of Engine 1258 can be sent to other tiles 125 via TSW1254 and TxRx 1252. In practical applications, an Engine 1258 may include modules that implement convolution, pooling, or other neural network operations. The specific circuitry or function of the Engine is not limited here. For simplicity, in this embodiment of the invention, the computation engine is simply referred to as the engine.

[0047] Those skilled in the art will recognize that resistive random-access memory (ReRAM), a novel type of non-volatile memory, has been widely used in neural network systems in recent years due to its integrated storage and computation capabilities. For example, a ReRAM crossbar composed of multiple memristor cells can be used to perform matrix multiplication and addition operations in a neural network system. In this embodiment of the invention, Engine 1258 may include one or more crossbars. The structure of the ReRAM crossbar can be as follows... Figure 10As shown, the following section will introduce how to perform matrix multiplication and addition operations using ReRAM crossbars. Based on the above description of the neural network, it can be seen that the neural network circuit provided in this embodiment includes multiple NN chips, each NN chip includes multiple tiles, each tile includes multiple processing devices (PEs), each PE includes multiple engines, and each engine is implemented by one or more ReRAM crossbars. Therefore, the neural network system provided in this embodiment can include multi-level computing nodes, for example, it can include four levels of computing nodes: the first level computing node is chip 115, the second level computing node is the tile within the chip, the third level computing node is the PE within the tile, and the fourth level computing node is the engine within the PE.

[0048] On the other hand, those skilled in the art will understand that a neural network system can include multiple neural network layers. In this embodiment of the invention, a neural network layer is a logical layer concept; one neural network layer refers to one neural network operation to be performed. Each layer of neural network computation is implemented by computation nodes. Neural network layers can include convolutional layers, pooling layers, etc. Figure 3 As shown, a neural network system can include n neural network layers (also known as an n-layer neural network), where n is an integer greater than or equal to 2. Figure 3 This shows a portion of the neural network layers in a neural network system, such as... Figure 3 As shown, the neural network system may include layers 302, 304, 306, 308, 310, up to layer n 312. Layer 302 can perform convolution operations; layer 304 can perform pooling operations on the output data of layer 302; layer 306 can perform convolution operations on the output data of layer 304; layer 308 can perform convolution operations on the output of layer 306; layer 310 can perform summation operations on the output data of layer 304 and layer 308, and so on. It is understood that... Figure 4 This is just a simple example and explanation of neural network layers in a neural network system. It does not restrict the specific operations of each neural network layer. For example, the fourth layer 308 can also be a pooling operation, and the fifth layer 310 can also be a convolution operation or a pooling operation, or other neural network operations.

[0049] In existing neural network systems, after the calculation of the i-th layer is completed, the calculation result of the i-th layer is temporarily stored in a preset cache. When performing the calculation of the (i+1)-th layer, the computing unit needs to reload the calculation result of the i-th layer and the weights of the (i+1)-th layer from the preset cache for calculation. Here, the i-th layer can be any layer in the neural network system. In this embodiment of the invention, since the neural network system's Engine uses a ReRAM crossbar, and since ReRAM has the advantage of in-memory computation, the weights can be configured on the ReRAM cell before calculation, and the calculation result can be directly sent to the next layer for pipelined computation. Therefore, each neural network layer only needs to cache a small amount of data; for example, each neural network layer only needs to cache enough input data for one window calculation. Furthermore, to achieve parallel and fast data processing, this embodiment of the invention provides a method for stream processing of data using a neural network. For clarity, the following describes... Figure 1 This section provides a brief overview of convolutional neural network systems and their stream processing capabilities.

[0050] like Figure 4 As shown, in order to achieve fast data processing, the computing nodes in the system can be divided into multiple node sets to perform calculations for different neural network layers respectively. Figure 4 To Figure 1 Taking the tile 125 partitioning in the neural network system shown as an example, this invention illustrates different sets of computational nodes used to implement neural network computation at different layers. Figure 4 As shown, multiple tiles 125 in chip 115 can be divided into multiple node sets. For example: a first node set 402, a second node set 404, a third node set 406, a fourth node set 408, and a fifth node set 410. Each node set includes at least one computing node (e.g., tile 125). Computing nodes within the same node set perform neural network operations on data entering the same neural network layer; data from different neural network layers is processed by computing nodes in different node sets. The processing result from one computing node is transmitted to computing nodes in other node sets for further processing. This pipelined processing method allows each neural network layer to cache only a small amount of data and enables multiple computing nodes to concurrently process the same data stream, improving processing efficiency. It should be noted that... Figure 4This example uses a tile to illustrate a set of computational nodes used to process different neural network layers (e.g., convolutional layers). In practical applications, a tile contains multiple physical nodes (PEs), each PE contains multiple engines, and the computational load varies across different application scenarios. Therefore, computational nodes in a neural network system can be divided into PEs, engines, or chips based on the actual application, allowing computational nodes in different sets to be used for processing operations of different neural network layers. In this way, the computational nodes referred to in this embodiment of the invention can be engines, PEs, tiles, or chips.

[0051] Furthermore, those skilled in the art will understand that when a computing node (e.g., tile 125) performs a neural network operation (e.g., convolution calculation), it can compute the data input to the computing node based on the weights of the corresponding neural network layer. For example, a tile 125 can perform a convolution operation on the input data input to that tile 125 based on the weights of the corresponding convolutional layer, such as performing matrix multiplication and addition on the weights and input data. Weights are typically used to represent the importance of input data to output data. In neural networks, weights are usually represented by a matrix. Figure 9 As shown, Figure 9 The j-row, k-column weight matrix shown can represent the weights of a neural network layer, where each element represents a weight value. In this embodiment, since the computational nodes of a node set are used to perform the operations of a neural network layer, computational nodes within the same node set can share weights, while the weights of computational nodes in different node sets can be different. In this embodiment, the weights in each computational node can be pre-configured. Specifically, each element in a weight matrix is ​​configured in a ReRAM cell within the corresponding crossbar array, thus enabling matrix multiplication and addition operations between the input data and the configured weights through the crossbar array. A brief explanation of how matrix multiplication and addition operations are implemented using crossbars will follow.

[0052] As described above, in this embodiment of the invention, during the implementation of neural network stream processing, the computing nodes in the neural network can be divided into node sets for processing different neural network layers, and corresponding weights can be configured. Thus, computing nodes in different node sets can perform corresponding calculations according to the configured weights. Furthermore, the computing nodes in each node set can send the calculation results to the computing nodes used to execute the next layer of neural network operations. Those skilled in the art will understand that during the implementation of neural network stream processing, if the computing resources for executing different layers of neural network operations are mismatched—for example, if there are few computing resources for executing upper-layer neural network operations, while there are relatively more computing resources for executing lower-layer neural network operations—it will lead to a waste of computing resources for the lower-layer computing nodes. To fully utilize the computing power of computing nodes and match the computing power of computing nodes executing different layers of neural network operations, this embodiment of the invention provides a computing resource allocation method for allocating computing nodes for executing different layers of neural network operations. This ensures that the computing power of computing nodes used to execute adjacent layers of neural network operations is matched, improving the data processing efficiency of the neural network system and preventing the waste of computing resources.

[0053] Figure 5 This is a flowchart illustrating a method for allocating computational resources in a neural network system, provided as an embodiment of the present invention. This method can be applied to... Figure 1 The neural network system shown. This method can be implemented by the host 105 during the deployment of the neural network or the configuration of the neural network system; specifically, it can be implemented by the processor 1052 in the host 105. For example... Figure 5 As shown, the method may include the following steps.

[0054] In step 502, network model information of the neural network system is obtained. This network model information includes the first output data volume of the first neural network layer and the second output data volume of the second neural network layer. The network model information can be determined according to actual application requirements. For example, the total number of neural network layers and the algorithm for each layer can be determined based on the application scenario of the neural network system. The network model information may include the total number of neural network layers in the neural network system, the algorithm for each layer, and the data output volume of each neural network layer. In this embodiment of the invention, the algorithm refers to the neural network operation to be performed; for example, the algorithm may refer to convolution operations, pooling operations, etc. Figure 3As shown, the neural network system of this embodiment can have n layers, where n is an integer not less than 2. In this step, the first neural network layer and the second neural network layer can be two layers out of the n layers that have an operational dependency. In this embodiment, two neural network layers with a dependency mean that the input data of one neural network layer includes the output data of the other neural network layer. Two neural network layers with a dependency can also be referred to as adjacent layers. For example, as... Figure 3 As shown, the output data of the first layer 302 is the input data of the second layer 304; therefore, the first layer 302 and the second layer 304 are dependent on each other. The output data of the second layer 304 is the input data of the third layer 306, and the input data of the fifth layer 310 includes the output data of the second layer 304; therefore, the second layer 304 and the third layer 306 are dependent on each other, and the second layer 304 and the fifth layer 310 are also dependent on each other. For clarity, in this embodiment of the invention, ... Figure 3 The first layer 302 shown is the first neural network layer, and the second layer 304 is the second neural network layer, as illustrated in the example.

[0055] In step 504, based on the deployment requirements of the neural network system, the first output data volume, and the second output data volume, the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer are determined. Here, N and M are both positive integers, and the ratio of N to M corresponds to the ratio of the first output data volume to the second output data volume. In practical applications, deployment requirements may include the computational latency of the neural network system, or the number of chips required to deploy the neural network system. Those skilled in the art will understand that neural network operations mainly perform matrix multiplication and addition operations, and the output data of each neural network layer is also a one-dimensional or multi-dimensional real matrix. Therefore, the first output data volume includes the number of rows and columns of the output data of the first neural network layer, and the second output data volume includes the number of rows and columns of the output data of the second neural network layer.

[0056] As mentioned earlier, when a computing node performs neural network operations, such as convolution or pooling, it needs to perform multiplication and addition calculations on the input data and the weights of the corresponding neural network layer. Since the weights are configured on cells in the crossbar, and the crossbars in the computing unit perform calculations on the input data in parallel, the number of weights can determine the parallel computing capability of multiple computing units performing neural network operations. In other words, the computing capability of a computing node performing a neural network operation is determined by the number of weights configured in the computing unit performing the neural network operation. In this embodiment of the invention, to match the computing capabilities of two neural network layers performing adjacent operations in the neural network system, the number of weights required for the first and second neural network layers can be determined based on specific deployment requirements and the first and second output data volumes. Since the weights of different neural network layers are not necessarily the same, for clarity, in this embodiment of the invention, the weights required for the operation of the first neural network layer are referred to as the first weights, and the weights required for the operation of the second neural network layer are referred to as the second weights. Performing an operation on the first neural network layer means that the computing node performs corresponding calculations on the data input to the first neural network layer based on the first weights; performing an operation on the first neural network layer means that the computing node performs corresponding calculations on the data input to the second neural network layer based on the second weights. The computation here can be performing neural network operations such as convolution or pooling.

[0057] The following sections will describe in detail how to determine the number of weights required for each layer of the neural network in this step, based on different deployment requirements. The number of weights required for each layer includes the number N of first weights required for the first neural network layer and the number M of second weights required for the second neural network layer. In this embodiment, weight refers to a weight matrix. The number of weights refers to the number of weight matrices required, or the number of copies of the weights. The number of weights can also be understood as how many identical weight matrices need to be configured.

[0058] In one scenario, when the deployment requirement of the neural network system is limited by its computational latency, to ensure that the computation of the entire neural network system does not exceed the set computational latency, the number of weights required for the first layer (i.e., the initial layer among all neural network layers in the system) can be determined first based on its data output, computational latency, and the computation frequency of the ReRAM crossbar used in the neural network system. Then, the number of weights required for each layer can be obtained based on the number of weights required for the first layer and the output data volume of each layer. Specifically, the number of weights required for the first layer (i.e., the initial layer) can be obtained using the following formula:

[0059]

[0060] (Formula 1)

[0061] in, Used to indicate the number of weights required to be configured in the first layer (i.e., the starting layer) of the neural network. This refers to the number of rows of output data in the first layer (i.e., the initial layer) of the neural network. The first layer (i.e., the starting layer) of the neural network outputs the number of columns. t is the set computation delay, and f is the computation frequency of the CrossBar in the computation unit. Those skilled in the art will understand that the value of f can be obtained based on the configuration parameters of the crossbar used. The amount of data output by the first layer of the neural network can be obtained based on the network model information obtained in step 502. It should be noted that, in this embodiment of the invention, the first layer of the neural network is the starting layer of all neural network layers in the neural network system. It can be understood that when the first neural network is the starting layer of all neural network layers in the neural network system, the number of first weights N is calculated according to Formula 1. The value of .

[0062] After obtaining the number of weights required for the first layer of the neural network, in order to improve the data processing efficiency of the neural network system, avoid bottlenecks or data waiting in pipelined parallel processing, and match the processing speed of adjacent neural network layers, in this embodiment of the invention, the ratio of the number of weights required by two adjacent layers can correspond to the ratio of the output data volume of two adjacent layers. For example, the ratio can be the same. Therefore, in this embodiment of the invention, the number of weights required for each layer of the neural network can be determined based on the ratio of the number of weights required for the first layer of the neural network to the output data volume of each layer of the neural network. Specifically, the number of weights required for each layer of the neural network can be calculated according to the following formula (II):

[0063]

[0064] (Formula 2)

[0065] in, Used to represent the number of weights required for the i-th layer; Used to represent the number of weights required for the (i-1)th layer. Used to represent the number of rows in the output data of the i-th layer. The number of columns used to represent the output data of the i-th layer. Used to represent the row number of the output data of the (i-1)th layer. The number of columns used to represent the output data of the (i-1)th layer, where i can be from 2 to n, and n is the total number of layers in the neural network system. Alternatively, in this embodiment of the invention, the ratio of the number of weights required to perform the neural network operation of the (i-1)th layer to the number of weights required to perform the neural network operation of the i-th layer corresponds to the ratio of the output data volume of the (i-1)th layer to the output data volume of the i-th layer.

[0066] Those skilled in the art will understand that the output data of each neural network layer can include multiple channels, where a channel refers to the number of kernels in each neural network layer. One kernel represents a feature extraction method, corresponding to the generation of a feature map; multiple feature maps constitute the output data of that layer. The weights used in a neural network layer include multiple kernels. Therefore, in practical applications, in another scenario, the output data volume of each layer can also consider the number of channels in each neural network layer. Specifically, after obtaining the number of weights required for the first neural network layer according to Formula 1 above, the number of weights required for each layer of the neural network can be obtained according to Formula 3 below:

[0067]

[0068] (Formula 3)

[0069] The difference between Formula 3 and Formula 2 is that Formula 3, based on Formula 2, further considers the number of channels in the output of each layer of the neural network. Wherein, C... i-1 C is used to represent the number of channels in the (i-1)th layer. i The number of channels in the i-th layer is used to represent the number of channels in the i-th layer, where i ranges from 2 to n, and n is the total number of layers in the neural network system, and n is an integer not less than 2. The number of channels in each layer of the neural network can be obtained from the network model information.

[0070] In this embodiment of the invention, after obtaining the number of weights required for the initial layer according to Formula 1, the number of weights required for each neural network layer can be calculated according to Formula 2 (or Formula 3) and the output data volume of each neural network layer contained in the network model information. For example, when the first neural network layer is the initial layer of all neural network layers in the neural network system, after calculating the number N of the first weight according to Formula 1, the number M of the second weight required for the second neural network layer can be obtained according to Formula 2, based on the value of N and the set first and second output data volumes. Alternatively, after obtaining the value of N, the value of M can be calculated according to the following formula: N / M = First output data volume / Second output data volume.

[0071] In another scenario, when the deployment requirement is the number of chips required for the neural network system, the number of weights required for the first layer of the neural network can be calculated by combining Formula 4 below and Formula 2 above, or by combining Formula 4 below and Formula 3 above.

[0072]

[0073] (Formula 4)

[0074] In Formula 4 above, xb1 represents the number of crossbars required to deploy one weight of the first (or initial) layer of the neural network. xb1 represents the number of weights required for the initial layer, and xb2 represents the number of crossbars required to deploy one weight in the second layer of the neural network. This represents the number of weights required for the second layer of the neural network. xb n This represents the number of crossbars required to deploy one set of weights in the nth layer of the neural network. The formula represents the number of weights required for the nth layer of the neural network, K is the number of chips required for the deployment of the neural network system, and L is the number of crossbars in each chip. Formula 4 above indicates that the total number of crossbars in each neural network layer is less than or equal to the total number of crossbars included in the chips of the neural network. The descriptions of Formulas 2 and 3 can be found in the preceding descriptions and will not be repeated here.

[0075] Those skilled in the art will understand that once the model of a neural network system is determined, the weights of each layer and the specifications of the crossbar used in the system (i.e., the number of rows and columns of ReRAM cells in the crossbar) are already determined. In other words, the network model information of the neural network system also includes the magnitude of a weight used in each layer and the specifications of the crossbar. Therefore, in this embodiment of the invention, the xb of the i-th layer of the neural network can be obtained based on the magnitude of the weights (i.e., the number of rows and columns of the weight matrix) and the specifications of the crossbar. i Where i ranges from 1 to n. The value of L can be obtained from the parameters of the chip used in the neural network system. In one embodiment of the invention, the number of weights required for the initial layer neural network is obtained according to Formula 4 and Formula 2 above (i.e., After that, the number of weights required for each layer can be obtained according to Formula 2 and the amount of output data for each layer obtained from the network model information. In another scenario, the number of weights required for the initial layer of the neural network is obtained according to Formulas 4 and 3 above (i.e.,...). After that, the number of weights that need to be configured for each layer can also be obtained according to Formula 3 and the amount of output data for each layer.

[0076] In step 506, according to the computational specifications of the computational units in the neural network system, N first weights are deployed to P computational units, and M second weights are deployed to Q computational units. Here, P and Q are positive integers. The P computational units are used to execute the operations of the first neural network layer, and the Q computational units are used to execute the operations of the second neural network layer. In this embodiment, the computational specifications of the computational unit refer to the number of crossbars contained in a computational unit. In practical applications, a computational unit may include one or more crossbars. Specifically, as mentioned above, since the network model information of the neural network system also includes the size of a weight used by each neural network layer and the specification information of the crossbars, a deployment relationship between weights and crossbars can be obtained. After obtaining the number of weights required for each layer of the neural network in step 504, the weights of each layer can be deployed to the corresponding number of computational units according to the number of crossbars contained in each computational unit. Specifically, the elements in the weight matrix are respectively configured in the ReRAM cells of the crossbars of the computational unit. In this embodiment, a computational unit may refer to a PE or an engine; a PE may include multiple engines, and an engine may include one or more crossbars. Since the weights in each layer may be different, a weight can be deployed on one or more engines.

[0077] Specifically, in this step, based on the deployment relationship between a weight and a crossbar, and the number of crossbars contained in the computation unit, the P computation units to which the N first weights need to be deployed and the Q computation units to which the M second weights need to be deployed can be determined. For example, the N first weights of the first neural network layer can be deployed to P computation units, and the M second weights can be deployed to Q computation units. Specifically, the elements of the N first weights are respectively configured into the ReRAM cells of the corresponding crossbars in the P computation units. The elements of the M second weights are respectively configured into the ReRAM cells of the corresponding crossbars in the Q computation units. Thus, the P computation units can perform the operation of the first neural network layer on the input data input to the P computation units based on the configured N first weights, and the Q computation units can perform the operation of the second neural network layer on the input data input to the Q computation units based on the configured Q second weights.

[0078] As can be seen from the above embodiments, the computing resource allocation method provided by the present invention considers the amount of data output by adjacent neural network layers when configuring the computing unit that executes each layer of neural network operation according to deployment requirements, so that the computing capabilities of computing nodes that execute different neural network layer operations are matched, thereby making full use of the computing capabilities of computing nodes and improving the efficiency of data processing.

[0079] Furthermore, in this embodiment of the invention, to further reduce the amount of data transmitted between computing units executing different neural network layers and save transmission bandwidth between computing units or computing nodes, computing units can be mapped to their parent computing nodes using the following method. As mentioned above, the neural network system may include four levels of computing nodes: a first-level computing node (chip), a second-level computing node (tile), a third-level computing node (PE), and a fourth-level computing node (engine). Figure 6 Taking the fourth-level computing node (engine) as an example, this paper describes in detail how to map the P computing units that need to be deployed with the N first weights and the Q computing units that need to be deployed with the M second weights to the upper-level computing node. This method can still be implemented by... Figure 1 and Figure 1A This is implemented by host computer 105 in the neural network system shown. Figure 6 As shown, the method may include the following steps.

[0080] In step 602, network model information of the neural network system is obtained. The network model information includes the first output data volume of the first neural network layer and the second output data volume of the second neural network layer in the neural network system. In step 604, based on the deployment requirements of the neural network system, the first output data volume, and the second output data volume, N first weights to be configured for the first neural network layer and M second weights to be configured for the second neural network layer are determined. In step 606, based on the computing specifications of the computing units in the neural network system, P computing units to be deployed for the N first weights and Q computing units to be deployed for the M second weights are determined. In this embodiment of the invention, steps 602, 604, and 606 can be referred to the relevant descriptions in steps 502, 504, and 506 above, respectively. The difference between step 606 and step 506 is that in step 606, after determining the P computing units to be deployed for the N first weights and the Q computing units to be deployed for the M second weights, the N first weights are not directly deployed to the P computing units; instead, the M second weights are deployed to the Q computing units. Instead, proceed to step 608.

[0081] In step 608, based on the number of computing units contained in the third-level computing nodes of the neural network system, the P computing units and the Q computing units are mapped to multiple third-level computing nodes. Specifically, as shown... Figure 6A As shown, Figure 6A This is a flowchart of a resource mapping method provided in an embodiment of the present invention. Figure 6A Taking the fourth-level computing node "engine" as an example, this section describes how to map "engine" to the third-level computing node "PE". Figure 6A As shown, the method may include the following steps.

[0082] In step 6082, the P computing units and the Q computing units are divided into m groups. Each group includes P / m computing units for executing the first neural network layer and Q / m computing units for executing the second neural network layer. Here, m is an integer not less than 2, and the values ​​of P / m and Q / m are both integers. Specifically, taking the P computing units as the computing units executing the (i-1)th layer and the Q computing units as the computing units executing the (i-1)th layer as an example... Figure 7 As shown, taking the (i-1)th layer requiring 8 computational units (P=8), the ith layer requiring 4 computational units (Q=4), and the (i+1)th layer requiring 4 computational units, divided into 2 groups (m=2), as an example, we can obtain the following... Figure 7The two groups shown are as follows: Group 1 includes 4 computational units in layer (i-1), 2 computational units in layer 1, and 2 computational units in layer (i+1). Similarly, Group 2 includes 4 computational units in layer (i-1), 2 computational units in layer 1, and 2 computational units in layer (i+1).

[0083] In step 6084, the computing units of each group are mapped to the third-level computing nodes according to the number of computing units contained in each third-level computing node. During the mapping process, efforts are made to map computing units performing operations on adjacent neural network layers to the same third-level node. For example... Figure 7 As shown, assuming the neural network system, each first-level computing node chip includes 8 second-level computing nodes tiles, each tile includes 2 third-level computing nodes PE, and each PE includes 4 engines. Then, for the first group, the 4 engines of the (i-1)th layer can be mapped to one third-level computing node PE (e.g., ...). Figure 7 On PE1, the two engines of layer i and the two engines of layer i+1 are mapped together to a third-level computing node PE (e.g., Figure 7 On PE2). Similarly, following the mapping method for the computing units in the first group, for the computing units in the second group, the four engines of layer i-1 can be mapped to PE3, and the two engines of layer i and the two engines of layer i+1 can be mapped together to PE4. In practical applications, after completing the mapping of the computing units in the first group, the computing units of other groups can be mapped in a mirror manner, following the mapping method of the first group.

[0084] Based on this mapping method, it is possible to maximize the execution of adjacent neural network layers (e.g., Figure 7 The computational units of layer i and layer i+1 are mapped to the same third-level computational node. Therefore, when the output data of layer i is sent to the computational unit of layer i+1, it only needs to be transmitted within the same third-level node (PE), without occupying the bandwidth between the three levels, thus improving data transmission speed and reducing the bandwidth consumption between nodes.

[0085] Back Figure 6 In step 610, based on the number of tertiary computing nodes included in the secondary computing nodes of the neural network system, the multiple tertiary computing nodes mapped by the P computing units and the Q computing units are mapped to multiple secondary computing nodes. In step 612, based on the number of secondary computing nodes included in each neural network chip, the multiple secondary computing nodes mapped by the P computing units and the Q computing units are mapped to multiple neural network chips. As described above, Figure 6AThis is illustrated by mapping the engine performing the i-th level operation to the third-level computing node. Similarly, according to... Figure 6A The method shown can also map third-level nodes to second-level nodes, and second-level nodes to first-level nodes. For example, as... Figure 7 As shown, for the first group, PE1, which performs operations at layer i-1, and PE2, which performs operations at layers i and i+1, can be further mapped to the same second-level computing node, Tile1. For the second group, PE3, which performs operations at layer i-1, and PE4, which performs operations at layers i and i+1, can be further mapped to the same second-level computing node, Tile2. Furthermore, Tile 1 and Tile 2, which perform operations at layers i-1, i, and i+1, can also be mapped to the same chip, chip1. In this way, the mapping relationship from the first-level computing node chip to the fourth-level computing node engine in the neural network system can be obtained.

[0086] In step 614, the N first weights and the M second weights are deployed to P computing units and Q computing units corresponding to the plurality of third-level nodes, the plurality of second-level computing nodes, and the plurality of first-level computing nodes, respectively. In this embodiment of the invention, according to... Figure 6A and Figure 7 The method described can obtain the mapping relationship from the first-level computing node (chip) to the fourth-level computing node (engine) in a neural network system. For example, it can obtain the mapping relationships between the P computing units and the Q computing units, respectively, and the plurality of third-level nodes, the plurality of second-level computing nodes, and the plurality of first-level computing nodes. Furthermore, in this step, the weights of the corresponding neural network layers can be deployed to the computing units of each level of computing node according to the obtained mapping relationships. For example, as... Figure 7 As shown in Figure A, the N weights of the (i-1)th layer can be deployed to the four computing units corresponding to chip1, tile1, and PE1, and the four computing units corresponding to chip1, tile2, and PE3, respectively. The M second weights of the i-th layer can be deployed to the two computing units corresponding to chip1, tile1, and PE2, and the two computing units corresponding to chip1, tile2, and PE4, respectively. Alternatively, the N weights of the (i-1)th layer can be deployed to the four computing units (engine) in chip1->tile1->PE1 and the four computing units in chip1->tile2->PE3, respectively. The M weights of the i-th layer can be deployed to the two computing units in chip1->tile1->PE2 and the two computing units in chip1->tile2->PE4, respectively.

[0087] This deployment method not only ensures that the computing capabilities of the computing units supporting adjacent neural network layer operations in the neural network system described in this embodiment of the invention are matched, but also allows as many computing units performing adjacent neural network layer operations as possible to be located in the same third-level computing node, as many third-level computing nodes performing adjacent neural network layers as possible to be located in the same second-level computing node, and as many second-level computing nodes performing adjacent neural network layers as possible to be located in the same first-level computing node (e.g., a neural network chip). This reduces the amount of data transmitted between computing nodes and improves the speed of data transmission between different neural network layers.

[0088] It should be noted that this embodiment of the invention describes the allocation process of computing resources for performing operations on different neural network layers in a neural network system containing four levels of computing nodes, using the fourth-level computing node (engine) as the computing unit. Alternatively, the above embodiment divides the set of operations for performing different neural network layers into sets using the engine as the granularity. In practical applications, allocation can also be performed using the third-level computing node (PE) as the computing unit. In this case, the mapping between the third-level computing node (PE), the second-level computing node (tile), and the first-level computing node (chip) can be established according to the above method. Of course, when the amount of data to be computed is very large, allocation can also be performed using the second-level computing node (tile) as the granularity. Alternatively, in this embodiment of the invention, the computing unit can be an engine, PE, tile, or chip, and is not limited thereto.

[0089] The foregoing has provided a detailed description of how the neural network system provided in the embodiments of the present invention configures computing resources. The following will further describe the neural network system from the perspective of data processing. Figure 8 This is a flowchart illustrating a data processing method provided in an embodiment of the present invention. The method is applied to... Figure 1 In the neural network system shown, Figure 1 The neural network system shown uses Figure 5-7 The method shown configures and allocates computational resources for performing operations on different neural network layers. For example... Figure 8 As shown, this method can be derived from... Figure 1 The method can be implemented using the neural network circuit shown, and may include the following steps.

[0090] In step 802, the P computing units in the neural network system receive first input data. These P computing units are used to execute the first neural network layer operation of the neural network system. In this embodiment, the first neural network layer can be any layer in the neural network system. The first input data is the data for which the first neural network layer operation needs to be executed. When the first neural network layer is... Figure 3When the first neural network layer 302 is the first layer of the neural network system shown, the first input data can be the data initially input into the neural network system. When the first neural network layer is not the first layer of the neural network system, the first input data can be the output data processed by other neural network layers.

[0091] In step 804, the P computing units perform calculations on the first input data according to the configured N first weights to obtain the first output data. In this embodiment of the invention, the first weight is a weight matrix. The N first weights refer to N weight matrices, and the N first weights can also be referred to as N copies of the first weights. The N first weights can be configured according to... Figure 5-7 The method shown is configured in the P computational units. Specifically, elements of the first weights are respectively configured into the ReRAM cells of the crossbars included in the P computational units, thereby enabling the crossbars in the P computational units to perform parallel computations on the input data based on the N first weights, fully utilizing the computational power of the crossbars in the P computational units. In this embodiment of the invention, after receiving the first input data, the P computational units can perform neural network operations on the received first input data based on the configured N first weights to obtain the first output data. For example, the crossbars in the P computational units can perform matrix multiplication and addition operations on the first input data and the configured first weights.

[0092] In step 806, the Q computational units in the neural network system receive second input data. These Q computational units are used to execute the second neural network layer operation of the neural network system, and the second input data includes the first output data. Specifically, in one case, the Q computational units may perform the second neural network layer operation only on the first output data of the P computational units. For example, the P computational units are used to execute... Figure 3 The operation of the first layer 302 shown is performed by the Q computing units. Figure 3 The operation of the second layer 302 is shown. In this case, the second input data is the first output data. In another case, the Q computing units can also be used to jointly perform a second neural network operation on the first output data of the first neural network layer and the output data of other neural network layers. For example, the P computing units can be used to perform... Figure 3 The second layer 304 of the neural network shown can be used to perform the Q computational units. Figure 3The neural network operation of the fifth layer 310 is shown. In this case, the Q computing units are used to perform operations on the output data of the second layer 304 and the fourth layer 308, and the second input data includes the first output data and the output data of the fourth layer 308.

[0093] In step 808, the Q computing units perform calculations on the second input data according to the configured M second weights to obtain the second output data. In this embodiment of the invention, the second weight is also a weight matrix. The M second weights refer to M weight matrices, and the M second weights can also be referred to as M copies of the second weights. Similar to step 804, the second weights can be calculated according to... Figure 5 The method shown is configured in the ReRAM cell of the crossbar included in the Q computing units. After receiving the second input data, the Q computing units can perform neural network operations on the received second input data based on the configured M second weights to obtain the second output data. For example, the crossbar in the Q computing units can perform matrix multiplication and addition operations on the second input data and the configured second weights. It should be noted that, in this embodiment of the invention, the ratio of N and M corresponds to the ratio of the amount of the first output data to the amount of the second output data.

[0094] To illustrate this clearly, the following is a simple description of how the ReRAM crossbar implements matrix multiplication and addition operations. For example... Figure 9 As shown, Figure 9 The j-row, k-column weight matrix shown can be a weight matrix for a neural network layer, where each element represents a weight value. Figure 10 This is a schematic diagram of the structure of a ReRAM crossbar in the computing unit provided in an embodiment of the present invention. For ease of description, the ReRAM crossbar in this embodiment of the present invention can be simply referred to as a crossbar. Figure 10 As shown, the crossbar includes multiple ReRAM cells, such as G. 1,1 G 2,1 The plurality of ReRAM cells constitute a neural network matrix. In this embodiment of the invention, during the configuration of the neural network, the following can be configured: Figure 9 The weights shown are from Figure 10 The bit lines of the crossbar shown (e.g.) Figure 10 (As shown in input port 1002) The input is placed into the crossbar, so that each element in the weights is assigned to the corresponding ReRAM cell. For example, Figure 9 The weight element W in 0,0 Configured to Figure 10 G 1,1 middle, Figure 9 The weight element W in 1,0 Configured to Figure 10 G 2,1 Medium. Each weight element corresponds to a ReRAM cell. During neural network computation, the input data passes through the word lines of the crossbar (e.g., ...). Figure 10 The input port 1004 shown is used to input the crossbar. It can be understood that the input data can be represented by voltage, allowing the input data to be multiplied by the weight values ​​configured in the ReRAM cell. The calculated result is output as voltage from the output terminal of each column of the crossbar (e.g., ...). Figure 10 The output port 1006 shown is the output port.

[0095] As mentioned earlier, when configuring the computing units that execute each layer of the neural network system, the amount of data output from adjacent neural network layers is considered, ensuring that the computing capabilities of the computing nodes executing operations on adjacent neural network layers are matched. Therefore, the data processing method provided by this invention can fully utilize the computing capabilities of computing nodes and improve the data processing efficiency of the neural network system.

[0096] In yet another embodiment of the invention, a resource allocation device is provided. This device can be applied to… Figure 1 and Figure 1A In the neural network system shown, the allocation of computing nodes to execute operations of different neural network layers ensures that the computing capabilities of nodes executing operations of adjacent neural network layers are matched, improving data processing efficiency and preventing waste of computing resources. It is understood that the resource allocation device can be located in the host, implemented by the processor in the host, or exist as a physical device independent of the processor. For example, it can function as a processor-independent compiler. Figure 11 As shown, the resource allocation device 1100 may include an acquisition module 1102, a calculation module 1104, and a deployment module 1106.

[0097] The acquisition module 1102 is used to acquire the data volume of the first output data of the first neural network layer and the data volume of the second output data of the second neural network layer in the neural network system. The input data of the second neural network layer includes the first output data. The calculation module 1104 is used to determine the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer according to the deployment requirements of the neural network system. Here, N and M are both positive integers, and the ratio of N to M corresponds to the ratio of the data volume of the first output data to the data volume of the second output data.

[0098] As previously described, the neural network system of this embodiment includes multiple neural network chips, each neural network chip includes multiple computing units, and each computing unit includes at least one Resistive Random Access Memory (ReRAM) crossbar. In one scenario, the deployment requirement includes computational latency. When the first neural network layer is the starting layer for all neural network layers in the neural network system, the computing module is used to determine the value of N based on the amount of the first output data, the computational latency, and the computation frequency of the ReRAM crossbar in the computing unit, and to determine the value of M based on the ratio of the amount of the first output data to the amount of the second output data and the value of N.

[0099] In another scenario, the deployment requirements include the number of chips in the neural network system, where the first neural network layer is the starting layer of the neural network system. The computing module is used to determine the value of N based on the number of chips, the number of ReRAM crossbars in each chip, the number of ReRAM crossbars required to deploy one weight of each neural network layer, and the ratio of the output data volume of adjacent neural network layers. The module also determines the value of M based on the ratio of the data volume of the first output data to the data volume of the second output data and the value of N.

[0100] Deployment module 1106 is used to deploy N first weights to P computing units and M second weights to Q computing units according to the computing specifications of the computing units in the neural network system, where P and Q are positive integers. The P computing units are used to perform the operations of the first neural network layer, and the Q computing units are used to perform the operations of the second neural network layer. The computing specifications of a computing unit refer to the number of crossbars included in a computing unit. In practical applications, a computing unit may include one or more crossbars. Specifically, after computing module 1104 obtains the number of weights required for each layer of the neural network, deployment module 1106 can deploy the weights of each layer to the corresponding computing units according to the number of crossbars included in each computing unit. Specifically, the elements in the weight matrix are respectively configured in the ReRAM cells of the crossbars of the computing units. In this embodiment of the invention, a computing unit may refer to a PE or an engine. A PE may include multiple engines, and an engine may include one or more crossbars. Since the weights of each layer may be different, a weight can be deployed on one or more engines.

[0101] As mentioned above, Figure 1 The illustrated neural network system includes multiple neural network chips, each neural network chip includes multiple secondary computing nodes, and each secondary computing node includes multiple computing units. To further reduce the amount of data transmission between computing units executing different neural network layers and save transmission bandwidth between computing units or computing nodes, the resource allocation device 1100 may further include a mapping module 1108, used to map computing units to their parent computing nodes. Specifically, after the computing module 1104 obtains the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer, the mapping module 1108 is used to establish a mapping relationship between the N first weights and the P computing units, and to establish a mapping relationship between the M second weights and the Q computing units. Further, the mapping module 1108 is also used to map the P computing units and the Q computing units to multiple secondary computing nodes according to the number of computing units contained in the secondary computing nodes of the neural network system, wherein at least a portion of the P computing units and at least a portion of the Q computing units are mapped to the same secondary computing node.

[0102] Furthermore, the mapping module 1108 is also configured to map the plurality of secondary computing nodes mapped by the P computing units and the Q computing units to the plurality of neural network chips according to the number of secondary computing nodes contained in each neural network chip. Specifically, at least a portion of the secondary computing nodes belonging to the P computing units and at least a portion of the secondary computing nodes belonging to the Q computing units are mapped to the same neural network chip.

[0103] In this embodiment of the invention, how the mapping module 1108 establishes the mapping relationship between N first weights and the P computing units, establishes the mapping relationship between the M second weights and the Q computing units, and how it maps the P computing units and the Q computing units to the parent computing nodes of the computing units respectively, can be found in the foregoing description of... Figure 6 , Figure 6A and Figure 7 The corresponding description will not be repeated here.

[0104] This invention also provides a computer program product for implementing the above-described resource allocation method, and a computer program product for implementing the above-described data processing method. All of these computer program products include a computer-readable storage medium storing program code. The instructions included in the program code are used to execute the method flow described in any of the foregoing method embodiments. Those skilled in the art will understand that the aforementioned storage medium includes various non-transitory machine-readable media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, optical disks, random-access memory (RAM), solid-state drives (SSDs), or non-volatile memory.

[0105] It should be noted that the embodiments provided in this application are merely illustrative. Those skilled in the art will understand that, for the sake of convenience and brevity, the descriptions of each embodiment have different focuses, and parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The features disclosed in the embodiments, claims, and drawings of this invention can exist independently or in combination. Features described in hardware form in the embodiments of this invention can be executed by software, and vice versa. No limitations are imposed here.

Claims

1. A method for allocating computing resources in a neural network system, characterized in that, include: The data volume of the first output data of the first neural network layer and the data volume of the second output data of the second neural network layer in the neural network system are obtained, wherein the input data of the second neural network layer includes the first output data. Based on the deployment requirements of the neural network system, determine the N first weights to be configured in the first neural network layer and the M second weights to be configured in the second neural network layer, where N and M are both positive integers, and the ratio of N to M corresponds to the ratio of the amount of data in the first output data to the amount of data in the second output data. According to the computational specifications of the computational units in the neural network system, N first weights are deployed to P computational units, and M second weights are deployed to Q computational units, where P and Q are both positive integers. The P computational units are used to perform the operations of the first neural network layer, and the Q computational units are used to perform the operations of the second neural network layer.

2. The method according to claim 1, characterized in that, The deployment requirements include computational latency, and the first neural network layer is the starting layer for all neural network layers in the neural network system. The process of determining the N first weights to be configured in the first neural network layer and the M second weights to be configured in the second neural network layer includes: The value of N is determined based on the amount of data in the first output data, the computation delay, and the computation frequency of the Resistive Random Access Memory (ReRAM) crossbar in the computation unit. The value of M is determined based on the ratio of the amount of data in the first output data to the amount of data in the second output data, and the value of N.

3. The method according to claim 1, characterized in that: The neural network system includes multiple neural network chips, each neural network chip includes multiple computing units, and each computing unit includes at least one resistive random access memory (ReRAM) crossbar. The deployment requirements include the number of chips in the neural network system, and the first neural network layer is the starting layer of the neural network system. The process of determining the N first weights to be configured in the first neural network layer and the M second weights to be configured in the second neural network layer includes: The value of N is determined based on the number of chips, the number of ReRAM crossbars in each chip, the number of ReRAM crossbars required to deploy one weight of each neural network layer, and the ratio of the output data volume of adjacent neural network layers. The value of M is determined based on the ratio of the amount of data in the first output data to the amount of data in the second output data, and the value of N.

4. The method according to claim 1, characterized in that, The neural network system includes multiple neural network chips, each neural network chip includes multiple secondary computing nodes, and each secondary computing node includes multiple computing units. The method further includes: Based on the number of computing units contained in the secondary computing nodes of the neural network system, the P computing units and the Q computing units are mapped to multiple secondary computing nodes, wherein at least a portion of the P computing units and at least a portion of the Q computing units are mapped to the same secondary computing node.

5. The method according to claim 4, characterized in that, Also includes: Based on the number of secondary computing nodes contained in each neural network chip, the P computing units and the multiple secondary computing nodes mapped by the Q computing units are mapped to the multiple neural network chips, wherein at least a portion of the secondary computing nodes belonging to the P computing units and at least a portion of the secondary computing nodes belonging to the Q computing units are mapped to the same neural network chip.

6. A neural network system, characterized in that, include: Multiple neural network chips, each of which includes multiple computing units; The processor, connected to the plurality of neural network chips, is used for: The data volume of the first output data of the first neural network layer and the data volume of the second output data of the second neural network layer in the neural network system are obtained, wherein the input data of the second neural network layer includes the first output data. Based on the deployment requirements of the neural network system, determine the N first weights to be configured in the first neural network layer and the M second weights to be configured in the second neural network layer, where N and M are both positive integers, and the ratio of N to M corresponds to the ratio of the amount of data in the first output data to the amount of data in the second output data. According to the computational specifications of the computational units in the neural network system, N first weights are deployed to P computational units among the plurality of computational units, and M second weights are deployed to Q computational units among the plurality of computational units, where P and Q are both positive integers. The P computational units are used to perform the operations of the first neural network layer, and the Q computational units are used to perform the operations of the second neural network layer.

7. The neural network system according to claim 6, characterized in that, The deployment requirements include computational latency, and the first neural network layer is the starting layer for all neural network layers in the neural network system. In the step of determining the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer, the processor is configured to: The value of N is determined based on the amount of data in the first output data, the computation delay, and the computation frequency of the Resistive Random Access Memory (ReRAM) crossbar in the computation unit. The value of M is determined based on the ratio of the amount of data in the first output data to the amount of data in the second output data, and the value of N.

8. The neural network system according to claim 6, characterized in that: Each computing unit includes at least one resistive random access memory (ReRAM) crossbar. The deployment requirements include the number of chips in the neural network system, where the first neural network layer is the starting layer of the neural network system. In the step of determining the N first weights to be configured for the first neural network layer and the M second weights to be configured for the second neural network layer, the processor is configured to: The value of N is determined based on the number of chips, the number of ReRAM crossbars in each chip, the number of ReRAM crossbars required to deploy one weight of each neural network layer, and the ratio of the output data volume of adjacent neural network layers. The value of M is determined based on the ratio of the amount of data in the first output data to the amount of data in the second output data, and the value of N.

9. The neural network system according to claim 6, characterized in that: The neural network system includes multiple neural network chips, each neural network chip includes multiple secondary computing nodes, and each secondary computing node includes multiple computing units. The processor is further used for: Based on the number of computing units contained in the secondary computing nodes of the neural network system, the P computing units and the Q computing units are mapped to multiple secondary computing nodes, wherein at least a portion of the P computing units and at least a portion of the Q computing units are mapped to the same secondary computing node.

10. The neural network system according to claim 9, characterized in that, The processor is also used for: Based on the number of secondary computing nodes contained in each neural network chip, the P computing units and the multiple secondary computing nodes mapped by the Q computing units are mapped to the multiple neural network chips, wherein at least a portion of the secondary computing nodes belonging to the P computing units and at least a portion of the secondary computing nodes belonging to the Q computing units are mapped to the same neural network chip.

11. A resource allocation device, characterized in that, include: The acquisition module is used to acquire the data volume of the first output data of the first neural network layer and the data volume of the second output data of the second neural network layer in the neural network system, wherein the input data of the second neural network layer includes the first output data; The calculation module is used to determine the N first weights to be configured in the first neural network layer and the M second weights to be configured in the second neural network layer according to the deployment requirements of the neural network system, wherein N and M are both positive integers, and the ratio of N to M corresponds to the ratio of the amount of data of the first output data to the amount of data of the second output data. The deployment module is used to deploy N first weights to P computing units and M second weights to Q computing units according to the computing specifications of the computing units in the neural network system, where P and Q are both positive integers. The P computing units are used to perform the operations of the first neural network layer and the Q computing units are used to perform the operations of the second neural network layer.

12. The resource allocation device according to claim 11, characterized in that, The deployment requirements include computational latency, the first neural network layer is the starting layer for all neural network layers in the neural network system, and the computation module is used for: The value of N is determined based on the amount of data in the first output data, the computation delay, and the computation frequency of the Resistive Random Access Memory (ReRAM) crossbar in the computation unit. The value of M is determined based on the ratio of the amount of data in the first output data to the amount of data in the second output data, and the value of N.

13. The resource allocation device according to claim 11, characterized in that, The neural network system includes multiple neural network chips, each neural network chip includes multiple computing units, and each computing unit includes at least one resistive random access memory (ReRAM) crossbar. The deployment requirements include the number of chips in the neural network system. The first neural network layer is the starting layer of the neural network system. The computing module is used for: The value of N is determined based on the number of chips, the number of ReRAM crossbars in each chip, the number of ReRAM crossbars required to deploy one weight of each neural network layer, and the ratio of the output data volume of adjacent neural network layers. The value of M is determined based on the ratio of the amount of data in the first output data to the amount of data in the second output data, and the value of N.

14. The resource allocation device according to claim 11, characterized in that, The neural network system includes multiple neural network chips, each neural network chip includes multiple secondary computing nodes, each secondary computing node includes multiple computing units, and the device further includes: The mapping module is used to map the P computing units and the Q computing units to multiple secondary computing nodes according to the number of computing units contained in the secondary computing nodes of the neural network system, wherein at least a portion of the P computing units and at least a portion of the Q computing units are mapped to the same secondary computing node.

15. The resource allocation device according to claim 14, characterized in that, The mapping module is also used for: Based on the number of secondary computing nodes contained in each neural network chip, the P computing units and the multiple secondary computing nodes mapped by the Q computing units are mapped to the multiple neural network chips, wherein at least a portion of the secondary computing nodes belonging to the P computing units and at least a portion of the secondary computing nodes belonging to the Q computing units are mapped to the same neural network chip.

16. A computer program product comprising program code, wherein the instructions included in the program code are executed by a computer to perform the computing resource allocation method as described in any one of claims 1-5.

17. A computer-readable storage medium comprising computer program instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-5.

Images