A processing device and related products

Abstract

The present application provides a processing device and related products, the processing device including an acquisition unit and a device allocation unit. The acquisition unit is used to obtain multiple nodes of a neural network model and device information of each of the multiple nodes. The device information of a node is information about the device that supports the operation corresponding to the node. The device information of the node includes one or more of the following: the type of device, the computing power of the device, and the copying capability of the device. The device allocation unit is used to allocate a corresponding execution device to each node based on the device information of each node. The use of the processing device provided by the present application can improve the performance of the neural network model during operation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI), and in particular to a processing device and related products. Background Art

[0002] In recent years, researchers have developed multiple deep learning frameworks, such as Tensorflow, MXNet, and Convolutional Architecture for Fast Feature Embedding (Caffe), to make it easier for people to learn, build, and train different types of neural networks.

[0003] Currently, when running neural network models based on deep learning frameworks, each node is typically assigned a specific device based on its registration status. However, when a node can run on multiple devices, this approach doesn't consider which device performs best, and therefore can lead to increased operating costs. Summary of the Invention

[0004] This application discloses a processing device and related products that can improve the performance of a neural network model during operation.

[0005] In a first aspect, the present application provides a processing device, the processing device comprising:

[0006] an acquiring unit, configured to acquire device information of a plurality of nodes of a neural network model and of each of the plurality of nodes, wherein the device information of a node is information of a device that supports an operation corresponding to the node, and the device information of the node includes one or more of the following: a type of device, a computing capability of the device, and a copy capability of the device;

[0007] The device allocation unit is used to allocate a corresponding execution device to each node according to the device information of each node.

[0008] The processing device provided in the present application can assign a suitable execution device to each node through the device information of each node in the neural network model. Compared with specifying the running device for each node according to the registration status of each node on the device, the use of this processing device can enable each node to be assigned to a more suitable execution device for calculation, thereby improving the overall calculation efficiency of the neural network model, reducing the calculation cost, and making the neural network model perform better during operation.

[0009] In one possible implementation, multiple nodes include a first node, and the device allocation unit is specifically used to: when there are multiple devices supporting the operation corresponding to the first node, allocate a corresponding execution device to the first node based on information of the multiple devices and the scale of the operation corresponding to the first node; wherein the scale of the operation corresponding to the first node depends on one or more of the following: the shape of the first node and the type of data input into the first node.

[0010] In one possible implementation, the multiple devices include a first device and a second device, the computing power of the first device is higher than the computing power of the second device, and the device allocation unit is used to: when the scale of the operation corresponding to the first node is greater than a preset scale, select the first device as the execution device of the first node; when the scale of the operation corresponding to the first node is less than or equal to the preset scale, select the second device as the execution device of the first node.

[0011] It can be seen that the processing device can allocate a suitable execution device to the node through the computing scale corresponding to the node and the device information of the node. For example, when the computing scale corresponding to the node is large, using the first device with stronger computing power to run it will shorten the computing time. When the computing scale corresponding to the node is small, using the second device to run it may get better computing performance. For example, before running the node on the first device, the data input to the node needs to be copied from the second device to the first device. This data copying process will waste time, resulting in the time taken to complete the computing of the node on the first device being longer than the time taken to complete the computing of the node on the second device. For another example, the time taken to complete the computing of the node on the first device and the second device is close. In this case, it is feasible to use either the first device or the second device.

[0012] In one possible implementation, the multiple nodes include multiple consecutive nodes, and the device allocation unit is specifically used to: when there are multiple devices supporting operations corresponding to the multiple consecutive nodes, allocate corresponding execution devices to the multiple consecutive nodes based on the information of the multiple devices and the number of the multiple consecutive nodes.

[0013] In one possible implementation, the multiple devices include a third device and a fourth device, the computing power of the third device is higher than that of the fourth device, and the data when the third device performs the operation is copied to the third device by the fourth device. The device allocation unit is used to: when the number of multiple consecutive nodes is greater than or equal to a preset number, select the third device as the execution device of the multiple consecutive nodes; when the number of multiple consecutive nodes is less than the preset number, select the fourth device as the execution device of the multiple consecutive nodes.

[0014] It can be seen that for multiple consecutive nodes, although data needs to be copied from the fourth device to the third device before the multiple consecutive nodes are assigned to the third device for operation, when the number of multiple consecutive nodes is large, a device with more powerful computing power is required to run in order to complete the operation in a shorter time. At this time, the time spent copying data from the fourth device to the third device has a smaller impact. Therefore, it is more appropriate to select the third device as the execution device for multiple consecutive nodes. On the contrary, when the number of multiple consecutive nodes is small, the time spent copying data from the fourth device to the third device has a greater impact. Therefore, it is more appropriate to select the third device as the execution device for multiple consecutive nodes. It can also be understood that assigning multiple consecutive nodes to the same device for calculation can greatly reduce the number of times data is copied between different devices, thereby saving computing costs and improving computing efficiency. Therefore, through the above implementation method, the device allocation unit can improve the computing efficiency of multiple consecutive nodes, thereby improving the performance of the neural network model during operation.

[0015] In one possible implementation, the acquisition unit is also used to obtain the connection relationship between multiple nodes; the device allocation unit is also used to allocate each node to the corresponding computing library for operation based on the information of the execution device corresponding to each node and the connection relationship between multiple nodes.

[0016] In one possible implementation, the device allocation unit is specifically used to: obtain at least one node with a fifth device as an execution device, wherein the information of the fifth device includes that a first computing library and a second computing library are configured in the fifth device, and the speed of running consecutive nodes on the first computing library is greater than the speed of running consecutive nodes on the second computing library, and at least one node includes the second node; determine whether there is a node connected to the second node in at least one node; if there is no node connected to the second node, allocate the second node to the second computing library for operation; if there is a node connected to the second node, when the first computing library includes computing instructions corresponding to the second node and the node connected to the second node, allocate the second node and the node connected to the second node to the first computing library for operation; when the first computing library does not include computing instructions corresponding to the second node and the node connected to the second node, allocate the second node and the node connected to the second node to the second computing library for operation.

[0017] It can be seen that through the above implementation, the device allocation unit can shorten the calculation time of multiple consecutive nodes, thereby improving the performance of the neural network model during operation.

[0018] In a second aspect, the present application provides a combined computing device, which includes: a processing device, an interface device, a storage device, and one or more computing devices, wherein:

[0019] a processing device for assigning nodes in the neural network model to corresponding computing devices;

[0020] A computing device for performing operations corresponding to nodes in the neural network model;

[0021] When the combined computing device includes multiple computing devices, the multiple computing devices can be connected and transmit data through a specific structure;

[0022] Among them, multiple computing devices are interconnected and transmit data through a fast peripheral interconnect bus PCIE bus to support the operation of larger-scale neural networks; multiple computing devices share the same control system or have their own control systems; multiple computing devices share memory or have their own memory; the interconnection method of multiple computing devices is any interconnection topology;

[0023] The storage device is connected to the processing device and one or more computing devices respectively, and is used to store data of the processing device and one or more computing devices.

[0024] Among them, the processing device is the processing device described in the first aspect and any implementation method of the first aspect.

[0025] In a third aspect, the present application provides a neural network chip, which includes the combined computing device described in the second aspect above.

[0026] In a fourth aspect, the present application provides a neural network chip packaging structure, which includes the neural network chip described in the third aspect above;

[0027] In a fifth aspect, the present application provides a board card, which includes the neural network chip packaging structure described in the fourth aspect above.

[0028] In a sixth aspect, the present application provides an electronic device comprising the neural network chip as described in the fourth aspect or the board as described in the fifth aspect.

[0029] In a seventh aspect, the present application provides a method for allocating node devices, the method comprising:

[0030] Obtaining device information of multiple nodes of the neural network model and each of the multiple nodes, wherein the device information of the node is information of a device that supports operations corresponding to the node, and the device information of the node includes one or more of the following: device type, device computing capability, and device copy capability;

[0031] According to the device information of each node, the corresponding execution device is assigned to each node.

[0032] In one possible implementation, multiple nodes include a first node, and a corresponding execution device is assigned to each node based on the device information of each node, including: when there are multiple devices supporting the operation corresponding to the first node, the corresponding execution device is assigned to the first node based on the information of the multiple devices and the scale of the operation corresponding to the first node; wherein the scale of the operation corresponding to the first node depends on one or more of the following: the shape of the first node and the type of data input into the first node.

[0033] In one possible implementation, the multiple devices include a first device and a second device, the computing power of the first device is higher than the computing power of the second device, and the corresponding execution device is allocated to the first node based on the information of the multiple devices and the scale of the operation corresponding to the first node, including: when the scale of the operation corresponding to the first node is greater than a preset scale, the first device is selected as the execution device of the first node; when the scale of the operation corresponding to the first node is less than or equal to the preset scale, the second device is selected as the execution device of the first node.

[0034] In one possible implementation, the multiple nodes include multiple consecutive nodes, and a corresponding execution device is assigned to each node based on the device information of each node, including: when there are multiple devices supporting operations corresponding to multiple consecutive nodes, corresponding execution devices are assigned to the multiple consecutive nodes based on the information of the multiple devices and the number of the multiple consecutive nodes.

[0035] In one possible implementation, the multiple devices include a third device and a fourth device, the computing power of the third device is higher than that of the fourth device, and the data when the third device performs the operation is copied to the third device by the fourth device. According to the information of the multiple devices and the number of multiple consecutive nodes, corresponding execution devices are allocated to the multiple consecutive nodes, including: when the number of multiple consecutive nodes is greater than or equal to a preset number, selecting the first device as the execution device of the multiple consecutive nodes; when the number of multiple consecutive nodes is less than the preset number, selecting the second device as the execution device of the multiple consecutive nodes.

[0036] In one possible implementation, after assigning a corresponding execution device to each node based on the device information of each node, the above method also includes: obtaining the connection relationship between multiple nodes; and assigning each node to the corresponding computing library to run based on the information of the execution device corresponding to each node and the connection relationship between the multiple nodes.

[0037] In one possible implementation, each node is assigned to a corresponding computing library based on information of an execution device corresponding to each node and a connection relationship between multiple nodes, including: obtaining at least one node with a fifth device as an execution device, wherein the information of the fifth device includes that a first computing library and a second computing library are configured in the fifth device, and a speed of running consecutive nodes on the first computing library is greater than a speed of running consecutive nodes on the second computing library, and at least one node includes a second node; determining whether there is a node connected to the second node in at least one node; if there is no node connected to the second node, assigning the second node to the second computing library for operation; if there is a node connected to the second node, when the first computing library includes computing instructions corresponding to the second node and the node connected to the second node, assigning the second node and the node connected to the second node to the first computing library for operation; when the first computing library does not include computing instructions corresponding to the second node and the node connected to the second node, assigning the second node and the node connected to the second node to the second computing library for operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present application or the background technology.

[0039] Figure 1 This is a schematic diagram of the operation of a neural network model provided by this application;

[0040] Figure 2 It is a structural schematic diagram of a processing device provided by this application;

[0041] Figure 3 This is a flowchart of a node device allocation method provided by this application;

[0042] Figure 4 It is a structural diagram of a combined computing device provided by this application;

[0043] Figure 5 This is a structural diagram of a board provided in this application. DETAILED DESCRIPTION

[0044] The technical solutions in this application are described clearly and completely below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments in this application without inventive effort are also within the scope of protection of this invention.

[0045] In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, we first introduce the scenarios to which the embodiments of the present application are applicable: running a neural network model based on a deep learning framework.

[0046] A neural network is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks and performs distributed parallel information processing. This network consists of a large number of interconnected nodes (or neurons). By adjusting the connections between these nodes, it uses input data and weights to generate output data, simulating the human brain's information processing process to process information and generate pattern recognition results. In recent years, with the rapid development of neural network technology, especially deep learning within artificial neural networks, neural networks have become the mainstream technology for deep learning research tasks such as computer vision, speech recognition, and natural language processing. To make it easier to learn, build, and train different types of neural networks, researchers have developed several deep learning frameworks, such as TensorFlow, MXNet, and Caffe.

[0047] When running a neural network model based on a deep learning framework, the various nodes that make up the neural network model are usually assigned to corresponding devices (including: Central Processing Unit (CPU) or Graphics Processing Unit (GPU)) to run, thereby further realizing the operation of the neural network model.

[0048] Currently, deep learning frameworks determine which device a node should be assigned to based on its device registration status. However, extensive practice has shown that this method of assigning nodes to specific devices can increase the cost of running neural networks. The following two examples briefly illustrate the problems with this method:

[0049] For example, when a node (node ​​A) can run on both the CPU and the GPU, Tensorflow will prioritize node A to run on the GPU. At this time, a situation may arise: if the performance of node A on the CPU is better than that on the GPU, then assigning node A to run on the GPU will lead to problems such as slow computing speed and high power consumption.

[0050] For example, Figure 1As shown, the neural network model includes nodes A, B, C, and D. Nodes A and C can run on both the CPU and the GPU, while nodes B and D can only run on the CPU. Therefore, Tensorflow assigns nodes A and C to run on the CPU, and assigns nodes B and D to run on the GPU. Furthermore, during the actual execution, each node is executed sequentially according to the connection relationship between these nodes, that is, node A → node B → node C → node D. Since nodes A and B, B and C, and C and D run on different devices, running these nodes requires switching back and forth between different devices (i.e., CPU → GPU → CPU → GPU). Frequently changing the running device causes data to be copied back and forth between the CPU and GPU, which incurs significant overhead.

[0051] In order to solve the above problems, the present application provides a processing device, which can be used to assign each node in the neural network model to a suitable execution device, thereby improving the performance of the neural network model during operation.

[0052] like Figure 2 As shown, Figure 2 It is a structural schematic diagram of a processing device provided in this application. Figure 2 The processing device 100 includes an acquisition unit 210 and a device allocation unit 220. Each unit in the processing device 100 will be described in detail below.

[0053] First, the acquisition unit 210 is introduced.

[0054] The acquisition unit 210 is used to acquire multiple nodes of the neural network model and device information of each of the multiple nodes.

[0055] In a specific embodiment, the nodes in the neural network model represent operators, such as a convolution operator or a batch normalization operator. The acquisition unit 210 can acquire multiple nodes of the neural network model in the following manner:

[0056] In a first approach, the acquisition unit 210 acquires a computational graph corresponding to the neural network model (hereinafter referred to as the computational graph), and then obtains a plurality of nodes of the neural network model according to the computational graph.

[0057] The computational graph is formed by loading a neural network model into a deep learning framework. It is a directed graph consisting of multiple nodes, connected by directed edges. These edges represent the flow of data between nodes. The computational graph can represent the computational process of the neural network model. Specifically, after the data input to the neural network model is passed to the computational graph, the neural network model's computation is completed by traversing the nodes in the computational graph according to the directed edges between the nodes, and then the neural network model's computational results are output.

[0058] Optionally, the calculation graph may be constructed by the acquisition unit 210 or by other devices or apparatuses, which is not specifically limited here.

[0059] In a second approach, the acquisition unit 210 may also directly acquire multiple nodes of the neural network model. Specifically, after other devices or apparatuses construct the computation graph, they send information about multiple nodes included in the computation graph to the acquisition unit 210, so that the acquisition unit 210 obtains the multiple nodes.

[0060] It is worth noting that the multiple nodes in this application can be all the nodes of the neural network model, or a part of all the nodes of the neural network model, which is not specifically limited here.

[0061] In a specific embodiment, a node device refers to a device that supports the operation corresponding to the node. Simply put, it refers to which device can execute the computing instructions corresponding to the node. The node device can specifically be a GPU, CPU, machine learning processing unit (MLU), digital signal processing unit (DSP), neural network processing unit (NPU), etc., without specific limitation here.

[0062] In a specific embodiment, the device information of a node is information about the device that supports the operation corresponding to the node, and the device information of the node includes one or more of the following: the type of device, the computing capability of the device (e.g., computing speed, precision, power consumption, etc.), and the copy capability of the device (e.g., copy speed, power consumption, etc.). The device type may be GPU, CPU, MLU, etc. The computing capability of the device refers to the device's ability to calculate data. Specifically, the faster the computing speed, the higher the precision, and the lower the power consumption of the device, the higher the computing capability of the device. The copy capability of the device refers to the ability to copy data from the device. Specifically, the faster the copy speed and the lower the power consumption, the higher the copy capability of the device.

[0063] In a specific embodiment, the acquiring unit 210 may acquire the device information of each of the multiple nodes in the following manner:

[0064] Method 1: After obtaining multiple nodes, the acquisition unit 210 searches for device information of each node based on the information of the multiple nodes. The device information of the node can be stored in the acquisition unit 210 or in other devices or apparatuses, which is not specifically limited here.

[0065] In a second approach, the node information and the node device information may be associated and stored, so that the acquisition unit can obtain the device information of each of the multiple nodes while obtaining multiple nodes.

[0066] Next, the device allocation unit 220 is introduced.

[0067] The device allocation unit 220 is configured to allocate a corresponding execution device to each node according to the device information of each node.

[0068] In this application, to facilitate device allocation unit 220 in allocating nodes in the neural network model to appropriate execution devices, three strategies are configured in device allocation unit 220: a first strategy, a second strategy, and a third strategy. In actual application, device allocation unit 220 can determine which execution device a node should be allocated to based on these three strategies, thereby improving the performance of the neural network model during operation. Below, device allocation unit 220 will be described based on these three strategies.

[0069] (1) First Strategy

[0070] It should be understood that when multiple devices support the operations corresponding to a node, the cost of computing the node on different devices will vary due to the different computing capabilities of the devices. For example, compared to a CPU, a GPU has more computing units, so a GPU has greater computing power and the node runs faster on the GPU. Based on this, the present application provides a first strategy, which can be specifically expressed as: assigning an execution device to a node based on the scale of the operation corresponding to the node and the information of the device that supports the operation corresponding to the node.

[0071] In a specific embodiment, the scale of the operation corresponding to the node depends on one or more of the following: the shape of the node, the type of data input to the node. Among them, the shape of the node can be understood as the scale of the node. The larger the shape of the node, the larger the scale of the operation corresponding to the node. For example, for the convolution operator (i.e., the node), the more convolution kernels there are in the node, the larger the scale of the node, and the more convolution operations are required in the node, that is, the larger the scale of the operation corresponding to the node. The type of data input to the node can specifically include integer and floating-point types. Compared with integer, when the data of the input node is floating-point data, the scale of the operation corresponding to the node is larger. One point that needs to be explained here is that the nodes of the neural network model can have one or more inputs, which is not specifically limited here.

[0072] In a specific embodiment, the multiple nodes include a first node. The device allocation unit 220 allocates an execution device to the first node according to a first policy. Specifically, when multiple devices support the operation corresponding to the first node, the device allocation unit 220 allocates a corresponding execution device to the first node based on information about the multiple devices and the scale of the operation corresponding to the first node. The scale of the operation corresponding to the first node depends on one or more of the following: the shape of the first node and the type of data input to the first node.

[0073] In a more specific embodiment, the multiple devices supporting the operation corresponding to the first node include a first device and a second device, wherein the computing capability of the first device is greater than that of the second device. The device allocation unit 220 is specifically configured to: select the first device as the execution device for the first node when the scale of the operation corresponding to the first node is greater than a preset scale; and select the second device as the execution device for the first node when the scale of the operation corresponding to the first node is less than or equal to the preset scale.

[0074] In a specific embodiment, the preset scale is pre-set by the user based on the computing power of the first device and the second device. Optionally, the preset scale includes a preset node scale and a preset data type. Therefore, when the scale of the first node is greater than the preset node scale, and / or the type of data input to the first node is a preset data type, the device allocation unit 220 selects the first device as the execution device of the first node. When the scale of the first node is less than or equal to the preset node scale, and / or the type of data input to the first node is not a preset data type, the device allocation unit 220 selects the second device as the execution device of the first node. Optionally, the preset scale can also be a standard set by comprehensively considering the preset node scale and the preset data type, etc., which is not specifically limited here.

[0075] It is understandable that for nodes with larger computing scales, using a first device with stronger computing power to run the operation will shorten the computing time. For nodes with smaller computing scales, using a second device to run the operation may result in better computing performance. For example, when the first device needs to perform the operation, the data required is copied from other devices (including the second device) to the first device. In this case, the data copying process wastes time, resulting in the time taken to complete the operation of the node on the first device being longer than the time taken to complete the operation of the node on the second device. For another example, if the time taken to complete the operation of the node on the first device and the second device is close, then either the first device or the second device can be used.

[0076] For example, the first node is a convolution operator, the first device is an MLU, and the second device is a CPU. It should be noted here that the MLU supports parallel computing and the number of bits that can be calculated by each single core in the MLU is higher than that of the CPU. In general, the computing power of the MLU is higher than that of the CPU. Therefore, according to the first strategy, when the dimension of the convolution kernel in the first node is higher (that is, the scale of the first node is larger than the preset node scale), and / or, the data input to the first node is floating-point data (that is, the type of the data input to the first node is a preset data type), the device allocation unit 220 allocates the first node to the MLU.

[0077] It can be seen that the device allocation unit 220 can allocate the nodes of the neural network model to appropriate execution devices for operation according to the first strategy. For example, for a node with a smaller operation scale (node ​​1), the time it takes to run on the first device and the time it takes to run on the second device are roughly the same. However, if the node runs on the second device, the data input to the node needs to be copied from the first device to the second device before running, which makes it take longer to complete the operation of the node on the second device. For another example, for a node with a larger operation scale (node ​​2), although the data input to the node needs to be copied from the first device to the second device before running on the second device, the time it takes to run on the second device is much shorter than the time it takes to run on the first device, which makes it take even shorter to complete the operation of the node on the second device. For nodes 1 and 2, using the first strategy provided by the present application, the device allocation unit 220 can directly allocate node 1 to the first device for operation and node 2 to the second device for operation by determining that the operation scale corresponding to node 1 is smaller than the preset scale and the operation scale corresponding to node 2 is larger than the preset scale, thereby shortening the operation time of nodes 1 and 2.

[0078] (2) Second Strategy

[0079] Consider such a situation: the nodes of the neural network model include node A and node B, node A and node B are connected by a directed edge, and node A and node B can run on both the CPU and the GPU. If both node A and node B are assigned to the GPU to run, although the computing power of the GPU is higher than that of the CPU, before the GPU performs the calculation, it is necessary to obtain data from the CPU first. This process (i.e., the process of copying data from the CPU to the GPU) will waste a lot of time, which may cause the time spent on executing the operations corresponding to node A and node B on the GPU to be longer than the time spent on executing the operations corresponding to node A and node B on the CPU. Based on this point, the present application provides a second strategy, which can be specifically expressed as: assigning an execution device to each of the consecutive nodes based on the device information of each node in the consecutive nodes and the number of consecutive nodes.

[0080] In a specific embodiment, the multiple nodes include multiple consecutive nodes, and the device allocation unit 220 allocates a corresponding execution device to each of the multiple consecutive nodes according to the second strategy. The specific process is: when there are multiple devices supporting operations corresponding to the multiple consecutive nodes, corresponding execution devices are allocated to the multiple consecutive nodes according to the information of the multiple devices and the number of the multiple consecutive nodes.

[0081] In a more specific embodiment, the devices supporting the operations corresponding to the plurality of consecutive nodes include a third device and a fourth device, wherein the computing power of the third device is higher than that of the fourth device, and the data required for the third device to perform the operations is copied from other devices (including the fourth device) to the third device. The device allocation unit 220 is specifically configured to: when the number of the plurality of consecutive nodes is greater than or equal to a preset number, select the third device as the execution device for the plurality of consecutive nodes; and when the number of the plurality of consecutive nodes is less than a preset number, select the fourth device as the execution device for the plurality of consecutive nodes. The preset number is pre-set by the user based on the computing power and copying capabilities of the third and fourth devices.

[0082] In a specific embodiment, the third device may be the first device or the second device mentioned above, or may not be the first device or the second device mentioned above. Similarly, the fourth device may be the first device or the second device mentioned above, or may not be the first device or the second device mentioned above. No specific limitation is made here.

[0083] It is understandable that although data needs to be copied from the fourth device to the third device before multiple consecutive nodes are assigned to run on the third device, when the number of multiple consecutive nodes is large, a device with more powerful computing power is required to run in order to complete the calculation in a shorter time. At this time, compared to the time of performing the calculation on the fourth device, the time spent copying data from the fourth device to the third device has a smaller impact. Therefore, it is more appropriate to select the third device as the execution device for multiple consecutive nodes. Conversely, when the number of multiple consecutive nodes is small, the time spent copying data from the fourth device to the third device has a greater impact. Therefore, it is more appropriate to select the third device as the execution device for multiple consecutive nodes. It is also understandable that assigning multiple consecutive nodes to the same device for calculation can greatly reduce the number of times data is copied between different devices, thereby saving computing costs and improving computing efficiency. Therefore, the device allocation unit 220 can improve the computing efficiency of multiple consecutive nodes in the above manner.

[0084] (3) The third strategy

[0085] Currently, in order to improve the computing speed of continuous nodes, some devices are configured with a computing library specifically for running continuous nodes. For example, two computing libraries are configured in MLU: Cambricon Neuware Machine Learning Library (CNML) and Cambricon Neuware Neural Network Library (CNNL). Among them, compared with CNNL, CNML supports accelerated computing of continuous nodes, that is, the computing speed of running continuous nodes on CNML is greater than the computing speed of running continuous nodes on CNNL. Based on this, the present application provides a third strategy, which can be specifically expressed as: assigning nodes to corresponding computing libraries based on the connection relationship between nodes and the information of the execution device corresponding to the node. Among them, the information of the execution device includes the information of the computing library in the execution device. The connection relationship between nodes refers to whether there is a directed edge between the nodes. When there is a directed edge between two nodes, it means that the two nodes are two connected nodes. When there is no directed edge between the two nodes, it means that the two nodes are two unconnected nodes. The computing library includes computing instructions corresponding to the nodes. Generally, in the process of running a neural network model based on a deep learning framework, the deep learning framework loads the computing instructions corresponding to the node onto the execution device by calling the computing library, so that the execution device completes the operation corresponding to the node.

[0086] In a specific embodiment, the acquisition unit 210 is also used to obtain the connection relationship between multiple nodes; the device allocation unit 220 is also used to allocate each of the multiple nodes to the corresponding computing library for operation based on the information of the execution device corresponding to each of the multiple nodes and the connection relationship between the multiple nodes.

[0087] In a specific embodiment, the device allocation unit 220 is specifically configured to: obtain at least one node that uses a fifth device as an execution device, wherein the fifth device information includes that a first computation library and a second computation library are configured in the fifth device, and the at least one node includes the second node; determine whether there is a node connected to the second node in the at least one node; if there is no node connected to the second node, allocate the second node to the second computation library for execution; if there is a node connected to the second node, when the first computation library includes computation instructions corresponding to the second node and the nodes connected to the second node, allocate the second node and the nodes connected to the second node to the first computation library for execution; and if the first computation library does not include computation instructions corresponding to the second node and the nodes connected to the second node, allocate the second node and the nodes connected to the second node to the second computation library for execution. The speed of executing consecutive nodes on the first computation library is greater than the speed of executing consecutive nodes on the second computation library. It can be seen that the device allocation unit 220 can shorten the computation time of multiple consecutive nodes according to the third strategy, thereby improving the performance of the neural network model during execution.

[0088] In a specific embodiment, the fifth device may be the first device, the second device, the third device or the fourth device, or may be a device different from the first device, the second device, the third device or the fourth device, which is not specifically limited here.

[0089] For example, a neural network model includes node 1, node 2, node 3, and node 4, and the connection relationship between these four nodes is: node 1 → node 2 → node 3 → node 4. Assume that the nodes with the fifth device as the execution device are node 1, node 2, and node 4. Since node 1 is connected to node 2, and the first computing library includes computing instructions corresponding to node 1 and node 2, the device allocation unit 210 can allocate node 1 and node 2 to the first computing library for execution. Since node 4 is not connected to both nodes 1 and 2, and the second computing library includes computing instructions corresponding to node 4, the device allocation unit 210 can allocate node 4 to the second computing library for execution.

[0090] The above content introduces in detail the three strategies provided in this application. It should be understood that in actual applications, the device allocation unit 220 can also simultaneously use one or more of the first strategy, the second strategy and the third strategy to allocate corresponding execution devices to each node in the neural network model.

[0091] Optionally, before the device allocation unit 220 uses the third strategy to allocate the nodes of the neural network model to the corresponding computing library, it first uses the first strategy and / or the second strategy to determine the execution device corresponding to the node. For example, assume that the multiple nodes of the neural network model include node A, node B and node C, and the connection relationship between these three nodes is node A→node B→node C. Among them, the devices that support the operation corresponding to node A include MLU and CPU, and the devices that support the operation corresponding to node B and node C are all MLUs, and the CNML in the MLU includes the calculation instructions corresponding to node A, node B and node C. Then, the device allocation unit 220 can allocate node A to the MLU for operation according to the first strategy, and then allocate node A, node B and node C to the CNML for calculation according to the third strategy.

[0092] Based on the processing device provided in the above embodiment, the present application also provides a method for allocating node devices. Figure 3 As shown, Figure 3 A node device allocation method provided by the present application is shown, and the method specifically includes the following steps:

[0093] S301: The processing device obtains multiple nodes of the neural network model and device information of each of the multiple nodes.

[0094] In a specific embodiment, the device information of a node is information about a device that supports operations corresponding to the node, and the device information of the node includes one or more of the following: type of device, computing capability of the device, and copy capability of the device;

[0095] S302: The processing device allocates a corresponding execution device to each node according to the device information of each node.

[0096] In a specific embodiment, multiple nodes include a first node, and a corresponding execution device is assigned to each node based on the device information of each node, including: when there are multiple devices supporting the operation corresponding to the first node, the corresponding execution device is assigned to the first node based on the information of the multiple devices and the scale of the operation corresponding to the first node; wherein the scale of the operation corresponding to the first node depends on one or more of the following: the shape of the first node and the type of data input into the first node.

[0097] In a specific embodiment, the multiple devices include a first device and a second device, the computing power of the first device is higher than the computing power of the second device, and the corresponding execution device is allocated to the first node based on the information of the multiple devices and the scale of the operation corresponding to the first node, including: when the scale of the operation corresponding to the first node is greater than the preset scale, the first device is selected as the execution device of the first node; when the scale of the operation corresponding to the first node is less than or equal to the preset scale, the second device is selected as the execution device of the first node.

[0098] In a specific embodiment, the multiple nodes include multiple consecutive nodes, and a corresponding execution device is assigned to each node based on the device information of each node, including: when there are multiple devices supporting operations corresponding to multiple consecutive nodes, corresponding execution devices are assigned to the multiple consecutive nodes based on the information of the multiple devices and the number of multiple consecutive nodes.

[0099] In a specific embodiment, the multiple devices include a third device and a fourth device, the computing power of the third device is higher than that of the fourth device, and the data when the third device performs the operation is copied to the third device by the fourth device. According to the information of the multiple devices and the number of multiple consecutive nodes, corresponding execution devices are allocated to the multiple consecutive nodes, including: when the number of multiple consecutive nodes is greater than or equal to a preset number, selecting the first device as the execution device of the multiple consecutive nodes; when the number of multiple consecutive nodes is less than the preset number, selecting the second device as the execution device of the multiple consecutive nodes.

[0100] In a specific embodiment, after assigning a corresponding execution device to each node based on the device information of each node, the above method also includes: obtaining the connection relationship between multiple nodes; and assigning each node to the corresponding computing library to run based on the information of the execution device corresponding to each node and the connection relationship between multiple nodes.

[0101] In a specific embodiment, each node is assigned to a corresponding computing library based on information of an execution device corresponding to each node and a connection relationship between multiple nodes, including: obtaining at least one node with a fifth device as an execution device, wherein the information of the fifth device includes that a first computing library and a second computing library are configured in the fifth device, and at least one node includes the second node; judging whether there is a node connected to the second node in at least one node; in the case that there is no node connected to the second node, assigning the second node to the second computing library for operation; in the case that there is a node connected to the second node, when the first computing library includes computing instructions corresponding to the second node and the node connected to the second node, assigning the second node and the node connected to the second node to the first computing library for operation; when the first computing library does not include computing instructions corresponding to the second node and the node connected to the second node, assigning the second node and the node connected to the second node to the second computing library for operation.

[0102] The processing device provided by the above method embodiment and Figure 2 The processing device 200 shown belongs to the same concept and will not be described in detail here for the sake of simplicity.

[0103] Based on the processing device provided in the above embodiment, the present application also provides a combined computing device. Figure 4 As shown, Figure 4 A schematic structural diagram of a combined computing device provided in this application is shown. Figure 4 In the embodiment, the combined computing device 400 includes a processing device 410 , an interface device 420 , a storage device 430 , and a computing processing device 440 .

[0104] In a specific embodiment, according to different application scenarios, the computing processing device 440 may include one or more computing devices 450, which are used to execute operations corresponding to nodes in the neural network model, that is, the computing device in the embodiment of the present application is used to execute the operations of the execution device corresponding to the nodes in the above embodiment.

[0105] In various embodiments, the computing processing device 440 may be configured to perform user-specified operations. In exemplary applications, the computing processing device may be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing processing device may be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, the computing processing device of the present disclosure may be considered to have a single-core structure or a homogeneous multi-core structure.

[0106] In a specific embodiment, the processing device 410 is used to determine the computing device corresponding to each node in the neural network model and assign each node to the corresponding computing device. Specifically, the processing device 410 can be configured as follows: Figure 2 The processing device 200 shown or Figure 3 The processing device in is used to perform corresponding operations.

[0107] In an exemplary operation, the processing device 410 can interact with the computing processing device 440 through the interface device 420 to facilitate the processing device 410 to assign the nodes in the neural network model to the corresponding computing device. Depending on the implementation, the processing device 410 may include one or more types of processors in general and / or special processors such as CPU, GPU, artificial intelligence processor, etc. These processors may include but are not limited to DSP, application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing processing device of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0108] In one specific embodiment, processing device 410 may also serve as an interface between computing device 440 (which may be embodied as an artificial intelligence computing device such as a neural network computing device) and external data and control, performing basic control including but not limited to data transfer and starting and / or stopping the computing device. In another embodiment, processing device 410 may also collaborate with computing device 440 to jointly complete computing tasks.

[0109] In a specific embodiment, the interface device 420 can be used to transmit data and control instructions between the processing device 410 and the computing device 440. For example, the computing device 440 can obtain data from the processing device 410 via the interface device 420 and write the data to the storage device (or memory) on the computing device 440 chip. Furthermore, the computing device 440 can also obtain control instructions (e.g., instructions to execute a certain node) from the processing device 410 via the interface device 420 and write them to the control cache on the computing device 440 chip. Alternatively or optionally, the interface device 420 can also read data from the storage device of the computing device 440 and transmit it to the processing device 410.

[0110] Additionally or alternatively, the combined computing device 400 may further include a storage device 430. Figure 4 As shown, storage device 430 is connected to processing device 410 and computing device 440, respectively. In one or more embodiments, storage device 430 may be used to store data of processing device 410 and / or computing device 440. For example, the data may be data that cannot be fully stored in the internal or on-chip storage device of processing device 410 or computing device 440.

[0111] In some embodiments, the present application further provides a chip (eg, Figure 5 In one possible implementation, the chip 540 is a system on chip (SoC) and integrates one or more components such as Figure 4 The chip 540 can be connected to the external interface device (such as Figure 5 The external interface device 550 shown is connected to other related components. The related components may be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, the chip 540 may be integrated with other processing units (such as a video codec) and / or interface modules (such as a DRAM interface). In some embodiments, the present application further provides a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present application further provides a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Figure 5 The board is described in detail.

[0112] like Figure 5 As shown, Figure 5 This is a structural diagram of a board provided in this application. Figure 5 In the embodiment, the board 500 includes a storage device 510 for storing data, which includes one or more storage units 520. The storage device 510 can be connected and transmit data with the control device 530 and the chip 540 described above by means of, for example, a bus. Furthermore, the board 500 also includes an external interface device 550, which is configured to perform data relay or transfer functions between the chip 540 (or the chip in the chip packaging structure) and the external device 560 (such as a server or computer, etc.). For example, the data to be processed can be transmitted to the chip by the external device 560 through the external interface device 550. For another example, the calculation result of the chip 540 can be transmitted back to the external device 560 via the external interface device 550. Depending on different application scenarios, the external interface device 550 can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0113] In one or more embodiments, the control device 530 in the board 500 may be configured to control the state of the chip. To this end, in one application scenario, the control device 530 may include a microcontroller (MCU) to control the working state of the chip.

[0114] According to the above combination Figure 4 and Figure 5 Based on the description, those skilled in the art can understand that the present application also provides an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined computing devices.

[0115] According to different application scenarios, the electronic equipment or devices provided in this application may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, Internet of Things terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, vehicles, household appliances, and / or medical equipment. The vehicles include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, range hoods; the medical equipment includes magnetic resonance imaging machines, ultrasound machines and / or electrocardiographs. The electronic equipment or devices disclosed herein can also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical fields and so on. Furthermore, the electronic equipment or devices provided in this application can also be used in application scenarios related to artificial intelligence, big data and / or cloud computing, such as the cloud, edge, and terminal. In one or more embodiments, the electronic device or device with high computing power provided by the present application can be applied to cloud devices (such as cloud servers), while the electronic device or device with low power consumption can be applied to terminal devices and / or edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0116] It should be noted that, for the purpose of simplicity, the present application describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present application is not limited by the order of the actions described. Therefore, based on the disclosure or teachings of the present application, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present application can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present application. In addition, depending on the different schemes, the description of some embodiments of the present application also has different emphases. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0117] In terms of specific implementation, based on the disclosure and teachings of this application, those skilled in the art can understand that several embodiments disclosed in this application can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article divides them based on the consideration of logical functions, and there may be other ways of division in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0118] In this application, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solution described in the embodiments of this disclosure. In addition, in some scenarios, multiple units in the embodiments of this application may be integrated into one unit or each unit may exist physically separately.

[0119] In some implementation scenarios, the above-mentioned integrated unit can be implemented in the form of a software program module. If implemented in the form of a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable memory. Based on this, when the technical solution provided by the present application is embodied in the form of a software product (such as a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to enable a computer device (such as a personal computer, a server or a network device, etc.) to perform some or all of the steps of the method described in the embodiment of the present application. The aforementioned memory may include, but is not limited to, various media that can store program code, such as a USB flash drive, a flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0120] In other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, that is, as specific hardware circuits, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0121] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The foregoing content can be better understood in accordance with the following terms:

[0123] Item A1, a processing device, the processing device comprising: an acquisition unit, used to obtain multiple nodes of a neural network model and device information of each of the multiple nodes, wherein the device information of the node is information of a device that supports the operation corresponding to the node, and the device information of the node includes one or more of the following: the type of device, the computing power of the device, and the copying capability of the device; a device allocation unit, used to allocate a corresponding execution device to each node based on the device information of each node.

[0124] Clause 2. According to the processing device described in Clause 1, the multiple nodes include a first node, and the device allocation unit is specifically used to: when there are multiple devices supporting the operation corresponding to the first node, allocate a corresponding execution device to the first node based on the information of the multiple devices and the scale of the operation corresponding to the first node; wherein the scale of the operation corresponding to the first node depends on one or more of the following: the shape of the first node and the type of data input into the first node.

[0125] Clause 3. According to the processing device described in Clause 2, the multiple devices include a first device and a second device, the computing power of the first device is higher than the computing power of the second device, and the device allocation unit is used to: when the scale of the operation corresponding to the first node is greater than the preset scale, select the first device as the execution device of the first node; when the scale of the operation corresponding to the first node is less than or equal to the preset scale, select the second device as the execution device of the first node.

[0126] Clause 4. According to the processing device described in Clause 1, the multiple nodes include multiple consecutive nodes, and the device allocation unit is specifically used to: when there are multiple devices supporting operations corresponding to the multiple consecutive nodes, allocate corresponding execution devices to the multiple consecutive nodes based on the information of the multiple devices and the number of the multiple consecutive nodes.

[0127] Clause 5. According to the processing device described in Clause 4, the multiple devices include a third device and a fourth device, the computing power of the third device is higher than the computing power of the fourth device, and the data when the third device performs the operation is copied from the fourth device to the third device, and the device allocation unit is used to: when the number of multiple consecutive nodes is greater than or equal to a preset number, select the third device as the execution device of the multiple consecutive nodes; when the number of multiple consecutive nodes is less than the preset number, select the fourth device as the execution device of the multiple consecutive nodes.

[0128] Clause 6. According to the processing device described in Clause 1, the acquisition unit is further used to obtain the connection relationship between multiple nodes; the device allocation unit is further used to allocate each node to the corresponding computing library for operation based on the information of the execution device corresponding to each node and the connection relationship between the multiple nodes.

[0129] Clause 7. According to the processing device described in Clause 6, the device allocation unit is specifically used to: obtain at least one node with a fifth device as an execution device, wherein the information of the fifth device includes that a first computing library and a second computing library are configured in the fifth device, and the speed of running consecutive nodes on the first computing library is greater than the speed of running consecutive nodes on the second computing library, and at least one node includes the second node; determine whether there is a node connected to the second node in at least one node; if there is no node connected to the second node, allocate the second node to the second computing library for operation; if there is a node connected to the second node, when the first computing library includes computing instructions corresponding to the second node and the node connected to the second node, allocate the second node and the node connected to the second node to the first computing library for operation; when the first computing library does not include computing instructions corresponding to the second node and the node connected to the second node, allocate the second node and the node connected to the second node to the second computing library for operation.

[0130] Clause 8, a combined computing device, comprising a processing device, an interface device, a storage device, and one or more computing devices as described in any one of Clauses 1-7. The processing device is used to assign nodes in a neural network model to corresponding computing devices; the computing device is used to perform operations corresponding to the nodes in the neural network model; when the combined computing device includes multiple computing devices, the multiple computing devices can be connected and transmit data through a specific structure; the multiple computing devices are interconnected and transmit data through a PCIE bus to support larger-scale neural network operations; the multiple computing devices share the same control system or have their own control systems; the multiple computing devices share memory or have their own memory; the multiple computing devices are interconnected in any interconnection topology; and the storage device is connected to the processing device and one or more computing devices, respectively, to store data from the processing device and one or more computing devices.

[0131] Item 9. A neural network chip comprising the combined computing device described in Item 8.

[0132] Clause 10, a node device allocator, the method comprising: obtaining multiple nodes of a neural network model and device information of each of the multiple nodes, wherein the device information of the node is information of a device that supports operations corresponding to the node, and the device information of the node includes one or more of the following: the type of device, the computing power of the device, and the copying capability of the device; and allocating a corresponding execution device to each node based on the device information of each node.

[0133] The embodiments of the present application are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. At the same time, changes or modifications made by those skilled in the art based on the ideas of the present application, the specific implementation methods, and the scope of application of the present application, all fall within the scope of protection of the present application. In summary, the contents of this specification should not be construed as limiting the present application.

Claims

1. A processing device, characterized in that The device comprises: an acquiring unit, configured to acquire device information of a plurality of nodes of a neural network model and each of the plurality of nodes, wherein the device information of the node is information of a device that supports an operation corresponding to the node, and the device information of the node includes one or more of the following: a type of the device, a computing capability of the device, and a copy capability of the device; A device allocation unit, configured to allocate a corresponding execution device to each node according to the device information of each node; The plurality of nodes include a plurality of consecutive nodes, and the device allocation unit is specifically configured to: When there are multiple devices supporting operations corresponding to the multiple consecutive nodes, corresponding execution devices are allocated to the multiple consecutive nodes according to information of the multiple devices and the number of the multiple consecutive nodes.

2. The device according to claim 1, characterized in that The plurality of nodes include a first node, and the device allocation unit is specifically configured to: When there are multiple devices that support the operation corresponding to the first node, assigning a corresponding execution device to the first node according to information of the multiple devices and the scale of the operation corresponding to the first node; The scale of the operation corresponding to the first node depends on one or more of the following: the shape of the first node and the type of data input into the first node.

3. The device according to claim 2, characterized in that The plurality of devices include a first device and a second device, the computing capability of the first device is higher than the computing capability of the second device, and the device allocation unit is configured to: When the scale of the operation corresponding to the first node is greater than a preset scale, selecting the first device as the execution device of the first node; When the scale of the operation corresponding to the first node is less than or equal to a preset scale, the second device is selected as the execution device of the first node.

4. The device according to claim 1, characterized in that The multiple devices include a third device and a fourth device, the computing capability of the third device is higher than that of the fourth device, and the data when the third device performs a calculation is copied from the fourth device to the third device, and the device allocation unit is configured to: When the number of the plurality of consecutive nodes is greater than or equal to a preset number, selecting the third device as an execution device for the plurality of consecutive nodes; When the number of the plurality of consecutive nodes is less than the preset number, the fourth device is selected as an execution device of the plurality of consecutive nodes.

5. The device according to claim 1, characterized in that The acquisition unit is further configured to acquire the connection relationship between the plurality of nodes; The device allocation unit is further configured to allocate each node to a corresponding computing library for operation based on information of an execution device corresponding to each node and a connection relationship between the multiple nodes.

6. The device according to claim 5, characterized in that The device allocation unit is specifically used for: Obtaining at least one node using a fifth device as an execution device, wherein information about the fifth device includes that a first computing library and a second computing library are configured in the fifth device, and a speed of running consecutive nodes on the first computing library is greater than a speed of running the consecutive nodes on the second computing library, and the at least one node includes the second node; Determining whether there is a node connected to the second node among the at least one node; In a case where there is no node connected to the second node, allocating the second node to a second computing database for operation; In the case where there is a node connected to the second node, When the first computing library includes computing instructions corresponding to the second node and the nodes connected to the second node, allocating the second node and the nodes connected to the second node to the first computing library for execution; When the first computing library does not include computing instructions corresponding to the second node and the nodes connected to the second node, the second node and the nodes connected to the second node are allocated to the second computing library for operation.

7. A combined computing device, characterized in that: The combined computing device comprises a processing device, an interface device, a storage device, and one or more computing devices according to any one of claims 1 to 6, The processing device is used to assign nodes in the neural network model to corresponding computing devices; The computing device is used to perform operations corresponding to nodes in the neural network model; When the combined computing device includes the plurality of computing devices, the plurality of computing devices may be connected and data may be transmitted between them through a specific structure; The plurality of computing devices are interconnected and transmit data via a PCIE bus to support larger-scale neural network operations; the plurality of computing devices share the same control system or have their own control systems; the plurality of computing devices share memory or have their own memory; and the plurality of computing devices are interconnected in any interconnection topology. The storage device is connected to the processing device and the one or more computing devices respectively, and is used to store data of the processing device and the one or more computing devices.

8. A neural network chip, characterized in that: The neural network chip includes the combined computing device as described in claim 7.

9. A method for allocating node devices, characterized in that: The method comprises: Acquire multiple nodes of a neural network model and device information of each of the multiple nodes, wherein the device information of the node is information about a device that supports an operation corresponding to the node, and the device information of the node includes one or more of the following: a type of the device, a computing capability of the device, and a copy capability of the device; Allocate a corresponding execution device to each node according to the device information of each node; The multiple nodes include multiple consecutive nodes. When there are multiple devices supporting operations corresponding to the multiple consecutive nodes, corresponding execution devices are allocated to the multiple consecutive nodes based on information of the multiple devices and the number of the multiple consecutive nodes.

Images