Calculation deployment method and device based on DNN model, equipment and storage medium

Abstract

The invention discloses a calculation deployment method, device and equipment based on a DNN model and a storage medium, and the method comprises the steps: receiving input data sent by a previous calculation node, processing the input data based on a first part network layer of a local DNN model, and obtaining a processing result, the input data is original data which is not processed or intermediate data which is processed; the processing result is sent to a next computing node, so that a second part of network layer of the next computing node processes the processing result until to-be-computed data corresponding to the input data is processed, and the first part of network layer and the second part of network layer are adjacent network layers in a DNN model. According to the invention, the running speed of the AIOT is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a computing deployment method, apparatus, device and storage medium based on a DNN model. Background Art

[0002] In recent years, deep neural network (DNN) models have been widely used in edge computing and the Internet of Things, forming the concept of AIoT (Artificial Intelligence of Things). AIoT combines sensors, devices, and Internet of Things technologies with artificial intelligence to transfer computing and analysis to the device side, thereby reducing data transmission and processing costs and improving data privacy and security.

[0003] However, during the operation process of the DNN model in AIoT, the DNN model is configured on a computing node. After the data is input, it is necessary to complete the calculation of all levels in the entire model for the input data before the next data can be input for calculation, which reduces the operating speed of AIoT. Summary of the invention

[0004] The main purpose of this application is to provide a computing deployment method, device, equipment and storage medium based on a DNN model, aiming to solve the technical problem of slow operation speed of AIoT in related technologies.

[0005] To achieve the above purpose, the embodiment of the present application provides a computing deployment method based on a DNN model, the method comprising:

[0006] Receive input data sent by the previous computing node, and process the input data based on the first part of the network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data;

[0007] Each computing node is configured with a complete DNN model. For one computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data.

[0008] The processing result is sent to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

[0009] In a possible implementation manner of the present application, each of the computing nodes is configured with a complete DNN model, wherein the step of configuring the DNN model to the computing node includes:

[0010] Dividing the DNN model into multiple DNN sub-models;

[0011] Obtaining the computing power requirement during the calculation process of the DNN sub-model, matching the computing power requirement with the computing power index of the computing node, and determining the DNN sub-model suitable for the computing node;

[0012] The DNN sub-model is configured to the computing node.

[0013] In a possible implementation manner of the present application, the step of dividing the input DNN model into a plurality of DNN sub-models includes:

[0014] Constructing a directed graph for the DNN model, traversing the computational vertices in the directed graph using a stack, and when traversing to multiple input nodes, pushing the multiple input nodes into the stack;

[0015] When traversing to multiple output nodes, pop the previous node of the multiple output nodes, and determine the divided DNN sub-model according to the calculation vertex in the directed graph popped from the stack;

[0016] After dividing a DNN sub-model, when the stack is empty, continue to divide the next DNN sub-model.

[0017] In a possible implementation manner of the present application, the step of matching the computing power requirement with the computing power indicator of the computing node and determining a DNN sub-model suitable for the computing node includes:

[0018] Determining a computing power index of the computing node;

[0019] According to the matching relationship between the computing power indicator and the computing power requirement, the computing node corresponding to the DNN sub-model is determined.

[0020] In a possible implementation manner of the present application, the step of determining the computing power index of the computing node includes:

[0021] The average time consumed by the computing nodes for processing data is obtained, and the average time consumed by each computing node is normalized to obtain a computing power index.

[0022] In a possible implementation manner of the present application, before the step of sending the processing result to the next computing node, the step includes:

[0023] Obtaining a computational graph, and determining the data dependency relationship between the computational nodes according to the computational graph, wherein the computational graph corresponds to a neural network layer of a DNN model, and the computational graph is used to reflect the data correlation between the various network layers in the DNN model;

[0024] The computing node into which the processing result is to be input is determined according to the data dependency relationship.

[0025] In a possible implementation of the present application, if multiple computing nodes are configured with the same DNN sub-model, the computing power indicators of the computing nodes are weighted to obtain the weight ratio of each computing node, and the computing node that executes the computing task is determined based on the weight ratio.

[0026] The present application also provides a computing deployment device based on a DNN model, and the computing deployment device based on a DNN model further includes:

[0027] A receiving module, used to receive input data sent by a previous computing node, and process the input data based on the first network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data;

[0028] Each computing node is configured with a complete DNN model. For one computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data.

[0029] A sending module is used to send the processing result to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

[0030] The present application also provides a computing deployment device based on a DNN model, which is a physical node device. The computing deployment device based on the DNN model includes: a memory, a processor, and a program of the computing deployment method based on the DNN model stored in the memory and executable on the processor. When the program of the computing deployment method based on the DNN model is executed by the processor, the steps of the computing deployment method based on the DNN model as described above can be implemented.

[0031] To achieve the above-mentioned purpose, a storage medium is also provided, on which a computing deployment program based on a DNN model is stored. When the computing deployment program based on the DNN model is executed by a processor, the steps of any of the above-mentioned computing deployment methods based on the DNN model are implemented.

[0032] The present application provides a computing deployment method, device, equipment and storage medium based on a DNN model. Compared with the related art, when the DNN model in the artificial intelligence Internet of Things is calculated, it is necessary to calculate all levels of the input data before the next data can be input for calculation, which leads to the slow operation speed of the entire artificial intelligence Internet of Things. In the present application, the input data sent by the previous computing node is received, and the input data is processed based on the first part of the network layer of the local DNN model to obtain the processing result, wherein the input data is the unprocessed original data or the processed intermediate data; wherein each computing node is configured with a complete DNN model, and for a computing node, only a part of the network layer is used to process the corresponding input data, wherein each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data; the processing result is sent to the next computing node for the second part of the network layer of the next computing node to process the processing result, until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model. In the present application, data is processed separately by multiple computing nodes. For a computing node, only a part of the network layer is used to process the corresponding input data, that is, only a part of the layer calculation in the DNN model needs to be performed on the data. Therefore, after the data is input to a computing node, it can be quickly input to the next computing node to complete the calculation process from the first part of the network layer to the second part of the network layer, without the need to execute the calculation process of all fragments in the complete DNN model, thereby reducing the calculation time of the computing node, thereby reducing the time interval between the current input data and the next input data. Compared with one computing node, multiple computing nodes each execute the calculation process of a part of the network layer of the model, which can increase the throughput of the calculation process in the same time, thereby improving the computing speed / operation speed of the entire artificial intelligence Internet. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flowchart of the first embodiment of the computing deployment method based on the DNN model of the present application;

[0034] Figure 2 A schematic diagram of adding additional input and output ports to the DNN model of the computing deployment method based on the DNN model of this application;

[0035] Figure 3 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0036] Figure 4 This is a schematic diagram of the distribution of two computing nodes involved in the computing deployment method based on the DNN model of this application;

[0037] Figure 5 This is a schematic diagram of the distribution of four computing nodes involved in the computing deployment method based on the DNN model of this application;

[0038] Figure 6 A schematic diagram of the division of DNN model segments involved in the computing deployment method based on the DNN model of this application;

[0039] Figure 7 This is a schematic diagram of the DNN sub-model division involved in the computing deployment method based on the DNN model of this application. DETAILED DESCRIPTION

[0040] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] The present application embodiment provides a computing deployment method based on a DNN model. In the first embodiment of the computing deployment method based on a DNN model of the present application, refer to Figure 1 , the method comprising:

[0042] Step S10, receiving input data sent by the previous computing node, and processing the input data based on the first part of the network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data;

[0043] Each computing node is configured with a complete DNN model. For one computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data.

[0044] Step S20, sending the processing result to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

[0045] The purpose of this embodiment is to reduce the time interval between the current input data and the next input data by decomposing the DNN model and arranging it on multiple computing nodes, thereby improving the operating speed of AIoT (Artificial Intelligence Internet of Things).

[0046] In this embodiment, the research and development background is:

[0047] Existing DNN models are widely used in edge computing and the Internet of Things, thus forming the concept of AIOT. AIOT transfers computing and analysis to the device side, thereby reducing data transmission and processing costs and improving data privacy and security. However, AIOT also has problems such as long transmission delays. A single device can only input the next data for processing after processing one data. For many continuous data (such as video frames), the running speed of a single device is slow, which cannot meet the needs of fast data processing.

[0048] The specific steps are as follows:

[0049] Step S10, receiving input data sent by the previous computing node, and processing the input data based on the first part of the network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data;

[0050] Each computing node is configured with a complete DNN model. For one computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data.

[0051] As an example, a computing deployment method based on a DNN model can be applied to a computing deployment device based on a DNN model. The computing deployment device based on a DNN model belongs to a computing deployment system based on a DNN model, and the computing deployment system based on a DNN model belongs to a computing deployment device based on a DNN model.

[0052] As an example, the first part of the network layer is a part of the network layer of the DNN model. The first part of the network layer can be a convolutional layer or a fully connected layer, without specific limitation.

[0053] As an example, a computing node is a physical device that can perform calculations, such as a smart camera, an image processing device, etc.

[0054] As an example, the input data may be video frames, image data, etc., or other data that can be input into a DNN model for processing, without specific limitation.

[0055] As an example, when performing calculations, multiple computing nodes cooperate. For example, after the first computing node receives input data, it processes the input data to obtain an intermediate result. The second computing node receives the intermediate result and processes it. Similarly, the subsequent computing nodes perform the same operation, thereby completing the process of multi-device collaborative processing of the DNN model.

[0056] As an example, the local network layer is a neural network layer of a DNN model, and the processing process is completed by the DNN model to obtain a processing result.

[0057] As an example, the input data may be original data and processed data. When the input data is original data, the current computing node is the first computing node. When the input data is processed intermediate data, the current computing node is a computing node other than the first computing node.

[0058] As an example, each computing node has multiple input and output interfaces. Since the computing node is configured with a complete DNN model and only executes the computing functions of a part of the network layer, the network layer that executes the computing functions can be changed by changing the positions of the input and output ports. For example, the computing node was originally set to execute the computing process of the first layer of the network layer. By changing the position of the output port, the computing node can execute the computing process of the first and second layers of the network layer.

[0059] As an example, the DNN model can be regarded as a black box. In a complete forward reasoning process, a batch of data is input from the first layer of the model, and is calculated in each layer in turn, and finally the result is output from the last layer. The whole process is carried out sequentially. The present invention adds multiple sets of additional input and output to the DNN model. For example, an output interface can be added after the second layer of a 4-layer DNN model, and an input interface can be added before the third layer. The structure is as follows: Figure 2 The model is deployed on two computing devices, one of which is configured as follows: the first layer inputs the original data, the second layer outputs the intermediate result, and then ends the operation; the other is configured as follows: the third layer inputs the intermediate result from the previous machine, and the fourth layer outputs the final result, as shown in Figure 4 As shown in the figure, the solid line box represents valid, and the dotted line box represents invalid; similarly, the following can also be implemented Figure 5The above example illustrates the working principle of the present invention in the simplest case. It should be noted that the network path in the above diagram is only a logical path. In the actual application process of deploying the DNN model, the situation may be much more complicated. For example, the actual number of DNN layers is generally deeper, and bypasses may occur. This method makes the DNN reasoning process streamlined, and the more intermediate interfaces are added and the more devices are deployed, the finer the granularity of the pipeline.

[0060] Wherein, before the step of sending the processing result to the next computing node, the step includes:

[0061] Step S101, obtaining a computation graph, and determining the data dependency relationship between the computation nodes according to the computation graph, wherein the computation graph corresponds to a neural network of a DNN model, and the computation graph is used to reflect the data correlation between the operators in the DNN model;

[0062] As an example, a computational graph is specifically a schematic diagram composed of the computations of a neural network. In the computational graph, matrix calculations are used as computational vertices, and data transfers are used as data edges, thus forming the computational graph.

[0063] As an example, the computational graph includes a multi-level expression of a neural network. The computational vertices in each network layer can be executed in parallel, while the two computational vertices between network layers have data dependencies and therefore need to be executed serially or in a pipeline. This is different from traditional visual intelligent perception systems, in which the perception of visual information mainly occurs within the same intelligent perception entity (such as a smart camera) or in an edge-cloud vertical collaborative system. In this embodiment, horizontal collaboration can be performed among multiple peer intelligent entities in the same network layer (such as smart cameras or edge intelligent devices).

[0064] As an example, the data dependency relationship of the computing vertices in the computing graph may be: data is input from the previous computing vertex, then the computing result is output, and then the computing result is input to the next computing vertex, and the next step of operation is performed again.

[0065] As an example, since the computational graph is a graph obtained by the computational abstraction of the neural network, the computational graph corresponds to the neural network of the DNN model.

[0066] Step S102: determining the computing node to which the processing result is to be input according to the data dependency relationship.

[0067] As an example, since a DNN model is configured in each computing node, the computing node that needs to be input for the data to be calculated can be determined based on the data dependency relationship between the DNN models in each computing node. For example, if there is a data dependency relationship between the first computing node and the second computing node, the output result of the first computing node needs to be input into the second computing node, thereby completing the distributed computing process of the data.

[0068] Step S20, sending the processing result to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

[0069] As an example, after determining the association relationship between computing nodes according to the data dependency relationship, the processing result is sent to the next computing node that processes the data accordingly.

[0070] As an example, the second network layer can be a convolutional layer or a fully connected layer, wherein the first network layer and the second network layer are adjacent, that is, the result obtained after the first network layer processes the data needs to be input into the second network layer for the next step of processing, so as to complete the processing of the input data.

[0071] As an example, the processing result may be an intermediate result obtained after processing the input data. During the processing, the input data is processed by multiple computing nodes, thereby obtaining multiple intermediate results until the data processing is completed.

[0072] As an example, during the computing node processing, since the computing node only uses a part of the network layer to process the input data, the processing time of a single computing node for data can be reduced. On the other hand, the time interval between the current input data and the next input data is reduced.

[0073] In this embodiment, during the data scheduling process, the data transmission process is completed through the data pipeline, which is the data transmission channel between each computing node. Since the network transmission protocol has a transmission length limit, directly transmitting data will cause greater pressure on the network. Therefore, by transmitting the data in blocks in a pipeline manner, each computing node can process multiple data blocks at the same time and pass the results to the next computing node. This method can maximize the use of bandwidth resources and improve the efficiency of data transmission while ensuring data dependencies.

[0074] The present application provides a computing deployment method, device, equipment and storage medium based on a DNN model. Compared with the related art, when the DNN model in the artificial intelligence Internet of Things is calculated, it is necessary to calculate all levels of the input data before the next data can be input for calculation, which leads to the slow operation speed of the entire artificial intelligence Internet of Things. In the present application, the input data sent by the previous computing node is received, and the input data is processed based on the first part of the network layer of the local DNN model to obtain the processing result, wherein the input data is the unprocessed original data or the processed intermediate data; wherein each computing node is configured with a complete DNN model, and for a computing node, only a part of the network layer is used to process the corresponding input data, wherein each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data; the processing result is sent to the next computing node for the second part of the network layer of the next computing node to process the processing result, until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model. In the present application, data is processed separately by multiple computing nodes. For a computing node, only a part of the network layer is used to process the corresponding input data, that is, only a part of the layer calculation in the DNN model needs to be performed on the data. Therefore, after the data is input to a computing node, it can be quickly input to the next computing node to complete the calculation process from the first part of the network layer to the second part of the network layer, without the need to execute the calculation process of all fragments in the complete DNN model, thereby reducing the calculation time of the computing node, thereby reducing the time interval between the current input data and the next input data. Compared with one computing node, multiple computing nodes each execute the calculation process of a part of the network layer of the model, which can increase the throughput of the calculation process in the same time, thereby improving the computing speed of the entire artificial intelligence Internet.

[0075] Further, based on the first embodiment of the present application, another embodiment of the present application is provided. In this embodiment, a complete DNN model is configured on each computing node, wherein the step of configuring the DNN model to the computing node includes:

[0076] Step A1, dividing the DNN model into multiple DNN sub-models;

[0077] As an example, a complete DNN model is configured on each computing node. Since the computing node only uses a part of the network layer to process data, the DNN model needs to be divided.

[0078] As an example, the DNN sub-model includes a portion of the network layers of the DNN model, and the DNN sub-model includes but is not limited to the sub-model obtained by completely cutting the DNN model.

[0079] In this embodiment, the DNN sub-model is obtained after dividing the network layer of the DNN model, rather than cutting the DNN model to obtain several sub-models in the true sense. Additional input and output ports are added to the DNN model, and the input and output of the specified data are equivalent to intercepting a part of the network layer in the model, thereby completing the division of the sub-model.

[0080] Step A2, obtaining the computing power requirement during the calculation process of the DNN sub-model, matching the computing power requirement with the computing power index of the computing node, and determining the DNN sub-model suitable for the computing node;

[0081] As an example, the DNN sub-model requires computing power corresponding to the computing process during the calculation process, so that the corresponding computing power requirements can be determined.

[0082] As an example, the computing power requirement may be 1000H, or other values, without limitation.

[0083] As an example, the computing power indicator reflects the computing capacity of a computing node. The data calculation process can only be completed when the computing power indicator of the computing node matches the computing power demand.

[0084] Step A3: configure the DNN sub-model to the computing node.

[0085] As an example, the DNN sub-model is configured into the corresponding computing node by matching the computing power index of the computing node with the computing power requirement of the DNN sub-model.

[0086] The step of dividing the input DNN model into multiple DNN sub-models includes:

[0087] Step B1, constructing a directed graph for the DNN model, using a stack to traverse the computational vertices in the directed graph, and when traversing to multiple input nodes, pushing the multiple input nodes into the stack;

[0088] As an example, a directed graph is a directional graph with the computational direction of the neural network of the DNN model as its main trunk, wherein, in the directed graph, similar to the computational graph, one computational vertex corresponds to one operator, and the network levels are divided according to the computing power, thereby completing the construction process of the directed graph corresponding to the DNN model.

[0089] As an example, the output nodes and input nodes in each network layer are traversed through the stack, and when multiple input nodes are traversed, the multiple input nodes are pushed into the stack.

[0090] Step B2, when traversing to multiple output nodes, pop the previous node of the multiple output nodes from the stack, and determine the divided DNN sub-model according to the calculation vertex in the directed graph popped from the stack;

[0091] As an example, when traversing to multiple output nodes, the previous node of the multiple output nodes is popped out of the stack, and the divided DNN sub-model is determined based on the calculation vertices in the directed graph popped out of the stack.

[0092] After dividing a DNN sub-model, when the stack is empty, continue to divide the next DNN sub-model.

[0093] As an example, in the process of dividing DNN sub-models, a sub-model must ensure that all complete branches are included. Each DNN sub-model is a black box with a single input and a single output. When there are branches in the model (such as ResNet), when selecting fragments, ensure that the branches are completely included to avoid truncation. The division method is as follows Figure 6 shown.

[0094] As an example, the process of dividing the DNN sub-model through the stack is last-in-first-out. When the stack is empty, the division process of a DNN sub-model is completed. When the stack is empty, the next DNN sub-model is divided. The specific process of dividing the DNN sub-model is as follows: Figure 7 shown.

[0095] As an example, Figure 7 As shown, the directed graph includes three computing vertices a, b, and c. By pushing a into the stack and popping a out of the stack, the three computing vertices a, b, and c are divided into the same sub-model. The division result is similar when there are four computing vertices a, b, c, and d.

[0096] The step of matching the computing power requirement with the computing power index of the computing node and determining a DNN sub-model suitable for the computing node includes:

[0097] Step C1, determining the computing power index of the computing node;

[0098] Step C2, determining the computing node corresponding to the DNN sub-model according to the matching relationship between the computing power indicator and the computing power requirement;

[0099] As an example, there is a corresponding matching relationship between the computing power index and the computing power demand, that is, a computing node with a large computing power demand is matched with a computing node with a strong computing power, and a computing node with a small computing power demand is matched with a computing node with a weak computing power.

[0100] As an example, if the computing power requirement of the DNN sub-model is greater than the first preset threshold, the DNN sub-model is configured to a computing node with strong computing power. The first preset threshold may be 1000H, 2000H, etc., without specific limitation. For example, when the computing power requirement is 1500H and the first preset threshold is 1000H, the DNN sub-model is configured to a computing node with a computing power value of about 1500H, so that the computing node matches the computing power requirement.

[0101] As an example, if the computing power requirement of the DNN sub-model is less than or equal to the first preset threshold, the DNN sub-model is configured to a computing node with weak computing power. The configuration method is the same as the above method and will not be repeated here.

[0102] As an example, the computing nodes corresponding to the DNN sub-models are matched accordingly according to the computing power requirements of the DNN sub-models.

[0103] The step of determining the computing power index of the computing node includes:

[0104] Step D1, obtaining the average time consumed by the computing nodes to process data, and normalizing the average time consumed by each computing node to obtain a computing power index.

[0105] As an example, the average time consumption is the time consumed by the computing node to process data. The average time consumption can be obtained by running a benchmark to roughly determine it. For example, the same model inference process is run on 4 machines respectively, and the average is taken multiple times. The average time consumption obtained can be used as the computing power indicator of the machine. After obtaining a set of average time consumption, the machine performance indicator can be normalized. For example: the benchmark result is [10ms, 12ms, 8ms, 7.5ms], then the computing power is normalized to [1 / 10, 1 / 12, 1 / 8, 1 / 7.5] / (1 / 10+1 / 12+1 / 8+1 / 7.5) to obtain [0.23, 0.19, 0.28, 0.30]; the inverse is calculated here because the processing time consumption is negatively correlated with the machine performance.

[0106] As an example, similarly, the computing power requirements of operators in the model can also be determined through the benchmark method. Give an input to the DNN model, run DNN reasoning, and use the tool to track the data flow throughout the process, time each vertex, and use it as the computing power indicator of the operator. Note that there is no need to find the inverse here, because the computing power required by the operator is positively correlated with the operator calculation time. After obtaining the calculation time of each operator in each DNN, normalize it to get the required computing power indicator.

[0107] As an example, in addition to computing power requirements, the cost of communication must also be considered in the process of collaboration among various computing nodes. When data flows in the DNN model, the vector dimensions of each layer output are changing. For image applications, these intermediate calculation results can be called feature maps. Generally speaking, during the forward reasoning process of a DNN model, the width and height of the feature map gradually decrease, and the number of channels gradually increases, so the data size change curve is spindle-shaped. It can be seen that the data transmission cost introduced may also be different in different position divisions of the DNN model, so while determining each DNN sub-model fragment based on computing power, the transmission cost can be introduced, and the quantization method can also use time consumption, and the transmission time consumption can be calculated based on the network bandwidth and data volume.

[0108] As an example, in the process of dividing the sub-model, take resnet18 as an example, input the data dimension [3,224,224], perform inference, track each operator and count the time consumption, and use it as the computing power indicator after averaging multiple times. The number of layers to avoid truncating branches has been determined in advance, and the results are as follows:

[0109]

[0110]

[0111] The module similar to layer0.0 contains complete branches and can be used as a unit for division. Assuming that the computing power of the four machines is the same, it is necessary to ensure that the sub-segments of the divided DNN model take as much time as possible. The result of running the algorithm is:

[0112] Layers 1 to 6: 1.23 (ms)

[0113] Layer 7 to 8: 1.02 (ms)

[0114] Layer 9 to 10: 1.03 (ms)

[0115] Layers 11 to 15: 1.18 (ms)

[0116] This results in a DNN partitioning scheme.

[0117] In this embodiment, the data transmission cost can be ignored, but it cannot be ignored when network bandwidth resources are limited.

[0118] In this embodiment, by matching the computing power requirements of the DNN sub-model with the instance indicators of the computing nodes, the network layers of multiple different parts of the DNN model can be configured correspondingly to the corresponding computing nodes, so that each computing node executes a part of the computing process, ensuring that the overall computing process is streamlined.

[0119] Furthermore, based on the first and second embodiments of the present application, another embodiment of the present application is provided. In this embodiment, if multiple computing nodes are configured with the same DNN sub-model, the computing power indicators of the computing nodes are weighted to obtain the weight ratio of each computing node, and the computing node that executes the computing task is determined based on the weight ratio.

[0120] As an example, during the configuration process, computing nodes may be configured with the same DNN sub-model. When multiple computing nodes are configured with the same DNN sub-model, it is necessary to schedule the computing tasks and select the optimal computing node to complete the corresponding computing tasks.

[0121] As an example, when the same DNN sub-model is included in multiple computing nodes, each computing node is weighted through polling scheduling. The weighting process is to add a corresponding weight ratio to each computing node. For example, the first computing node and the second computing node are configured with the same DNN sub-model, but the computing power of the first computing node is stronger than that of the second computing node. At this time, the weight ratio of the first computing node is increased to 2 / 3, and the weight ratio of the second computing node is increased to 1 / 3. According to the weight ratio, the first computing node can be selected to perform the computing task.

[0122] In this embodiment, multiple computing nodes configured with the same DNN sub-model are weighted, so that corresponding computing tasks are allocated according to the weight ratio, thereby avoiding insufficient computing power during calculation and optimizing the calculation process.

[0123] Reference Figure 3 , Figure 3 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application.

[0124] like Figure 3 As shown, the computing deployment device based on the DNN model may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005.

[0125] Optionally, the computing deployment device based on the DNN model may also include a user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, a WiFi module, etc. The user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. The network interface may include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0126] Those skilled in the art will understand that Figure 3The DNN model-based computing deployment device structure shown in the figure does not constitute a limitation on the DNN model-based computing deployment device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0127] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, and a computing deployment program based on a DNN model. The operating system is a program that manages and controls the hardware and software resources of the computing deployment device based on the DNN model, and supports the operation of the computing deployment program based on the DNN model and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and to communicate with other hardware and software in the computing deployment system based on the DNN model.

[0128] exist Figure 3 In the DNN model-based computing deployment device shown, the processor 1001 is used to execute the DNN model-based computing deployment program stored in the memory 1005 to implement the steps of the DNN model-based computing deployment method described in any one of the above items.

[0129] The specific implementation methods of the computing deployment device based on the DNN model in the present application are basically the same as the above-mentioned embodiments of the computing deployment method based on the DNN model, and will not be repeated here.

[0130] The present application also provides a computing deployment device based on a DNN model, and the computing deployment device based on a DNN model includes:

[0131] A receiving module, used to receive input data sent by a previous computing node, and process the input data based on the first network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data;

[0132] Each computing node is configured with a complete DNN model. For one computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data.

[0133] A sending module is used to send the processing result to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

[0134] In a possible implementation manner of the present application, the computing deployment device based on the DNN model further includes:

[0135] A configuration module, used for configuring the DNN model to the computing node;

[0136] In a possible implementation manner of the present application, the configuration module includes:

[0137] A division unit, used for dividing the DNN model into a plurality of DNN sub-models;

[0138] An acquisition unit, used to acquire the computing power requirement during the calculation process of the DNN sub-model, match the computing power requirement with the computing power index of the computing node, and determine the DNN sub-model suitable for the computing node;

[0139] A configuration unit is used to configure the DNN sub-model to the computing node.

[0140] In a possible implementation manner of the present application, the division unit includes:

[0141] A construction subunit is used to construct a directed graph for the DNN model, traverse the computational vertices in the directed graph using a stack, and push the multiple input nodes into the stack when traversing to multiple input nodes;

[0142] The first determination subunit is used to pop the previous node of the multiple output nodes when traversing to the multiple output nodes, and determine the divided DNN sub-model according to the calculation vertex in the directed graph popped from the stack.

[0143] In a possible implementation manner of the present application, the acquisition unit includes:

[0144] A second determination subunit, used to determine the computing power index of the computing node;

[0145] The third determination subunit is used to determine the computing node corresponding to the DNN sub-model according to the matching relationship between the computing power indicator and the computing power requirement.

[0146] In a possible implementation manner of the present application, the second determining subunit includes:

[0147] The processing component is used to obtain the average time consumed by the computing node to process data, and normalize the average time consumed by each computing node to obtain a computing power indicator.

[0148] In a possible implementation manner of the present application, the computing deployment device based on the DNN model further includes:

[0149] An acquisition module, used to acquire a computational graph, and determine the data dependency relationship between the computational nodes according to the computational graph, wherein the computational graph corresponds to the neural network layer of the DNN model, and the computational graph is used to reflect the data association between the various network layers in the DNN model;

[0150] A determination module is used to determine the computing node to which the processing result is to be input according to the data dependency relationship.

[0151] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0152] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, including multiple instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0154] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A computing deployment method based on a DNN model, characterized in that: The method comprises the following steps: Receive input data sent by the previous computing node, and process the input data based on the first part of the network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data; Each computing node is configured with a complete DNN model. For a computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data. The processing result is sent to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

2. The computing deployment method based on the DNN model according to claim 1, characterized in that: A complete DNN model is configured on each of the computing nodes, wherein the step of configuring the DNN model to the computing node comprises: Dividing the DNN model into multiple DNN sub-models; Obtaining the computing power requirement during the calculation process of the DNN sub-model, matching the computing power requirement with the computing power index of the computing node, and determining the DNN sub-model suitable for the computing node; The DNN sub-model is configured to the computing node.

3. The computing deployment method based on the DNN model as claimed in claim 2, characterized in that: The step of dividing the input DNN model into a plurality of DNN sub-models comprises: Constructing a directed graph for the DNN model, traversing the computational vertices in the directed graph using a stack, and when traversing to multiple input nodes, pushing the multiple input nodes into the stack; When traversing to multiple output nodes, pop the previous node of the multiple output nodes, and determine the divided DNN sub-model according to the calculation vertex in the directed graph popped from the stack; After dividing a DNN sub-model, when the stack is empty, continue to divide the next DNN sub-model.

4. The computing deployment method based on the DNN model as claimed in claim 2, characterized in that: The step of matching the computing power requirement with the computing power indicator of the computing node and determining a DNN sub-model suitable for the computing node includes: Determining the computing power index of the computing node; According to the matching relationship between the computing power indicator and the computing power requirement, the computing node corresponding to the DNN sub-model is determined.

5. The computing deployment method based on the DNN model as claimed in claim 4, characterized in that: The step of determining the computing power index of the computing node includes: The average time consumed by the computing nodes for processing data is obtained, and the average time consumed by each computing node is normalized to obtain a computing power index.

6. The computing deployment method based on the DNN model according to claim 1, characterized in that: Before the step of sending the processing result to the next computing node, the method further comprises: Obtaining a computational graph, and determining the data dependency relationship between the computational nodes according to the computational graph, wherein the computational graph corresponds to a neural network layer of a DNN model, and the computational graph is used to reflect the data association between the various network layers in the DNN model; The computing node into which the processing result is to be input is determined according to the data dependency relationship.

7. The computing deployment method based on the DNN model as claimed in claim 2, characterized in that: If multiple computing nodes are configured with the same DNN sub-model, the computing power indicators of the computing nodes are weighted to obtain the weight ratio of each computing node, and the computing node that executes the computing task is determined based on the weight ratio.

8. A computing deployment device based on a DNN model, characterized in that: The computing deployment device based on the DNN model includes: A receiving module, used to receive input data sent by a previous computing node, and process the input data based on the first network layer of the local DNN model to obtain a processing result, wherein the input data is unprocessed original data or processed intermediate data; Each computing node is configured with a complete DNN model. For one computing node, only a part of the network layer is used to process the corresponding input data. Each computing node has multiple input and output interfaces instead of only one input and output interface, so that only a part of the network layer can be used to process the corresponding input data. A sending module is used to send the processing result to the next computing node so that the second part of the network layer of the next computing node processes the processing result until the processing of the data to be calculated corresponding to the input data is completed, wherein the first part of the network layer and the second part of the network layer are adjacent network layers in the DNN model.

9. A computing deployment device based on a DNN model, characterized in that: The device includes: a memory, a processor, and a DNN model-based computing deployment program stored in the memory and executable on the processor, wherein the DNN model-based computing deployment program is configured to implement the steps of the DNN model-based computing deployment method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computing deployment program based on a DNN model, and when the computing deployment program based on a DNN model is executed by a processor, the steps of the computing deployment method based on a DNN model as described in any one of claims 1 to 7 are implemented.

Images