Neural network split scheduling method, device and storage medium for stream task

Abstract

The application provides a neural network splitting scheduling method for streaming tasks, an electronic device and a computer readable storage medium. Potential games are performed on each layer of a neural network, Nash equilibrium is found, scheduling decisions of each layer are optimized, the neural network is split and scheduled according to the optimized scheduling decisions, time consumption is short, the method can be applied to streaming tasks, and the method can be applied to large-scale problems.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of neural network splitting scheduling, and particularly relates to a large-scale neural network splitting scheduling method for streaming tasks, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the popular application of a large number of sensing devices in the Internet of Everything, an application program based on a deep neural network cannot be efficiently executed on a terminal due to huge resource requirements, an initial solution is to split and schedule a deep neural network task to the cloud, but this will introduce considerable communication delay and expose user privacy.

[0003] In order to further accelerate the inference speed of the deep neural network on the Internet of Things terminal and protect the user privacy, researchers have proposed the concept of dividing and scheduling the deep neural network model on multiple computing units to shorten the inference time.

[0004] The deep neural network is usually trained only once and then used to process a series of data. Many deep neural network applications need to process streaming tasks with high-speed release and request fast response, however, existing streaming task splitting and scheduling algorithms are very time-consuming.

[0005] One of the typical streaming tasks is a video stream, in a smart city, many applications such as video monitoring and unmanned driving involve real-time processing of the video stream. The time complexity of the latest streaming task model OCDST is an exponential function of the number of neural network layers and a power function of the number of computing units, and OCDST cannot adapt to large-scale problems. SUMMARY

[0006] Therefore, in view of the above technical problems, the application provides a large-scale neural network splitting scheduling method for streaming tasks, an electronic device and a computer readable storage medium.

[0007] The technical scheme adopted by the application is as follows:

[0008] On the one hand, a neural network splitting scheduling method for streaming tasks is provided, characterized by comprising:

[0009] S101, determining the computation amount of each layer of the neural network and determining the computation speed of each computing unit;

[0010] S102, calculating the single computation delay of each layer scheduled to each computing unit by the following formula:

[0011]

[0012] Wherein, c i ∈C={c1,c2,...,c Nrepresents the computation amount of layer i, represents the computation speed of computation unit k, represents the single computation delay of layer i scheduled to computation unit k;

[0013] S103, the computation unit with the minimum single computation delay of each layer is taken as the target computation unit of the layer, and the scheduling decision of each layer is formed;

[0014] S104, the scheduling decision is updated and optimized: Nash equilibrium is found through potential game of each layer, so that the global running time of each layer is the shortest;

[0015] S105, according to the scheduling decision of each layer after the optimization update, the neural network is split and scheduled to the target computation unit of the corresponding scheduling decision.

[0016] On the other hand, an electronic device is provided, characterized by comprising a storage module, the storage module comprises instructions loaded and executed by a processor, the instructions make the processor execute the above-mentioned neural network split scheduling method for streaming tasks when executed.

[0017] In another aspect, a computer readable storage medium is provided, which stores one or more programs, characterized in that the one or more programs, when executed by a processor, implement the above-mentioned neural network split scheduling method for streaming tasks.

[0018] The present application carries out potential game of each layer of neural network, finds out Nash equilibrium, optimizes the scheduling decision of each layer, so that the neural network is split and scheduled according to the optimized scheduling decision, the time consumption is short, can be applied to streaming tasks, and can be applied to large-scale problems. BRIEF DESCRIPTION OF DRAWINGS

[0019] The present application will be described in detail below in combination with the drawings and specific embodiments:

[0020] Figure 1 is the flow chart of the present application;

[0021] Figure 2 is the schematic diagram of CDNN local running;

[0022] Figure 3 is the schematic diagram of CDNN running shared by multiple computation units;

[0023] Figure 4 is the schematic diagram of GCNN running shared by four computation units;

[0024] Figure 5 is the schematic diagram of deriving the average processing time of task flow;

[0025] Figure 6A schematic diagram for simplifying AlexNet into CDNN for the present application;

[0026] Figure 7 A schematic diagram for comparing the number of frames processed by different models at different times;

[0027] Figure 8 A schematic diagram for comparing the inference speed of different models at 125Mbyte bandwidth;

[0028] Figure 9 A schematic diagram for comparing the inference speed of different models at 2.5Gbyte bandwidth. DETAILED DESCRIPTION

[0029] As Figure 1 shown, the embodiment of the present specification provides a neural network splitting scheduling method for streaming tasks, comprising:

[0030] S101, determining the calculation amount of each layer of the neural network, and determining the calculation speed of each calculation unit.

[0031] Wherein, the neural network is located on a user equipment (UE), therefore, the calculation amount (floating point operation times, FLOPs) of each layer of the neural network is known, while the calculation speed (floating point operation times per second, FLOPS) of each calculation unit can be obtained from each calculation unit.

[0032] S102, calculating the single calculation delay of each layer scheduled to each calculation unit by the following formula:

[0033]

[0034] Wherein, c i ∈C={c1,c2,...,c N} represents the calculation amount of layer i, represents the calculation speed of calculation unit k, represents the single calculation delay of layer i scheduled to calculation unit k.

[0035] S103, taking the calculation unit with the minimum single calculation delay of each layer as the target calculation unit of the layer, and forming the scheduling decision of each layer.

[0036] S104, optimizing and updating the scheduling decision: finding the Nash equilibrium through potential game of each layer, so as to make the global running time of each layer shortest.

[0037] As Figure 2As shown, taking CDNN (Chain Neural Network) as an example, when it runs locally in UE, the input DT is first read from the hard disk to the random access memory RAM, and then the CDNN processes the input DT as a whole to generate the expected output OP, the delay of this process includes the reading time of the hard disk Processing time of CDNN And the reading time of RAM And the writing time When CDNN runs on a graphics processing unit GPU, And The data exchange time between RAM and GPU.

[0038] As Figure 3 shown, when CDNN is shared by multiple computing units, for serial computing, the first computing unit receives input from its memory or external device, then executes its CDNN part to generate intermediate results, and then transmits these results to the next computing unit, this process will be repeated until the last computing unit completes the calculation and returns the result to the user. Where CDNN is divided into two parts and scheduled to computing units fog1 (f1) and fog2 (f2), fog1 receives input data DT from mobile phone u and generates intermediate results MR from the input, and then transmits them to fog2 to perform the same operation, finally, the output OP is returned to the mobile phone. We assume that CDNN runs on GPU, so the task delay T c is composed of processing time Transmission time between mobile phone, fog1, fog2 And the transmission time between RAM and GPU

[0039] It can be seen that the delay of the split and scheduled CDNN in processing a single task is the sum of transmission and processing time, all transmissions and processing must run in strict order like a pipeline, otherwise, it will not return valid results, however, when the user issues a large number of tasks that need to be processed in order, the delay of these tasks will be the product of the number of tasks and the delay of a single task. That is, if there are S tasks, the total data set will be S x T c However, each data transmission time and processing time is less than T c Each part in the pipeline will enter an idle state after completing its task, which will waste a lot of time.

[0040] ​Considering a specific task, it is continuously and rapidly released and needs to be processed in order. We call it a streaming task. Assuming that the release speed of the streaming task is always higher than the execution speed, reducing the average task latency can improve the quality of service, and we achieve this goal by compressing the idle time. In fact, for each task, each part in the pipeline does the same thing: gets data from its previous part (for the head of the pipeline, the previous part is the UE), executes the processing task, and publishes data to its next part. They can be regarded as independent filters, triggered only by its previous part and triggering its next part. Generally, a filter in the pipeline needs to wait for the arrival of input before it can start execution, but when processing a streaming task, when filter i is processing data <i2,j>, it can simultaneously get intermediate data <i2,j+1> from the previous filter and transmit intermediate data <i1,j-1> to the next filter, that is, all filters will continuously obtain input, so they will not be idle for too long. Among them, <i k k

[0041] Assuming that there are a total of filters, where represents the filtering time of the gth filter F g . If there is a filter F h , whose filtering time is greater than the previous filter F h-1 (assuming F h-1 exists), that is, In this way, after F h-1 finishes filtering at time , F h is still filtering the previous data, and F h-1 must wait for a time of length before sending its output. In this example, the filtering delay of F h-1 and F h is that is, the larger delay will be passed to the front filter. On the other hand, if there is When F u+1 finishes filtering at time , it needs to idle or sleep during , because F u is filtering the previous data. At this time, the filtering delay of F u and F u+1 is ​​That is, larger delays will also be passed to the late filter. Therefore, the maximum delay will spread to the entire filter flow and determine the delay of the entire task flow. Task delay optimization can be achieved by minimizing the maximum filter delay, and the splitting and scheduling of CDNN become the following min-max problem:

[0042]

[0043] The available fog / edge devices are usually low-performance computing units without GPUs, and the transmission speed in the local area network (LAN) is very fast, for example, the widely deployed 5G network can reach a rate of 2.5 gigabytes per second, which means that the transmission of 25 megabytes of data only takes 0.01 seconds, while the computing unit usually takes several seconds or even minutes to implement complex neural network inference. Therefore, we assume that the computing time is always greater than the transmission time, and the problem becomes minimizing the maximum computing time:

[0044] min max(t i ),t i ∈φ, where Φ represents the set of filter computing times. This model is equivalent to:

[0045] Where the minimization goal is the difference in computing time between all filters for computing.

[0046] Therefore, constructing the potential game of the DNN layer and finding the Nash equilibrium can solve this problem, and accordingly, the specific process of step S104 is as follows:

[0047] 1. Potential game construction:

[0048] Let and represent the layers and computing units of the neural network respectively, and the scheduling decision of layer i is represented as a i =k, The overall scheduling decision is represented as a=(a1,a2,...,a N ).

[0049] The actual delay of layer i can be calculated by the following formula:

[0050]

[0051] Where I {x} is the indicator function, and a -i is the scheduling decision of the layer other than layer i. Specifically, I {true} =1,I {false} =0.

[0052] We call the potential game in the present application as the layer unit competition UCL, and define it as is the scheduling decision space of the ith layer, represents the set of k.

[0053] u i is the loss function defined as follows:

[0054] represents the influence of layer i on other layers, represents the influence of layer j on other layers, which is called local altruism.

[0055] Definition 1: Given a scheduling decision a -i , the optimal response function b i of layer i -i (a i ) is the following set:

[0056]

[0057] Definition 2: The scheduling result is a pure decision Nash equilibrium if and only if At the pure decision Nash equilibrium point no layer can further reduce the loss by changing its decision while keeping other decisions unchanged, so we can obtain the optimal update scheduling scheme by finding the pure decision Nash equilibrium point.

[0058] where a′ i represents any scheduling decision of layer i, a′ i is not equal to a i , a -i represents the scheduling decision of all layers except the ith layer when reaching the Nash equilibrium solution, represents the pure decision Nash equilibrium point of all layers except the ith layer.

[0059] Proof of the existence of Nash equilibrium:

[0060] The existence of the pure decision Nash equilibrium of UCL is summarized by the following theorem 1.

[0061] Theorem 1: UCL has at least one pure decision Nash equilibrium and satisfies the finite improvement property.

[0062] Proof: First, we prove that this game is a complete potential game, and its potential function is:

[0063]

[0064] This function satisfies

[0065]

[0066] We prove this by deriving the right-hand side from the left-hand side:

[0067]

[0068] Therefore, UCL game is a complete potential game, and it is worth noting that a finite potential game has at least one pure strategy Nash equilibrium, and in addition, all better and optimal response sequences will converge to a pure strategy Nash equilibrium even if the starting point is different, that is, they guarantee the finite improvement property.

[0069] 2. Iterative optimization:

[0070] For layer i, in the τth iteration, if then select a i ∈b i (a -i (τ)) to update its scheduling decision to reduce its loss, and if a i (τ)∈b i (a -i (τ)), then nothing will be done, and the original scheduling decision will be maintained.

[0071] In an iteration, updating the scheduling decisions of multiple layers at the same time may make the system unstable, in order to avoid this problem, UCL only allows one iteration to update the scheduling decision of one layer:

[0072] In each iteration, the scheduling decision of a layer is selected for updating from the layers that satisfy .

[0073] Specifically, in an iteration, the layers that satisfy will all send an update request to UCL, and UCL will select the scheduling decision of a layer for updating according to the preset rule, and then enter the next iteration. Finally, if no layer requests updating, the iteration will end.

[0074] Wherein, the preset rule can adopt the roulette strategy.

[0075] It is worth noting that if two connected layers are scheduled to different units, a transmission from one unit to another will be generated. Since we assume that the transmission speed is always greater than the computation speed, many such transmissions can occur after scheduling. However, although we can obtain huge transmission bandwidth through a wired LAN, too many transmission procedures will introduce new factors that increase the delay, such as the thread scheduling of the operating system. Therefore, in order to make the experiment persuasive, we add some constraints to the scheduling decision-making process - in the initialization process (S102 and S103), the scheduling decision of each layer is calculated and formed in the order of the layers. That is, all layers can only be scheduled to the unit where the previous layer is located, or the unit that has not participated in the scheduling. In the iteration process, all layers can only request to be scheduled to the unit where the previous layer, the next layer, or no layer is located. With these constraints, the system becomes more stable.

[0076] In the two processes of initialization and iteration, the latter is related to the number of iterations R, the number of layers N and the number of computing units M. In each iteration τ of UCL, N layers calculate their optimal responses and request to update on M units. Therefore, the total number of calculations is R x N x M, that is, the time complexity of UCL is O(RNM). The running time of UCL only linearly increases when M or N increases. On the other hand, the time complexity of OCDST is O(M N ), which is an exponential function of N and a power function of M. It is undeniable that the hierarchical scheduling scheme derived by OCDST is definitely globally optimal, however, since it needs to examine all possible solutions in the solution space, its complexity is exactly the size of the solution space. If N = 100, M = 2, the number of iterations of OCDST will exceed 10 30 , however, an 8-core Intel Core I7-800 CPU can only iterate about 10 8 times per second when using C++ language. Therefore, in this case, the running time of OCDST will exceed 10 17 days. On the other hand, UCL only has linear time complexity, so if we set R = 10 4 and keep other parameters, the running time is only 0.02s.

[0077] S105, according to the scheduling decision of each layer after optimization and update, split and schedule the neural network to the target computing unit of the corresponding scheduling decision.

[0078] The method of the present application is applicable to CDNN, of course, it can also be generalized to GCNN (Group Convolutional Neural Network), which is described as follows.

[0079] As Figure 4As shown, G has three layers (layer 1, layer 2 and layer 3), where the second layer (layer 2) is divided into two sub-layers. For ease of analysis, we call each circle (layer or sub-layer) in the figure a node. G schedules the nodes to four computing units to perform streaming tasks. Each computing unit takes a certain amount of time to complete its computation and sends the intermediate result to the unit where the next node is located. In this example, t1, t4, t5, t8 are computation delays, and the others are transmission delays.

[0080] After the scheduling ends, G starts to receive task inputs. As shown, Figure 5 the first task (task 1) is first processed by the first layer, and then transmitted to the second layer. During the transmission, the first layer starts to perform task 2, and this process is repeated continuously. The same situation occurs in the second and third layers. All nodes of G will repeatedly receive the next task, perform the current task, and transmit the previous task. In addition, we assume that the transmission from one node to multiple nodes is also parallel, i.e., multiple nodes equally share the upload bandwidth of the previous node.

[0081] Next we derive the average processing time of the task stream. We assume that a layer starts to compute after all its sub-layers receive inputs, and starts to transmit after all computations are completed, which ensures that all nodes can be scheduled to units with matching resources. Therefore, each task is processed by the first layer in t1 time, and then sent to the second layer in t'2 = max(t2, t3) time. After the second layer experiences t'4 = max(t4, t5) processing time, the task will be sent to the third layer in t'6 = max(t6, t7) time and processed in t8 time. That is, G becomes a pipeline whose all computation and transmission times are t'2, t'4, t'6 and t8, and serves as its delay. The average delay of the pipeline processing the task stream is the maximum delay of all states, i.e., max(t1, t'2, t'4, t'6, t8) (or ).

[0082] Grouped convolution is derived from AlexNet. Therefore, if we can simplify AlexNet, it means that most GCNNs can be simplified. The simplification process is shown in Figure 6 (1) is the original AlexNet with grouped convolution, Figure 6 (1) can be converted into the node view of Figure 6 (2), Figure 6 Figure 6 ​​(2) The nodes in the two dashed boxes form four chains, and the scheduling model shows that the average delay of a chain is the maximum filter delay, thus, the chain can be abstracted as a node whose delay is the maximum filter delay, as shown in Figure 6 (3). Note that grouping convolution allows the grouping layer to cross-transmit data, which is not allowed in the basic GCNN structure, thus, we add two virtual nodes to break the cross point, as shown in Figure 6 (4). They are only responsible for receiving data from n1 and n2 and then sending them to n3 and n4. Since the delay of a grouped layer is the maximum computation or transmission time of its child layers, we further abstract the grouping layer as a node, finally, we simplify the complex GCNN to CDNN as shown in Figure 6 (5), thus, the method of the present application is applicable to GCNN.

[0083] The method of the present application is described below through specific experiments.

[0084] 1. Experimental configuration

[0085] Neural network: The neural networks involved in the experiment are YOLOv3

[28] , AlexNet, VGG-16 and ResNet-18. All neural networks except YOLOv3 are built in the PyTorch framework, thus we perform their splitting and scheduling based on the PyTorch implementation of YOLOv3 on GitHub. All neural networks first undergo an extraction process since their layers are encapsulated, and then the FLOPs of each layer are calculated.

[0086] Image set: The image set comes from the video frames of the BDD autonomous driving dataset, which is the largest open autonomous driving video dataset containing 100K videos and ten tasks for evaluating the progress of autonomous driving image recognition algorithms. The videos in this dataset are obtained from the cameras on the autonomous driving car, each video is about 40 seconds long, the resolution is 720p, and the frame rate is 30 frames per second.

[0087] Computing units: We conducted experiments on five hosts, two of which have 8-core Intel(R) core(TM) i7-8700 CPUs (3.20 GHz) and 64 GB of random access memory, and the other hosts are equipped with 8-core Intel(R) core(TM) i7-6820EQ CPUs (2.80 GHz) and 32 GB of memory, and the UE is a notebook computer with a 4-core Intel(R) core(TM) i5-4210M CPU (2.60 GHz) and 8 GB of memory. We ensured that they were in an experimental state - all programs except the necessary programs of the operating system will be closed to ensure that computing and transmission resources are exclusively for the scheduled neural network.

[0088] Network construction: Gigabit routers are used to build local area networks (LANs) with wired transmission bandwidths of 125 Mbytes / s, which are used to support real machine experiments, and experiments based on higher bandwidths are simulation experiments.

[0089] FLOPs: We obtain the FLOPs of each layer of the neural network through the python module "thop". It is also feasible to derive them theoretically. For example, the FLOPs of the convolutional layer with 3x3 or 1x1 filter, and the LeakyRelu layer (the main component of YOLOv3), are 18, 1, and the number of negative numbers, respectively.

[0090] FLOPS: The FLOPS of the computing unit are estimated by timing the execution time of a certain number of floating-point operations. To ensure the accuracy of the FLOPS, we directly measure the time of executing YOLOv3 on the host. When an image of size 3x480x640 is used as input, the FLOPs of YOLOv3 are 58596780900, and its execution delay on the two hosts is about 0.68s and 0.51s. Therefore, their computing power is 86.17GFLOPS and 114.90GFLOPS (1GFLOPS is 10 9 FLOPS).

[0091] Evaluation process: We start the experiment with YOLOv3, which is an accurate object detector, and we design a complete streaming task publishing and processing flow for it, in which the task source publishes continuous video frames at a speed of 30FPS, and the scheduled YOLOv3 detects objects on these frames at the highest speed, and the experimental results are represented by a graph. We also tested the generalization ability of UCL. In this experiment, we compared UCL, local operation (LOCAL), single-task model (SINGLE), and existing streaming task model (OCDST).

[0092] Notes: Since all the related neural networks take images as input in this experiment, one image (frame) is one task and a stream of images is one task stream here, and the inference speed of all the neural networks is measured in frames per second (FPS). OCDST cannot be applied to large-scale device networks and neural networks, so we simplify the scenario when testing OCDST. For example, we simplify YOLOv3 to have only 25 layers by considering multiple layers together, and then input it into OCDST. The experimental results can be affected by the operating system and transmission noise.

[0093] 2. Scheduled Yolov3 processing stream task

[0094] We deploy the task source on one of the hosts to ensure that the publishing speed of tasks (images) is fixed (30 FPS) and always greater than the detection speed, and is not affected by transmission noise. We test the amount of tasks processed in a specific time, first setting the time to 10 seconds, and we compare UCL with local operation (LOCAL), single-task model (SINGLE), and existing stream task model (OCDST). In 10 seconds, the frame rates of the four models are 83, 4, 19, and 37, respectively. The inference speed of UCL is more than twice that of OCDST, and far exceeds that of other models.

[0095] Then, we extend the time to 100 seconds to show the growth trend of the frames processed, as shown in Figure 7 The detection amount of all models shows a linear growth trend, but the growth rate of UCL is much faster than that of other models, and the superiority of UCL becomes more and more obvious as the allowed response time increases.

[0096] 3. Generalization measurement

[0097] In addition to YOLOv3, AlexNet, VGG-16, and ResNet-18 are also input into the four models for testing to verify the generalization of UCL. We first measure their inference speeds at the initial 125 Mbyte bandwidth, and the results are shown in Figure 8 . Figure 8 It shows that UCL is superior to other models in almost all cases, but when scheduling AlexNet, the performance of UCL is less than half of that of OCDST, because UCL assumes that the transmission speed is always greater than the computing speed, but the 125 Mbyte bandwidth is not enough to support this assumption. Therefore, we repeat this experiment on a 5G network with a bandwidth of 2.5 Gbyte, and the results Figure 9 show that UCL performs best at a suitable transmission speed.

[0098] Based on the same inventive concept, the embodiments of the present specification also provide an electronic device, comprising a storage module, the storage module comprising instructions loaded and executed by a processor, the instructions, when executed, causing the processor to perform the steps described in the above-mentioned neural network splitting scheduling method for streaming tasks according to various exemplary embodiments of the present application.

[0099] The storage module can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) and / or a cache memory unit, and can further include a read-only memory (ROM).

[0100] Based on the same inventive concept, the embodiments of the present specification also provide a computer-readable storage medium storing one or more programs, the one or more programs, when executed by a processor, implementing the steps described in the above-mentioned neural network splitting scheduling method for streaming tasks according to various exemplary embodiments of the present application.

[0101] The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, and the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet service provider. The present application can be a system, a method, an apparatus, a computer program product, or any combination of the above.

[0103] However, it should be appreciated by those skilled in the art that the above-described embodiments are merely given by way of illustration and are not used to limit the present application, and any changes, variations and modifications of the above-described embodiments within the spirit and scope of the present application will be embraced by the claims of the present application.

Claims

1. A neural network split scheduling method for a streaming task, characterized in that, Comprise: S101, determine the amount of calculation of each layer of neural network, determine the calculation speed of each calculation unit; S102, calculate the single calculation delay of each layer scheduled to each calculation unit by the following formula: wherein c i ∈ C = {c1, c2,..., c N} denotes the computation amount of layer i, denotes the computation speed of computation unit k, denotes the single computation latency of layer i scheduled to computation unit k; S103, take the calculation unit with the minimum single calculation delay of each layer as the target calculation unit of the layer, and form the scheduling decision of each layer; S104, optimize and update the scheduling decision: find the Nash equilibrium through potential game of each layer, so as to make the global running time of each layer shortest: Potential game construction: Let and denote the layers and the computational units of the neural network, respectively, and let a i denote the scheduling decision for layer i. Let k denote the total scheduling decision, a = (a1, a2,..., ak). N ) denote the total scheduling decision. The potential game is defined as is the scheduling decision space of the i-th layer, u i is the loss function defined as follows: the influence of layer i on other layers, the influence of layer j on other layers; Definition 1 : The optimal response function b -i of layer i for a given scheduling decision a i (a -i ) is the set of Definition 2: A scheduling outcome is a pure-strategy Nash equilibrium if and only if is a pure-strategy Nash equilibrium if and only if At a pure-strategy Nash equilibrium point No layer can further reduce its loss by changing its strategy while keeping the strategies of the other layers. wherein a i represents any scheduling decision of layer i, a i i does not equal a -i represents the scheduling decision of all layers except layer i when reaching a Nash equilibrium solution, represents the pure strategy Nash equilibrium point of all layers except layer i;​ Iterative optimization: For layer i, in the τ-th iteration, if then select a i ∈b i (a -i (τ)) to update its scheduling decision to reduce its loss, if a i (τ)∈b i (a -i (τ)), then keep the original scheduling decision; S105, according to the scheduling decision of each layer after optimization and update, split and schedule the neural network to the target calculation unit of the corresponding scheduling decision. 2.The method for neural network splitting scheduling of streaming tasks according to claim 1, characterized in that, The iterative optimization further comprises only allowing to update the scheduling decision of one layer in one iteration: In each iteration, the scheduling decision of one tier is updated according to preset rules from tiers satisfying .

3. The method of claim 2, wherein, The preset rule adopts roulette strategy.

4. The method of claim 1, wherein, The determination of the calculation speed of each calculation unit further comprises: Obtain the corresponding calculation speed from each calculation unit.

5. The method of claim 1, wherein, The neural network is CDNN or GCNN.

6. The method of claim 5, wherein, Also include simplifying GCNN to CDNN.

7. An electronic device, comprising: A storage module, the storage module comprises instructions loaded and executed by the processor, the instructions when executed cause the processor to execute a neural network split scheduling method for streaming tasks according to any one of claims 1-6.

8. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions executable by one or more processors to perform all or a subset of the steps of any of methods 1-7. The one or more programs, when executed by the processor, implement the neural network split scheduling method for streaming tasks according to any one of claims 1-6.

Images