Parallel computing method and system for edge clusters

Abstract

The present invention provides a parallel computing method and system for edge clusters, the method comprising: obtaining the computing power of each computing device in the edge cluster; dividing the multi-layer convolutional neural network including the main branch and the sub-branch into multiple multi-fusion layer blocks in a multi-fusion layer block manner, allocating a workload corresponding to the computing power of each computing device to each computing device according to the computing power of each computing device obtained, and when each computing device executes the calculation of the multi-fusion layer block in the main branch and reaches the sub-branch, saving the calculation result of the current main branch, and executing the convolution layer of the sub-branch at the current position by each computing device based on the calculation result of the current main branch, and executing the fully connected layer by the computing device with the strongest computing power and obtaining the entropy value of the input sample according to the execution result and comparing the entropy value with the preset confidence threshold to determine whether to continue to execute the main branch or exit the calculation. The present invention can optimize the effect of executing existing large neural networks on resource-constrained devices.

Description

Technical Field

[0001] The present invention relates to the field of deep neural network technology, and in particular to a scalable parallel computing method and device for edge clusters. Background Art

[0002] Deep Neural Networks (DNN) have become an indispensable tool for solving complex problems such as image recognition and natural language processing. While DNN provides powerful feature extraction capabilities, it also places higher demands on the computing power of devices. At the same time, the number of IoT devices has grown exponentially in recent years. In order to cope with the data explosion caused by the exponential growth of IoT devices, edge computing has emerged. Edge computing refers to performing computing tasks at the edge of the network where data is generated, such as terminal devices or edge servers, to provide computing services with low latency. However, it is difficult for edge devices to quickly execute large DNN models, and applications in edge computing scenarios generally have high requirements for task latency. Running deep neural networks on resource-limited edge devices will bring unacceptable latency, which hinders artificial intelligence from empowering edge devices. In edge computing scenarios, edge devices usually have different latency and accuracy requirements for different types of tasks. Taking traffic cameras as an example, the task of detecting traffic jams has high latency requirements and low accuracy requirements; the task of recognizing license plates has high accuracy requirements and low latency requirements. Therefore, it is very necessary to provide scalable deep learning services according to application requirements. However, current deep learning models are usually not scalable, their inference latency is fixed, and they are difficult to dynamically adapt to tasks with different requirements.

[0003] The Chinese patent application with patent application number CN202010100106.4 proposes a parallel reasoning method for neural network encoding and decoding tools, which includes: expanding the boundaries of the image to be processed to obtain an image with complete boundaries; dividing the image with complete boundaries into image blocks of the same size according to the raster scanning order; organizing the image blocks of the same size into several tensors, and sending the several tensors as a group of tensors into the neural network for forward propagation processing at the same time, wherein the batch size value of the tensor is set according to the neural network encoding tool and the hardware video memory size. This method can reduce the occupation of GPU video memory during the execution of the neural network, but there is still much room for improvement in accelerating the speed of neural network reasoning.

[0004] In order to improve the inference speed of deep learning models on resource-constrained devices, people often offload computing tasks to the cloud or edge servers. En Li et al. proposed a framework for terminal devices and edge servers to collaboratively execute DNN (see "On-demand accelerating deep neural network inference via edge computing." IEEE Transactions on Wireless Communications 19(1), 447–457(2020)). The terminal device executes the complex front end of the DNN and then sends the intermediate data to the edge server, which performs the remaining computing tasks. Since the computing power of the edge server is much greater than that of the terminal device, the DNN task can be completed quickly. Although this method can effectively reduce the overall task latency, its performance is highly dependent on the network quality between the terminal device and the remote server. When the network quality between the device and the server decreases, the execution speed of the DNN task also drops sharply. In addition, this server-assisted method requires sending the intermediate data to the edge server or cloud server, which may cause the leakage of private data.

[0005] The complex structure of the DNN model is the fundamental reason why it requires a lot of computing resources. Therefore, another method to accelerate the DNN inference speed focuses on optimizing the DNN model structure to reduce the required amount of calculation. For example, pruning the original DNN model, removing the weights with low importance in the model to reduce the amount of calculation; or using low-bit weights instead of the original floating-point weights. This method effectively reduces the model's demand for device memory. However, while reducing the amount of model calculation, this will greatly reduce the model's feature extraction capabilities, and the model's accuracy will be greatly attenuated.

[0006] Mao et al. proposed to distribute the reasoning tasks to multiple devices for parallel computing (see "Modnn: Local distributed mobile computing system for deep neural network." In: Design, Automation Test in Europe Conference Exhibition (DATE), 2017. pp. 1396–1401 (2017)), thereby accelerating the reasoning speed of DNN on devices with weak computing power. However, in the task distribution process, they adopted a hierarchical approach, that is, each device needs to exchange its own data after executing each layer of the DNN model to remove the data dependency problem. This hierarchical approach causes frequent communication between devices, and the computing speed is also highly dependent on the network quality.

[0007] Therefore, how to improve the running speed of neural network models on resource-constrained edge cluster devices while ensuring the accuracy of neural network models and provide scalable deep neural networks according to task requirements has become an urgent problem to be solved. Summary of the invention

[0008] In view of this, an embodiment of the present invention provides a scalable parallel computing method and device for edge clusters to eliminate or improve one or more defects in the prior art.

[0009] One aspect of the present invention provides a scalable parallel computing method for edge clusters, the method comprising the following steps:

[0010] A computing power acquisition step is used to obtain the computing power of each computing device in the edge cluster; a multi-fusion layer block generation step is used to divide the multi-layer convolutional neural network including the main branch and the sub-branches corresponding to the different exit points of the preset main branch into a multi-fusion layer block, and for each convolution layer of the divided multi-fusion layer block, a workload corresponding to its computing power is allocated to each computing device according to the computing power of each computing device obtained, so that the maximum difference between the computing time of each computing device executing the continuous convolution layer in the current multi-fusion layer block and the average computing time is less than a predetermined synchronization threshold; each computing device exchanges data between computing devices after completing the task of multiple continuous convolution layers in a multi-fusion layer block; a branch selection step, when each computing device executes the calculation of the multi-fusion layer block in the main branch and reaches the sub-branch, the calculation result of the current main branch is saved, and based on the calculation result of the current main branch, each computing device executes the convolution layer of the sub-branch at the current position, and the computing device with the strongest computing power executes the fully connected layer, and obtains the entropy value of the input sample according to the execution result of the fully connected layer, and determines whether to continue to execute the main branch or exit the calculation according to the comparison result of the obtained entropy value with the preset confidence threshold.

[0011] In some embodiments of the present invention, the multi-fusion layer block generation step includes: an initial multi-fusion layer block setting step: setting a multi-fusion layer block including at least one convolutional layer for the multi-layer convolutional neural network; a multi-fusion layer block expansion step: estimating the redundant data calculation overhead of the multi-fusion layer block based on the computing power of each computing device obtained, calculating the communication saving overhead based on the bandwidth of the network where the edge cluster is located, and expanding the multi-fusion layer block based on the calculated redundant data calculation overhead and communication saving overhead, so as to end the convolution layer when the redundant data calculation overhead is greater than the communication saving overhead and the adjacent convolution layer behind the end convolution layer is not an exit point or a multi-layer convolutional neural network. When the last layer of the network is reached, the convolution layer adjacent to the end convolution layer is added to the multi-fusion layer block and used as the end convolution layer; a task allocation step: according to the obtained computing power of each computing device, a workload corresponding to its computing power is allocated to each computing device participating in the calculation, so that the maximum difference between the computing time of each computing device executing the continuous convolution layers in the current multi-fusion layer block and the average computing time is less than a predetermined synchronization threshold; the task allocation step and the multi-fusion layer block expansion step are repeated until the redundant data calculation overhead is greater than or equal to the communication saving overhead, the expansion reaches the exit point or the expansion reaches the last layer of the multi-layer convolutional neural network.

[0012] In some embodiments of the present invention, the task allocation step includes: the computational amount of the workload allocated to the computing device is the product of the ratio of the computing power of the computing device to the sum of the computing powers of all computing devices participating in the calculation and the computational amount of the total workload; the workload allocated to the computing device is the workload of the ending layer of the current fusion layer block, and the workload of other convolution layers of the current fusion layer block except the ending layer is expanded layer by layer from the workload of the ending layer toward the starting layer of the current fusion layer block to obtain the workload of all convolution layers of the current fusion layer block, and the workload of each convolution layer is expanded in a manner of expanding the workload boundary of the convolution layer adjacent to the ending layer outward by half the size of the convolution kernel.

[0013] In some embodiments of the present invention, the task allocation step also includes: when the maximum difference between the calculation time of each computing device executing the continuous convolutional layers in the current multi-fusion layer block and the average calculation time is greater than a predetermined synchronization threshold, the workload of the computing device corresponding to the maximum workload is reduced and the workload of the computing device corresponding to the minimum workload is increased until the maximum difference between the calculation time and the average calculation time is less than the predetermined synchronization threshold.

[0014] In some embodiments of the present invention, the multi-fusion layer block generation step also includes: when the expansion of a multi-fusion layer block is completed, all computing devices involved in the calculation send the calculation results to the computing device with the strongest computing power for aggregation, and the computing device redistributes the workload based on the calculation results, and then starts the expansion of the next multi-fusion layer block, and repeats the multi-fusion layer block generation step until the last layer in the multi-fusion layer block is the last layer of the main branch or sub-branch of the multi-layer convolutional neural network.

[0015] In some embodiments of the present invention, the multi-fusion layer block expansion step also includes: calculating the time used to transmit all convolutional layers except the starting layer and the ending layer in the multi-fusion layer block as the communication saving overhead; calculating the time taken by the computing device to calculate the expansion amount of all convolutional layers except the ending layer in the multi-fusion layer block as the redundant data calculation overhead.

[0016] In some embodiments of the present invention, the method for calculating the confidence threshold is as follows: recording the output entropy value of all test data in each branch when training a multi-layer convolutional neural network; calculating feasible solutions for the exit probabilities of multiple groups of exit points based on the conditions that the sum of the product of the calculation time of the multi-layer convolutional neural network in calculating each branch and the exit probability of the corresponding exit point is less than or equal to the maximum calculation time required for the multi-layer convolutional neural network and the sum of the exit probabilities of all exit points is 1, and each group of feasible solutions includes the exit probabilities of all preset exit points; the maximum calculation time is set based on the delay performance requirements or accuracy performance requirements of the multi-layer convolutional neural network; recording the calculation accuracy of the multi-layer convolutional neural network at each exit point and calculating the sum of the calculation accuracy of each exit point and the calculation accuracy of all exit points weight ratio; multiply each exit point probability of each group of feasible solutions by the weight ratio corresponding to its exit point and sum them to obtain the score of each group of feasible solutions, and select a group of feasible solutions with a high score; when the entropy value obtained by the method of calculating the entropy value is proportional to the calculation accuracy, select the maximum value of the output entropy value of each test data, and the ratio of the number of digits of all the maximum values ​​obtained when arranged in descending order to the number of all the maximum values ​​is equal to the entropy value of the exit point probability as the confidence threshold of the exit point, or when the entropy value obtained by the method of calculating the entropy value is inversely proportional to the calculation accuracy, select the minimum value of the output entropy value of each test data, and the ratio of the number of digits of all the minimum values ​​obtained when arranged in ascending order to the number of all the minimum values ​​is equal to the entropy value of the exit point probability as the confidence threshold of the exit point.

[0017] In some embodiments of the present invention, the method further includes training a multi-layer convolutional neural network with branches; the computing power acquisition step is to obtain a computing power linear regression model of each device after running multiple convolutional layers with different parameters for each device in the edge cluster.

[0018] Another aspect of the present invention provides a scalable parallel computing system for edge clusters, the system comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the method described in any one of the above embodiments.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0020] The parallel computing method and device for edge clusters provided by the present invention can provide edge devices with the ability to balance the accuracy and responsiveness of multi-layer convolutional neural networks, thereby adjusting the response time of multi-layer convolutional neural networks according to different tasks, further improving the operating speed of the prior art in edge computing scenarios, and overcoming the problem of the lack of scalability of deep neural networks in the prior art.

[0021] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0022] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0024] Figure 1 The present invention is a logical flow chart of a parallel computing method for edge clusters in one embodiment of the present invention.

[0025] Figure 2 The figure is a computational diagram of a scalable parallel computing method for edge clusters in one embodiment of the present invention.

[0026] Figure 3 Schematic diagram of data dependency in one embodiment of the present invention.

[0027] Figure 4 Schematic diagram of a multi-layer fusion method in one embodiment of the present invention.

[0028] Figure 5The figure is a performance comparison diagram of the present invention and the existing parallel computing method in one embodiment of the present invention.

[0029] Figure 6 This is a comparison chart of average computing delay performance based on different neural networks in one embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0031] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0032] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0033] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0034] To solve one or more problems in the prior art, the present invention provides a scalable parallel computing method and device for edge clusters.

[0035] The present invention proposes a scalable multi-device parallel computing framework with on-demand exit in view of the problems that the calculation depth of the multi-layer convolutional neural network cannot be dynamically adjusted according to the task requirements and the calculation speed is low in the traditional calculation of the multi-layer convolutional neural network. The computing resources of idle heterogeneous devices in the edge environment are fully utilized, and multiple devices jointly participate in the multi-layer convolutional neural network reasoning task, thereby improving the execution speed of the multi-layer convolutional neural network task on the device with weak computing power. Specifically, branches are added to different convolutional layer positions of the multi-layer convolutional neural network. If the exit condition of the branch is met after entering the branch during the calculation process and exits, the total computing delay will be reduced. Therefore, the addition of branches can make the multi-layer convolutional neural network with branches dynamically adapt to the low delay requirement or high accuracy requirement of the task, so as to realize the dynamic adjustment of the calculation depth (i.e. scalability) of the multi-layer convolutional neural network. For the optimized calculation speed of the multi-layer convolutional neural network with the same depth, the present invention adopts the method of calculating the multi-layer convolutional neural network with branches in the form of layer fusion blocks and distributing the workload according to the computing power of the computing device. The method is less affected by the network interference and reduces the total task delay. In addition, the data of the present invention is only transmitted in the edge cluster, avoiding sending the data to the remote server and thus avoiding the risk of data leakage.

[0036] like Figure 1 The figure shows a logic flow chart of the implementation of the parallel computing method for edge clusters of the present invention. The implementation of the parallel computing method of the present invention includes three steps: a computing power acquisition step S110, which is used to obtain the computing power of each device in the edge cluster; a multi-fusion layer block generation step S120, which is used to divide the multi-layer convolutional neural network including the main branch and the sub-branches corresponding to the different exit points of the preset main branch into multi-fusion layer blocks, and for each convolution layer of the divided multi-fusion layer blocks, according to the computing power of each computing device, the workload corresponding to its computing power is allocated to each computing device according to the computing power of each computing device, so that the computing time of each computing device to execute the continuous convolution layer in the current multi-fusion layer block is equal to the average computing time. The maximum difference between them is less than the synchronization point waiting time (predetermined synchronization threshold); each computing device exchanges data between computing devices after completing the task of multiple consecutive convolutional layers in a multi-fusion layer block; branch selection step S130, when each computing device executes the calculation of the multi-fusion layer block in the main branch and reaches the sub-branch, saves the calculation result of the current main branch, and executes the convolution layer of the sub-branch at the current position based on the calculation result of the current main branch, and the computing device with the strongest computing power executes the fully connected layer, and obtains the entropy value of the input sample according to the execution result of the fully connected layer, and determines whether to continue to execute the main branch or exit the calculation according to the comparison result of the obtained entropy value with the preset confidence threshold.

[0037] In the present invention, for the convenience of description, the workload is a picture, and the computing task is picture recognition, but the present invention is not limited to this. The focus of the present invention is how to optimize the calculation of a multi-layer convolutional neural network in an edge cluster, and is not limited to the use of a specific multi-layer convolutional neural network. The present invention is applicable to a variety of existing multi-layer convolutional neural networks such as AlexNet, VGG, ResNet and other multi-layer convolutional neural networks. After the variant (i.e., optimizing the calculation in the edge cluster for different multi-layer convolutional neural networks), the present invention can be applied to the field of target recognition or speech processing, etc. Here, the convolutional neural network in the field of image classification is taken as an example.

[0038] Next, the computing power acquisition step S110 of the present invention is described in detail. Secondly, the computing power acquisition step is the preparatory stage of the present invention. In this stage, a multi-layer convolutional neural network with branches is also trained, and the output entropy value of the training data is recorded, and the confidence threshold is calculated based on the output entropy value. The confidence threshold calculation step is step S140, which is described in detail below.

[0039] Computing power acquisition step S110: In some embodiments of the present invention, since edge device clusters are generally heterogeneous, different devices have different computing power and network resources. Therefore, it is necessary to reasonably allocate multi-layer convolutional neural network reasoning tasks to various heterogeneous devices. Based on the above requirements, it is necessary to obtain the computing power of each computing device in the edge cluster in the preparation stage. Since the computing power of the computing device is fixed, each computing device only needs to train its computing power once. The way to obtain the computing power of the computing device in the edge cluster is: each computing device D in the edge cluster k ={D1,D2,…,D K}Run a series of convolutional layers with different parameters to train its computing power. Taking the workload as an example, for the feature map M{Ch M ,H,W}, the convolution kernel is K{Ch K ,F,F}, the computational FLOPS can be expressed as:

[0040] FLOPs=2HW(Ch M F 2 +1)Ch K (1)

[0041] Where H and W refer to the height and width of the feature map, F refers to the size of the convolution kernel, and Ch M and Ch K They refer to the number of input and output channels of the convolutional neural network respectively. From this, the computing power of the computing device can be trained to obtain the linear regression model C k , then for a specific convolutional layer L, its execution time is expressed as C k(L), then each computing device will broadcast the trained computing power model to other devices participating in collaborative reasoning, and eventually each device will obtain the computing power value of other devices. After that, the device that initiates the multi-layer convolutional neural network reasoning task will allocate workload to each device according to its computing power. However, the present invention is not limited to this. For example, in other embodiments of the invention, the hardware information of the computing device in the edge cluster is read, and its computing power is obtained based on its processor and memory.

[0042] Calculate confidence threshold step S140: In order to meet the needs of different real-time tasks, we take advantage of the ability to balance latency and accuracy for the device (scalability of parallel computing methods). The early exit mechanism adds branches to the original multi-layer convolutional neural network, so that simple input samples can calculate highly reliable results in the shallow layers of the multi-layer convolutional neural network. How to determine the confidence threshold of each exit branch according to the task latency requirements is the problem we need to solve. For each exit branch n{1,2,…,N}, the probability of exiting at this exit point is P n , based on the computing power model of each device and the network bandwidth quality obtained in the preparation phase, we can predict the execution time T of each branch n The user sets T based on the latency performance requirements or accuracy performance requirements of the multi-layer convolutional neural network. th ; then Pn needs to satisfy:

[0043]

[0044] That is: according to the calculation time T of each branch of the multi-layer convolutional neural network n The exit probability P of the exit point corresponding to this branch n The sum of the products is less than or equal to the maximum computation time T required for the multi-layer convolutional neural network th , and the conditional simultaneous equations whose sum of the exit probabilities of all exit points is 1, calculate the feasible solution P of the exit probabilities of multiple groups of exit points, and the calculation time of the calculation branch is T n is the computation time of all convolutional layers from the first layer of the multi-layer convolutional neural network to the fully connected layers of the branch. Each set of feasible solutions includes the exit probabilities of all preset exit points; Figure 2 Taking the multi-layer convolutional neural network with branches as an example, a feasible solution is P = (40%, 40%, 20%), that is, the exit probability of branch 1 and branch 2 is 40%; there is a 20% probability of not exiting early in the calculation until the main branch is executed.

[0045] The entire training data set is calculated at each exit point, the output entropy of all test data in each branch when training the multi-layer convolutional neural network is recorded, and the entropy list L of each sample is recorded.

[0046] Since the design concept of the present invention tends to exit the calculation as late as possible before the calculation time reaches the maximum calculation time (so as to have a higher calculation accuracy in meeting the calculation time requirement), the following feasible solution selection scheme is designed:

[0047] Record the calculation accuracy of each exit point of the multi-layer convolutional neural network during the training phase, denoted as Acc1, Acc2, ..., AccN. And calculate the weight ratio ω of the calculation accuracy of each exit point branch to the sum of the calculation accuracy of all exit points n Expressed as

[0048] The probability P of each exit point of each feasible solution is n The weight ratio ω corresponding to its exit point n Multiply and sum to get the score of each feasible solution, and select a feasible solution P with a high score;

[0049] In the actual calculation process, in one embodiment of the present invention, the entropy value of the branch calculation result is expressed as:

[0050]

[0051] where y is a vector containing the calculated probabilities for all possible class labels, is the set of all results of the data set. Input samples with entropy values ​​greater than the confidence threshold can exit the network here. Since the parallel computing method designed by the present invention has multiple synchronization points, this structure has good adaptability to the added branches. The entropy value obtained by the method of calculating the entropy value is inversely proportional to the calculation accuracy. The minimum value in the output entropy value of each test data is selected, and the ratio of the number of bits of all the minimum values ​​obtained when arranged in ascending order to the number of all minimum values ​​is equal to the entropy value of the probability of the exit point as the confidence threshold of the exit point, and the confidence threshold is set for all exit points according to this method.

[0052] In another embodiment of the present invention, the entropy value obtained by calculating the entropy value is proportional to the accuracy of calculating the entropy value, the maximum value among the output entropy values ​​of each test data is selected, and the ratio of the number of bits of all the maximum values ​​obtained when arranged in descending order to the number of all the maximum values ​​is equal to the entropy value of the probability of the exit point as the confidence threshold of the exit point, and the confidence threshold is set for all exit points in this way.

[0053] To introduce the multi-fusion layer block generation step S120, the reason why the present invention selects the multi-fusion layer block should be described first: in some embodiments of the present invention, such as Figure 3As shown in the figure, due to the inherent tight coupling of the convolutional neural network structure, assigning the convolutional reasoning process of the multi-layer convolutional neural network to multiple devices will generate data dependencies between the devices. Assuming that the number of devices involved in the calculation is three, the input feature map is assigned to three devices. After a 2*2 convolution kernel performs a convolution operation, the output feature map is as follows Figure 3 To get 12 in the output data on the right, the content of the 2*2 matrix in the input data on the left is needed (square box), and these data are stored in two different devices. To overcome the convolution kernel of F*F for the computing device in the edge cluster, each device needs its assigned feature map partition to expand along the edge. to contain the data required for convolution calculations.

[0054] Each device can thus execute multiple convolutional layers in succession without exchanging overlapping data, thus avoiding additional communication overhead. When allocating tasks, we first divide the feature map of the last convolutional layer of the fusion block according to the device computing power and network bandwidth, and then the workload of each convolutional layer in the fusion layer block is expanded outward along the edge of the workload of the adjacent convolutional layer behind. The workload obtained after that, that is, the workload of each convolutional layer, is expanded by half the size of the convolution kernel from the workload boundary of the adjacent convolutional layer toward the end layer. And it is recursively expanded layer by layer to the starting layer (that is, the first layer) of the fusion layer block, so as to obtain the workload of all convolutional layers in the fusion layer block.

[0055] In one embodiment of the present invention, the layer fusion method causes the device to perform redundant calculations on overlapping data. And as the number of fusion layers increases, the amount of redundant calculations will also increase layer by layer. Therefore, we have made a trade-off between additional communication and redundant calculations, and adopted a multi-fusion layer block method to reduce the large amount of redundant calculations caused by too many fusion layers. Its structure is as follows: Figure 4 As shown, Figure 4 The left side of the figure shows a single fusion layer block, and the right side shows a multi-fusion layer block. The multi-fusion layer block divides the entire multi-layer convolutional neural network into multiple blocks. Each device continuously performs tasks within a block, and then each device reaches a synchronization point. At the synchronization point, each device will summarize the feature graph at this time and summarize it to the computing device with the strongest computing power in the edge cluster, and it will redistribute it. By comparison Figure 4 As can be seen from the left and right figures, in order to calculate the fourth layer, the amount of computation of a single device using the single fusion layer block method is greater than that of the multi-fusion layer block. This is because, although the workload of each convolutional layer is expanded along the edge of the workload of the adjacent convolutional layer behind (in this embodiment, the feature map), However, when the workload of the starting layer in the fusion layer block is expanded, the workload of the adjacent convolutional layer behind it is large, so the expansion amount is also large (for example, the larger the area of ​​the image, the larger the side length. For regular polygons, the expansion amount is the number of sides, side length, and The more redundant calculations are caused, the more obvious this phenomenon is when there are too many convolutional layers in the fusion layer block. The single fusion layer block method can be understood as having only one fusion layer block when calculating a multi-layer convolutional neural network, where the workload of each convolutional layer is extended by the workload of the adjacent convolutional layer behind it. Therefore, the overall amount of redundant calculations generated by the single fusion layer block method is quite huge. The multi-fusion layer block method proposed in the present invention can effectively avoid a large amount of redundant calculations caused by fusing too many layers.

[0056] Another factor that affects the execution time of each independent fusion layer block is the delay caused by data transmission. Since edge devices are usually in the same network, in this invention, our research focuses on edge clusters in the same local area network. We use B to represent the bandwidth of the local area network where the edge cluster is located. We adopt a bottom-up communication data size analysis method, that is, first calculate the size of the last layer of the fusion layer block. If device D k The size of the workload of the last layer of the layer fusion block is W end , then the amount of data required to transmit the last layer is 4W end Bytes, because each floating point number is 4 bytes. According to the expansion of the last layer of feature maps layer by layer The size of the feature map of the first layer of the fusion layer block is W first Then device D k The total time to execute a fusion layer block with V fusion layers can be expressed as:

[0057]

[0058] The multi-fusion layer block generation step S120 is used to divide the multi-layer convolutional neural network including the main branch and the sub-branches corresponding to the different exit points of the preset main branch into multi-fusion layer blocks. For each convolution layer of the divided multi-fusion layer blocks, a workload corresponding to the computing power of each computing device is allocated to each computing device according to the computing power of each computing device obtained, so that the maximum difference between the computing time of each computing device executing the continuous convolution layer in the current multi-fusion layer block and the average computing time is less than a predetermined synchronization threshold; each computing device exchanges data between computing devices after completing the tasks of multiple continuous convolution layers in a multi-fusion layer block. The multi-fusion layer block generation step is based on the computing device D participating in the calculation. k , computing capability model of each device C k , the trained multi-layer convolutional neural network with branches is used to carry out the training. The output is the fusion layer block.

[0059] The multi-fusion layer block generation step S120 is further divided into an initial multi-fusion layer block setting step S121, a multi-fusion layer block expansion step S122, and a task allocation step S123.

[0060] First, the initial multi-fusion layer block setting step S121 is to set a multi-fusion layer block including at least one convolutional layer for the multi-layer convolutional neural network; it is known that the number of devices participating in this parallel calculation is M, and each device is denoted as D k , where k = 1, 2, ..., M. In one embodiment of the present invention, the first layer of the multi-layer convolutional neural network is firstly used as the first layer L of the fusion layer block first , the last layer of the fusion layer block (ending the convolutional layer) L end Point to L first , at this time, the multi-fusion layer block has only one convolutional layer.

[0061] Secondly, in the multi-fusion layer block expansion step S122:

[0062] When the convolutional layer behind the end convolutional layer is not the convolutional layer where the exit point is located in the main branch or is not the last convolutional layer of each branch: expand backward L end , denoted as L end =L end+1 , that is, the convolution layer adjacent to the ending convolution layer is added to the multi-fusion layer block and used as the ending convolution layer; at this time, the fusion layer block has two convolution layers.

[0063] Whenever the number of convolutional layers of the fusion layer block increases, the task allocation step S123 is executed. At this time, step S123 performs task allocation based on the current fusion layer block. The size of the fusion layer block is expressed as {L first , L end}:

[0064] The task allocation step S123 is specifically as follows: according to the obtained computing capabilities of each computing device, a workload corresponding to its computing capability is allocated to each computing device participating in the calculation. The workload allocated to the computing device is the workload of the end layer of the current fusion layer block. The workload of other convolution layers of the current fusion layer block except the end layer is expanded layer by layer from the workload of the end layer toward the starting layer of the current fusion layer block to obtain the workload of all convolution layers of the current fusion layer block. The expansion amount of the workload of each convolution layer is the workload of the workload boundary extending outward by half the convolution kernel size.

[0065] The amount of computational workload allocated to each computing device is the product of the ratio of the computing power of the computing device to the sum of the computing power of all computing devices participating in the calculation and the amount of computational workload of the total workload; specifically expressed as And calculate T according to formula 4k Calculate the time it takes for all computing devices to calculate the current workload Note the execution time of each device T k With the average time T avg The difference is To make is less than the predetermined synchronization threshold, that is, to make the computing time of the slowest computing device among all computing devices equal to the average computing time T avg The phase difference is greater than the predetermined synchronization threshold, which is not accepted in the present invention. When the synchronization threshold is greater than the predetermined synchronization threshold, the steps of reducing the workload of the device with the largest workload by one pixel and increasing the workload of the device with the smallest workload by one pixel are repeated. Is it less than a predetermined synchronization threshold? If the relationship is less than, stop this step. The predetermined synchronization threshold is the sum of the waiting factor ζ and T set at the synchronization point. avg That is, when the maximum difference between the computing time of each computing device executing the continuous convolutional layers in the current multi-fusion layer block and the average computing time is greater than the predetermined synchronization threshold, the workload of the computing device corresponding to the maximum workload is reduced and the workload of the computing device corresponding to the minimum workload is increased until the maximum difference between the computing time and the average computing time is less than the predetermined synchronization threshold.

[0066] When the number of fusion layers of the multi-fusion layer block is increased each time and S123 is executed, step S122 is continued to be executed, specifically:

[0067] Based on the computing capacity C of each computing device obtained k Estimate the redundant data W of the multi-fusion layer block oc The computational cost is expressed as C k (W oc ), that is, the time taken by the computing device to calculate the expansion amount of all convolutional layers except the end layer in the multi-fusion layer block is used as the redundant data calculation overhead. oc The specific method for obtaining is that the device initiating parallel computing in the task load distribution algorithm distributes the workload W according to the computing power of the computing device. k , W k Represents the workload of the last layer of the current fusion layer block, the redundant data W oc For each convolutional layer W k The sum of the expansion amounts of F / 2 at the edge expansion. In one embodiment of the present invention, the redundant data W of all computing devices participating in the calculation ocThe redundant data computing overhead with the longest computing time is selected from the computing overheads to participate in subsequent calculations, that is, the computing device takes longer to calculate the redundant data than other computing devices.

[0068] The communication cost saving is calculated based on the bandwidth B of the network where the edge cluster is located, which is expressed as That is, the time used to transmit all convolutional layers except the starting layer and the ending layer in the multi-fusion layer block is calculated as the communication saving cost. In one embodiment of the present invention, the communication saving cost with the longest transmission time is selected from the communication saving costs of all computing devices to participate in subsequent calculations.

[0069] The multi-fusion layer block is extended based on the redundant data calculation overhead and communication saving overhead. Specifically, when C k (W oc ) is greater than When the expansion of the multi-fusion layer block is completed, all computing devices involved in the calculation will send the calculation results to the computing device with the strongest computing power for aggregation, and the computing device will redistribute the workload based on the calculation results before starting the expansion of the next multi-fusion layer block. The multi-fusion layer block generation steps are repeated until the last layer in the multi-fusion layer block is the last layer of the main branch or sub-branch of the multi-layer convolutional neural network.

[0070] In summary, when the redundant data calculation overhead is greater than the communication saving overhead and the convolution layer behind the ending convolution layer is not the exit point or the last layer of the multi-layer convolutional neural network, the multi-fusion layer block is expanded by adding the adjacent convolution layer behind the ending convolution layer to the multi-fusion layer block and using it as the ending convolution layer; when the adjacent convolution layer behind the ending layer is the last layer of the branch, the multi-fusion layer block is stopped from being expanded and the fusion layer fast generation step is stopped; when the adjacent convolution layer behind the ending layer is the convolution layer where the exit point of the branch is located or the redundant data calculation overhead is greater than the communication saving overhead, the multi-fusion layer block is stopped from being expanded and the next round of multi-fusion block generation step is started.

[0071] Next, the branch selection step S130 is described: Figure 2 The branch selection step is described in detail. Figure 2The figure shows a calculation schematic diagram of the parallel computing method for edge clusters of the present invention. In this embodiment, three computing devices in the edge cluster participate in the parallel calculation of the multi-layer convolutional neural network with two exit points, but the present invention is not limited to this. Multiple exit points and corresponding branches can be added artificially in the multi-layer convolutional neural network. First, device 1 obtains a computing task of image recognition, and the image is a workload. Device 1 initiates a parallel computing task and allocates the workload. At this time, the multi-fusion layer block expansion step is run to obtain a multi-fusion layer block. When each computing device executes the calculation of the multi-fusion layer block in the main branch and reaches the sub-branch, that is, when the calculation reaches the exit point corresponding to branch 1, the calculation result of the current main branch is saved. In this embodiment, the feature map calculated in the main branch at this time is retained for each device. Based on the calculation result of the current main branch, each computing device executes the convolution layer of the sub-branch at the current position, and starts to execute the convolution layer of branch 1 in the form of a multi-fusion layer block. When the calculation reaches the fully connected layer of the convolutional neural network of branch 1, each device sends the calculated feature map to the device with the strongest computing power in the cluster, which is device 2 in this embodiment. The fully connected layer is executed by the computing device with the strongest computing power. In this embodiment, device 2 executes the fully connected layer, and obtains the entropy value of the input sample based on the execution result of the fully connected layer. It determines whether to continue to execute the main branch or exit the calculation based on the comparison result of the obtained entropy value and the preset confidence threshold. In this embodiment, it is specifically to determine whether the execution result meets the preset confidence threshold. If the calculation result at this time has a sufficiently high confidence, device 2 sends the result to the task initiating device device 1, otherwise, device 2 will send a command to continue to execute the main network to device 1 and device 3, and the edge cluster will restore the feature map just retained and continue to execute the main network. When the calculation reaches the exit point corresponding to branch 2, the processing method of each device is the same as when the calculation reaches branch 1, and it is determined whether the confidence of branch 2 meets the preset confidence threshold. If it meets, the calculation of the convolutional neural network is terminated, otherwise all convolutional neural networks will be executed, and finally the device with the strongest computing power executes the fully connected layer of the main branch and sends the result to device 1. In some embodiments of the present invention, exit points and branches correspond one to one, and a multi-layer convolutional neural network with branches can have multiple pairs of exit points and branches. The position of each exit point is set manually, and the branched multi-layer convolutional neural network is a simplified main-branch multi-layer neural network used to improve the calculation speed.

[0072] like Figure 5 Shown is a performance comparison chart of the present invention and the existing parallel computing method. Figure 5 (a) is a comparison of the average latency performance of the present invention and the existing parallel computing method MoDNN when using different numbers of computing devices to process the CIFAR10 dataset for the AlexNet convolutional neural network. Figure 5(b) is a comparison of the average latency performance of the present invention and the existing parallel computing method MoDNN when using different numbers of computing devices to process the CIFAR10 data set for the ResNet50 convolutional neural network. Figure 5 (c) is a comparison of the average latency performance of the present invention and the existing parallel computing method MoDNN when using different numbers of computing devices to process the ImageNet dataset for the ResNet50 convolutional neural network. Figure 5 In the figure, the present invention is also compared with two existing cloud-assisted deep learning acceleration methods, i.e., the two dotted horizontal lines in the figure. When the number of devices is more than five, the present invention can calculate the convolutional neural network faster than the cloud-assisted acceleration method. The present invention uses two Raspberry Pi 4B processors with Cortex-A72, two CPUs set to single-core 800MHZ virtual machines, and two CPUs set to single-core 1000MHZ virtual machines to simulate an edge cluster with heterogeneous computing capabilities. In order to compare with the reasoning acceleration forms of "edge" and "edge cloud", we use a PC with a CPU of i5-8400 and a main frequency of 2.8GHZ to simulate the edge server, and use a server equipped with 4 3090 graphics cards to simulate the edge server. The present invention takes AlexNet, ResNet50 and ResNet101, these existing typical convolutional neural network models, as examples to optimize these convolutional neural network models and realize the parallel computing method for edge clusters of the present invention. Because AlexNet, ResNet50 and ResNet101, the existing typical convolutional neural network models, have different depths, that is, different numbers of convolutional layers, the CIFAR10 and ImageNet datasets are used to evaluate the performance of the method of the present invention under tasks of different difficulty. Figure 5 In for The present invention compares the performance of AlexNet neural network and ResNet with MoDNN when the exit rate is Pn=(40%, 40%, 20%), and the change of average inference delay when the network bandwidth is 100Mbps. A set of feasible solutions with the exit rate of (40%, 40%, 20%) has ideal acceleration performance. Because it can traverse all branches of the multi-layer convolutional neural network with branches proposed by the present invention. n=(40%, 40%, 20%) means that 40% of the test data exit from the first exit point branch, 40% of the test data exit from the second exit point branch, and 20% of the test data completes the calculation of all convolutional layers in the main branch. It can be seen that as the number of computing devices in the edge cluster increases, the average latency of the convolutional neural network task is also decreasing. The method of the present invention can achieve relatively ideal performance in three tasks of different scales and difficulties. Since the present invention adds an early exit branch to the convolutional neural network, the average latency of the method of the present invention is always lower than that of MoDNN. At the same time, because the present invention adopts a multi-fusion layer block method instead of a layered method of MoDNN, the communication time of the present invention is also lower than that of MoDNN. When the number of devices exceeds four, the parallel computing method we proposed can complete the reasoning task of the convolutional neural network at a faster speed than edgent and the three-partition network. This is because although it uses edge servers and the cloud to accelerate reasoning, some tasks still reside on edge devices. In addition, the method of the present invention eliminates the need to upload data to the cloud server, thereby avoiding the risk of privacy leakage.

[0073] like Figure 6 The figure shows the average computing delay performance comparison of the present invention based on different neural networks. The parallel computing method of the present invention can adjust the reasoning time according to the task delay requirements. Figure 6 (a) is a performance diagram of the parallel computing method of the present invention for the AlexNet convolutional neural network on the CIFAR10 dataset. Figure 6 (b) is a performance diagram of the parallel computing method of the present invention for the ResNet50 convolutional neural network on the CIFAR10 data set, Figure 6 (c) is a performance diagram of the parallel computing method of the present invention for the ResNet50 convolutional neural network on the ImageNet dataset. Figure 6 (a) Taking the performance of the AlexNet neural network on the CIFAR10 dataset as an example, when the delay requirement is 60ms, a single device can only achieve 78% accuracy, while multiple devices in parallel can reach 79% accuracy, and a single device cannot complete the inference task within 38ms. As the number of devices increases, the present invention can achieve higher accuracy under the specified delay requirements. For example, when the delay requirement is 24ms, two devices can achieve 76.6% accuracy, while four devices can achieve 78.3% accuracy. Due to the complexity of the ImageNet dataset, the accuracy of the early exit branch is not as high as that on the CIFAR10 dataset, so Figure 6 The average accuracy in (c) will be more attenuated than in (b), but still within an acceptable range. Figure 6 (b) and Figure 6In (c), due to the particularity of the ResNet convolutional neural network structure, ResNet with multiple exit points will experience an acceleration bottleneck when running on a single device. This is because the accuracy of the previous exit branch is not high, and the execution time of the subsequent branch is relatively long. In order to meet the latency requirements, most input samples can only exit from the first branch with lower accuracy. However, the multi-device parallel computing method of the present invention can eliminate the impact of the acceleration bottleneck and provide a convolutional neural network inference service with higher accuracy while ensuring low latency.

[0074] The invention proposes a parallel computing method for edge clusters, which makes full use of the computing resources of idle heterogeneous devices in the edge environment, and multiple devices jointly participate in the multi-layer convolutional neural network computing tasks, thereby improving the execution speed of multi-layer convolutional neural network tasks on devices with weak computing power. Unlike the "end-edge-cloud" architecture, the architecture proposed by the invention can keep the task in the local cluster without the help of the cloud, thereby reducing the workload of the cloud, and the data is always retained in the local cluster, avoiding the risk of privacy leakage. At the same time, the method proposed by us has the ability to balance between dynamic response time and accuracy, so it has scalability. It can adapt to the needs of different tasks in the edge environment, and users can adjust the task delay according to needs. It has the advantages of reducing the total time of executing multi-layer convolutional neural networks in edge clusters and improving the utilization rate of idle computing resources of edge clusters; it has an adaptive workload distribution algorithm based on heterogeneous edge devices and different network conditions, and adopts multi-layer fusion technology to further reduce communication and additional computing overhead; a method for determining the branch network exit point threshold according to user delay requirements, so that multi-layer convolutional neural networks can complete reasoning tasks within a specified time, etc.

[0075] Corresponding to the above method, the present invention also provides a parallel computing system for edge clusters, the system comprising a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0076] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0077] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0078] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0079] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A parallel computing method for edge clusters, characterized in that: The method comprises the following steps: A computing power acquisition step is used to obtain the computing power of each computing device in the edge cluster; The multi-fusion layer block generation step is used to divide the multi-layer convolutional neural network including the main branch and the sub-branches corresponding to the different exit points of the preset main branch into multi-fusion layer blocks, and for each convolution layer of the divided multi-fusion layer blocks, the workload corresponding to the computing capacity of each computing device is allocated to each computing device according to the computing capacity of each computing device obtained, so that the maximum difference between the computing time of each computing device executing the continuous convolution layer in the current multi-fusion layer block and the average computing time is less than a predetermined synchronization threshold; each computing device exchanges data between computing devices after completing the tasks of multiple continuous convolution layers in a multi-fusion layer block; In the branch selection step, when each computing device performs the calculation of the multi-fusion layer block in the main branch and reaches the sub-branch, the calculation result of the current main branch is saved, and the convolution layer of the sub-branch at the current position is executed by each computing device based on the calculation result of the current main branch. The computing device with the strongest computing power executes the fully connected layer, and the entropy value of the input sample is obtained according to the execution result of the fully connected layer. According to the comparison result of the obtained entropy value and the preset confidence threshold, it is determined whether to continue to execute the main branch or exit the calculation; The step of generating a multi-fusion layer block includes: Initial multi-fusion layer block setting step: setting a multi-fusion layer block including at least one convolutional layer for the multi-layer convolutional neural network; Multi-fusion layer block expansion step: based on the obtained computing power of each computing device, the redundant data computing overhead of the multi-fusion layer block is estimated, the communication saving overhead is calculated based on the bandwidth of the network where the edge cluster is located, and the multi-fusion layer block is expanded based on the calculated redundant data computing overhead and communication saving overhead, so that when the redundant data computing overhead is greater than the communication saving overhead and the convolution layer adjacent to the end convolution layer is not an exit point or the last layer of the multi-layer convolutional neural network, the convolution layer adjacent to the end convolution layer is added to the multi-fusion layer block and used as the end convolution layer; Task allocation step: allocating workloads corresponding to the computing capabilities of the computing devices involved in the calculation according to the computing capabilities of the computing devices, so that the maximum difference between the computing time of each computing device executing the continuous convolutional layers in the current multi-fusion layer block and the average computing time is less than a predetermined synchronization threshold; Repeat the task assignment step and the multi-fusion layer block expansion step until the redundant data calculation overhead is greater than or equal to the communication saving overhead, the expansion reaches the exit point, or the expansion reaches the last layer of the multi-layer convolutional neural network.

2. The method according to claim 1, characterized in that: The task allocation step comprises: The computing amount of the workload allocated to a computing device is the product of the ratio of the computing power of the computing device to the sum of the computing power of all computing devices participating in the calculation and the computing amount of the total workload; The workload allocated to the computing device is the workload of the end layer of the current fusion layer block. The workload of the convolution layers other than the end layer of the current fusion layer block is expanded layer by layer from the workload of the end layer toward the starting layer of the current fusion layer block to obtain the workload of all convolution layers of the current fusion layer block. The workload of each convolution layer is expanded by expanding the workload boundary of the adjacent convolution layer toward the end layer by half the size of the convolution kernel.

3. The method according to claim 1, characterized in that: The task allocation step also includes: When the maximum difference between the calculation time of each computing device executing the continuous convolutional layers in the current multi-fusion layer block and the average calculation time is greater than the predetermined synchronization threshold, the workload of the computing device corresponding to the maximum workload is reduced and the workload of the computing device corresponding to the minimum workload is increased until the maximum difference between the calculation time and the average calculation time is less than the predetermined synchronization threshold.

4. The method according to claim 1, characterized in that: The multi-fusion layer block generation step also includes: When the expansion of a multi-fusion layer block is completed, all computing devices involved in the calculation send the calculation results to the computing device with the strongest computing power for aggregation, and the computing device redistributes the workload based on the calculation results before starting the expansion of the next multi-fusion layer block. The multi-fusion layer block generation steps are repeated until the last layer in the multi-fusion layer block is the last layer of the main branch or sub-branch of the multi-layer convolutional neural network.

5. The method according to claim 2, characterized in that: The multi-fusion layer block expansion step also includes: The time used to transmit all convolutional layers except the starting and ending layers in the multi-fusion layer block is calculated as the communication cost saving; The time taken by the computing device to calculate the expansion amounts of all convolutional layers except the end layer in the multi-fusion layer block is calculated as redundant data calculation overhead.

6. The method according to claim 1, characterized in that The confidence threshold is calculated as follows: Record the output entropy of all test data in each branch when training a multi-layer convolutional neural network; According to the conditions that the sum of the product of the computing time of each branch of the multi-layer convolutional neural network and the exit probability of the corresponding exit point is less than or equal to the maximum computing time required for the multi-layer convolutional neural network and the sum of the exit probabilities of all exit points is 1, a feasible solution of the exit probabilities of multiple groups of exit points is calculated, and each group of feasible solutions includes the exit probabilities of all preset exit points; The maximum computing time is set based on the latency performance requirement or accuracy performance requirement of the multi-layer convolutional neural network; Record the calculation accuracy of the multi-layer convolutional neural network at each exit point and calculate the weight ratio of the calculation accuracy of each exit point to the sum of the calculation accuracy of all exit points; Multiply the probability of each exit point of each feasible solution group by the weight ratio corresponding to its exit point and sum them up to get the score of each feasible solution group, and select a feasible solution group with a high score; When the entropy value obtained by the method of calculating the entropy value is proportional to the calculation accuracy, the maximum value among the output entropy values ​​of each test data is selected, and the ratio of the number of bits when all the maximum values ​​are arranged in descending order to the number of all maximum values ​​is equal to the entropy value of the probability of the exit point as the confidence threshold of the exit point, or when the entropy value obtained by the method of calculating the entropy value is inversely proportional to the calculation accuracy, the minimum value among the output entropy values ​​of each test data is selected, and the ratio of the number of bits when all the minimum values ​​are arranged in ascending order to the number of all minimum values ​​is equal to the entropy value of the probability of the exit point as the confidence threshold of the exit point.

7. The method according to claim 1, characterized in that The method also includes training a multi-layer convolutional neural network with branches; the computing power acquisition step is to run multiple convolutional layers with different parameters for each device in the edge cluster to obtain a computing power linear regression model for each device.

8. A parallel computing system for edge clusters, comprising a processor and a memory, characterized in that: The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images