Multi-outlet neural network-oriented outlet setting method, apparatus and device, and storage medium

Abstract

The invention discloses an outlet setting method and device for a multi-outlet neural network, equipment and a storage medium. The method comprises the steps of abstracting a neural network into a plurality of logic layers and a final outlet branch, quantifying inference overhead of the logic layers and the outlet branch, constructing an outlet setting optimization problem of multi-outlet neural network task inference, and performing approximate representation; according to the abstracted neural network model, determining the weight of each outlet branch by inserting outlets in different logic layers and using a video task for training, reasoning the video task frame by frame on the model with the trained weight, and obtaining key information to reflect the reasoning difficulty of the video task and the relevance between continuous frames; and the optimization problem after original approximation is converted, and a new optimization problem is solved by using a search method based on an ant colony algorithm. According to the exit rule and exit setting method, the expected time delay of the multi-exit neural network reasoning task can be effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an outlet setting method, device, equipment and storage medium for a multi-outlet neural network. Background Art

[0002] Target recognition plays an important role in traffic control and security monitoring. It can distinguish the object that the user is interested in from other objects. Real-time and high precision are often the guarantee of user experience. How to quickly and accurately mark the target object under the constraints of limited resources is a problem that needs to be solved urgently.

[0003] Traditional neural network models reason about tasks of different complexity, requiring the task to completely go through each layer of reasoning before exiting. In fact, simple tasks do not require so much calculation. The task features extracted at the shallow level of the neural network are sufficient to correctly identify the target. Continuing reasoning will not only increase latency, but may also cause overfitting problems. Multi-exit neural networks are a method that can flexibly allocate computing resources of different sizes according to the difficulty of task reasoning. Multi-exit neural networks, also known as early exit neural networks, do not require modifying the backbone network. They only introduce classifiers as exit branches in the middle layer of the neural network to allow tasks to flexibly choose appropriate exits to exit, which avoids the time and computational cost of retraining the entire network. Early task exit can not only reduce reasoning latency, but also effectively avoid the problem of overthinking.

[0004] However, in the existing research work on multi-exit neural networks, most of them manually determine the exit positions or insert exits at intervals of fixed floating-point operations, and the model depth, number of exits, and exit positions do not change after deployment. In fact, an unchanging model is not always suitable for tasks with variable reasoning difficulty. When there are more simple tasks, a model with too few shallow exits is not applicable because these simple tasks are forced to be transmitted to deeper exits; similarly, when there are more complex tasks, a model with too many shallow exits is not applicable because complex tasks will generate a lot of redundant calculations at shallow exits.

[0005] The traditional exit selection method is to try to exit each time the task reaches an exit branch. If the exit criteria are met, it will exit from the exit of that layer, otherwise it will continue to go deeper. Although there are some methods to predict exits, and auxiliary tasks skip some unnecessary exit attempts to reduce redundant calculations, reasonable exit settings are still necessary. Using the skip exit method on a reasonable exit setting strategy can further improve the efficiency of the model.

[0006] The two key issues in exit settings are the number of exits and the location of exits. When too many exits are set or even exits are inserted at each logical layer, the training and optimization costs of the model will be prohibitive. If too few exits are set, it may be difficult to find a suitable exit for the task. An appropriate exit setting solution improves the performance of the model and optimizes the user experience.

[0007] In order to quickly and accurately identify the target object under the constraint of limited resources, the present invention considers an exit rule for skipping inappropriate exits in a multi-exit neural network based on the correlation of consecutive video frames as an auxiliary task, and proposes an exit setting method under this exit rule, which makes the expected delay of the multi-exit neural network reasoning task as small as possible by inserting a suitable number of exits at suitable positions of the neural network. Summary of the invention

[0008] Purpose of the invention: The present invention is aimed at target recognition scenarios of video tasks, and proposes a prediction method for predicting the exit exit of a task by utilizing the correlation between consecutive frames. Combining the prediction method with traditional exit rules, a new exit rule that takes into account both prediction overhead and prediction accuracy is proposed. Under this exit rule, the exit setting of a multi-exit neural network is implemented, which can effectively reduce the expected delay of task reasoning.

[0009] The present invention also provides an outlet setting device, equipment and storage medium for a multi-outlet neural network.

[0010] Technical solution: In order to achieve the above invention objectives, the technical solution of the present invention is as follows:

[0011] An output setting method for a multi-output neural network comprises the following steps:

[0012] Abstract the neural network into several logic layers and a final exit branch, quantify the reasoning overhead of the logic layer and the exit branch, and construct an exit setting optimization problem for multi-exit neural network task reasoning. The goal of this optimization problem is to find an exit setting scheme that minimizes the expected computational consumption of the task and minimizes the reasoning delay. Based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated to obtain a form that is easy to solve.

[0013] According to the abstracted neural network model, the weight of each exit branch is determined by inserting exits at different logical layers and training with video tasks. The video task is inferred frame by frame on the model with trained weights. The relative inference difficulty between consecutive frames is determined according to the classification output values ​​of the previous and next frame tasks, and the probability of each exit being skipped by a task is counted.

[0014] According to the relative reasoning difficulty between consecutive frames and the probability that each exit is skipped by the task, the original approximated optimization problem is converted into an equivalent optimization problem based on the difference between the benefit introduced by inserting the exit branch and the redundant computation amount. The new optimization problem is solved by a search method based on the ant colony algorithm, and finally the exit setting scheme with the largest difference between the benefit and the redundant computation amount is obtained as the final exit setting scheme of the multi-exit neural network.

[0015] Furthermore, the neural network is abstracted into several logic layers and a final export branch, the reasoning overhead of the logic layer and export branch is quantified, and the export setting optimization problem of multi-export neural network task reasoning is constructed, including:

[0016] Abstract the neural network model: According to the neural network composed of convolution layer, pooling layer, relu function, fully connected layer, and softmax layer, the original model is abstracted into several logical layers and a final output. The fully connected layer and softmax layer at the end of the model are abstracted as the final output. Each convolution layer and all the pooling layers and relu functions from the convolution layer to the next convolution layer or the final output are abstracted as a logical layer.

[0017] Quantify the inference cost of the logic layer and the export branch: Use floating-point operations to represent the expected computational cost of model inference. The floating-point operations of the j-th logic layer are expressed as Indicates that It represents the expected floating point operations generated by the exit branch reasoning of a task set by the logic layer j;

[0018] Optimization problem modeling: Consider a neural network model with L logical layers. The goal of the optimization problem is to find an exit setting combination X = {x1, x2, ...x j ,...x L |x j ∈{0, 1}, j∈L}, where x j Indicates whether an exit branch is inserted in the jth logical layer. If inserted, it is 1, otherwise it is 0. The logical layer 1 and the logical layer L have the first exit and the final exit respectively. There is x1 = 1, x L =1; let event A be the classification output distribution of the current frame at the first exit being “flat” compared to the previous frame, event B be the classification output distribution of the current frame at the first exit being “unflat” compared to the previous frame, T(π) represents the calculation of the expected consumption of task reasoning when the exit is set to π, e2 represents the exit of the task in the previous frame when the exit is set to π, μB(π) j It represents the probability that the task exits from the exits set by logic layers 1 to j when event B occurs when the exit is set to π; then when the exit is set to π, the expected consumption generated by task reasoning is expressed as:

[0019] E[T(π)]=P(A)·E[T(π)|A]+P(B)·E[T(π)|B]

[0020] in,

[0021]

[0022]

[0023] e2=f means that the previous frame exits from the exit set by the logical layer f.

[0024] Furthermore, based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated, including:

[0025] Assuming that when event A or B occurs, the task exits from the best exit, the expected approximate value of the consumption expected to be generated by task reasoning is expressed as:

[0026]

[0027] Among them, μ(π) j represents the probability of the task exiting from the exit set at logical layer 1~j when the exit is set to π; v j represents the probability that the task skips the exit set by the logic layer j;

[0028] Let μ j represents the probability that the best exit of the task is in the logical layers 1 to j when all candidate logical layers have exits; when the logical layer j has an exit, x j =1, μ(π) j =μ j ; When there is no exit at logical layer j, x j =0, μ(π) j =μ(π) j-1 , then μ(π) j Expressed as: μ(π) j =max i∈{0，1，2，3...j} {X i ·μ i}.

[0029] Furthermore, the weight of each export branch is determined by inserting exports at different logic layers and using video tasks for training, including:

[0030] For the neural network whose logic layer meets the specified conditions, the export branch is inserted in all the logic layers, and the video task is used for training to obtain the weight coefficient of each export branch;

[0031] For the neural network whose logic layer does not meet the specified conditions, the weight coefficient of each export branch is obtained by training by inserting exports in different logic layers multiple times.

[0032] Furthermore, the relative difficulty of reasoning between consecutive frames is determined based on the classification output values ​​of the tasks in the previous and next frames, and the probability of each exit being skipped by a task is counted, including:

[0033] Compare the confidence of the feature map and classification output value extracted by the current frame task i at the first exit with the intermediate features and classification output value extracted by the previous frame task i-1 at this exit. If the former is higher than the latter, it is determined that the reasoning difficulty of task i is higher. Task i will skip all exit branches lower than the exit of task i-1 in subsequent reasoning. Task i will try to exit one by one from the exit of task i-1 until the exit requirements are met or the final exit is reached. If the reasoning difficulty of task i cannot be determined to be higher than that of task i-1, task i will follow the traditional exit rules and try to exit one by one at the exit.

[0034] According to the classification output distribution, the optimal exit distribution of the task is obtained and μ is estimated. j size;

[0035] According to the maximum classification output distribution of the task and the maximum output value of the continuous frames at the first exit, estimate v j size.

[0036] Furthermore, according to the relative reasoning difficulty between consecutive frames and the probability that each exit is skipped by a task, the original approximated optimization problem is converted into an equivalent optimization problem based on the difference between the benefit introduced by inserting the exit branch and the redundant computation, including:

[0037] The expected redundant computation introduced by setting the egress at logical layer i is expressed as:

[0038]

[0039] Compared with setting an exit at layer j, the expected benefit of reasoning about a task is expressed as:

[0040]

[0041] The logical layer k is the logical layer where the next exit after the jth layer exit is located, and μ k-1 =μ j , μ k ≥μ j ;

[0042] Then, an exit is set at the logic layer i, and the difference between the revenue generated by a task and the redundant computing amount generated is expressed as:

[0043] R(k)j =I (k)j -C j

[0044] When the export is set to X, the difference between the total benefit and the total redundant computation of the entire model for reasoning a task is expressed as:

[0045] R(X)=I X -C X

[0046] Suppose the computational cost of a task through the reasoning of the original model is A, where A is a constant, then:

[0047] R(X)+O(x)=A

[0048] Since A is a constant, solving X when O(X) is the smallest is solving X when R(X) is the largest. Therefore, the approximated optimization problem is transformed into:

[0049] arg max(R(X))

[0050] x j ·R (k)j >=0,j∈{1,2,3...L}

[0051]

[0052] Furthermore, the search method based on ant colony algorithm is used to solve new optimization problems, including:

[0053] h) Initializing the location of the logical layer where the first exit is located;

[0054] i) Initialize the pheromone concentration important factor α, the difference between the benefit and the redundant calculation amount important factor β, the pheromone volatility ρ, the number of ants m, the maximum number of iterations, and the pheromone concentration of each path;

[0055] j) At the beginning of each iteration, m ants are placed at the final exit of the neural network and move toward the shallow logic layer. The ants are asked to randomly choose the next step according to the pheromone concentration of different paths until they reach the first exit.

[0056] k) A movement path of an ant is a combination of exit settings. When a group of ants completes a complete movement, the pheromone concentration on each path will be updated. The increase in pheromone concentration depends on the difference between the corresponding exit setting method benefit and the redundant calculation amount. At the end of each round of iteration, the pheromone concentration of each path is reduced according to the pheromone volatility ρ;

[0057] l) Repeat c)-d) until the maximum number of iterations is reached;

[0058] m) The path with the largest pheromone concentration is the optimal exit setting method under this first exit setting mode, the position of the first exit is reset, and b)-e) are repeated to obtain the optimal exit setting method under different first exit setting modes;

[0059] n) The exit setting scheme with the largest difference between the benefit and the redundant calculation amount is taken as the final exit setting scheme of the multi-exit neural network.

[0060] An outlet setting device for a multi-outlet neural network, comprising:

[0061] The module for constructing the exit setting optimization problem is used to abstract the neural network into several logical layers and a final exit branch, quantify the reasoning overhead of the logical layers and exit branches, and construct the exit setting optimization problem for multi-exit neural network task reasoning. The goal of this optimization problem is to find an exit setting scheme that minimizes the expected computational consumption of the task and thus minimizes the reasoning latency. Based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated to obtain a form that is easy to solve.

[0062] The key information acquisition module is used to determine the weight of each exit branch by inserting exits at different logical layers and training with video tasks based on the abstracted neural network model, reasoning the video task frame by frame on the model with trained weights, determining the relative reasoning difficulty between consecutive frames based on the classification output values ​​of the previous and next frame tasks, and counting the probability of each exit being skipped by a task;

[0063] The optimization problem conversion and solution module is used to convert the original approximated optimization problem into an equivalent optimization problem based on the difference between the benefit and redundant computation introduced by inserting the exit branch according to the relative reasoning difficulty between consecutive frames and the probability that each exit is skipped by the task, and use the search method based on the ant colony algorithm to solve the new optimization problem, and finally obtain the exit setting plan with the largest difference between the benefit and the redundant computation as the final exit setting plan of the multi-exit neural network.

[0064] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the exit setting method for a multi-exit neural network as described above are implemented.

[0065] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for setting an outlet for a multi-outlet neural network as described above are implemented.

[0066] Beneficial effects: The present invention discloses an exit setting method for a multi-exit neural network, which reduces the delay of task reasoning as much as possible under a small precision constraint. In combination with the target recognition video task, the exit of the continuous video frame task is predicted, with the aim of reducing the extra computational effort introduced by the task attempting to exit at the wrong exit branch, and a new exit rule for the video task in a multi-exit neural network is proposed. Under this exit rule, the computational consumption of the task is modeled, and an optimization problem is proposed. Since some parameters in the original optimization problem are difficult to obtain, approximate values ​​are used to replace these parameters, and experiments have proved that the actual values ​​are similar to the approximate values. The converted approximate problem is still difficult to solve, so it is converted into an equivalent optimization problem, and an exit setting method based on an ant colony algorithm is proposed on the new optimization problem, using the expected computational consumption of the multi-exit neural network to evaluate the performance of the multi-exit model under the exit setting method. The exit rule and exit setting method of the present invention can effectively reduce the expected delay of multi-exit neural network reasoning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a task execution flow chart of the outlet setting method based on the multi-outlet neural network of the present invention;

[0068] Figure 2 It is a schematic diagram that abstracts the original VGG16 model into several logical layers and a final export;

[0069] Figure 3 and Figure 4 It is a comparison chart of the actual exit probability distribution of the task and the optimal exit probability distribution under two exit settings;

[0070] Figure 5 This is a flow chart of the export setting method based on the ant colony algorithm. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0072] Reference Figure 1 The present invention provides an outlet setting method based on a multi-outlet neural network, comprising the following steps:

[0073] Step (1): Model abstraction and information estimation

[0074] Neural networks are usually composed of convolutional layers, pooling layers, relu functions, fully connected layers, and softmax layers. The original model can be abstracted into several logical layers and a final output. Specifically, the fully connected layer and softmax layer at the end of the model are abstracted as the final output, and each convolutional layer and all the pooling layers and relu functions from the convolutional layer to the next convolutional layer or the final output are abstracted as a logical layer. Taking VGG16 as an example, Figure 2Given the logical layer architecture of VGG16, VGG16 can be abstracted into 13 logical layers and one final export.

[0075] In order to quantify the inference cost of the model under different export settings, the present invention uses floating point operations (FLOPs) to represent the expected computational cost of model inference. The floating point operations of a logic layer can be expressed as the sum of the floating point operations of the convolution layer, pooling layer, and relu activation function that constitute the logic layer, so the floating point operations of the j-th logic layer can be expressed as:

[0076]

[0077] Among them, l j1 , l j2 , l j3 They represent the number of convolutional layers, pooling layers, and activation function layers in the jth logical layer, respectively. conv 、c pool 、c Relu Indicates the floating-point number computation amount of one inference task of the convolution layer, pooling layer, and relu activation function that constitute the logic layer of this layer.

[0078] use It represents the expected floating-point operation amount generated by the exit branch reasoning of a task set by the logic layer j, and stipulates that the exit branch exit structure inserted in any candidate logic layer is the same, consisting of several fully connected layers (FC layers) and a softmax classifier. If it is to identify the target quantity of the task, the softmax layer is replaced by a linear function and a sigmoid function. The linear function is used for linear prediction of the target quantity, and the output of the sigmoid function is used to compare with the confidence threshold to decide whether the task exits at this exit.

[0079] The floating-point operations of an exit branch are:

[0080]

[0081] Where l4 represents the number of fully connected layers in an output branch, c FC 、c softmax Indicates the size of the floating-point operations of the fully connected layer of the exit branch and the softmax layer.

[0082] Step (2), exit rule formulation

[0083] In order to ensure that the task is exited at the correct exit and reduce redundant calculation of the task, the present invention proposes an exit rule that combines exit prediction with a traditional exit method.

[0084] The more complex the image features of a task sample, the lower the discrimination between categories of its feature map. In other words, the probability distribution of complex tasks output on the same softmax layer is flatter than that of simple tasks. Therefore, complex task samples need to go through more layers of reasoning than simple tasks before they can be correctly classified on the softmax layer. This is also the reason why the exit point of complex tasks in multi-exit neural networks is deeper than that of simple tasks. Based on the above facts, we can draw a conclusion that the relative depth of the exit points of two task samples can be determined based on the output difference of the two task samples on the same softmax layer. The sharper the probability distribution on the same softmax layer, the easier it is to distinguish the task categories and the shallower the exit.

[0085] Although the outputs of different tasks on the softmax layer can map the relative difficulty of reasoning between tasks to a certain extent, there are still two limitations. First, the complexity of the image is related to many factors such as image quality, image content, and model complexity. For tasks with low correlation and large differences in image features, it is not necessarily possible to rely on the softmax output to predict the relative complexity of the task. Simple tasks may present a more "flat" output on the softmax layer than complex tasks due to overfitting of the model. Second, the complexity boundary of the task is difficult to grasp simply by relying on the output results of softmax. It can only predict the relative depth between the task exits through the output results, and cannot accurately predict the specific appropriate reasoning depth of the task. If you want to establish an accurate relationship between softmax output and exit between tasks with low correlation, you need to introduce a lot of training costs.

[0086] In order to reduce the redundant calculation size of the task without introducing huge training costs, the present invention proposes an exit rule that uses the correlation between the previous and next frames of the video to assist the task in skipping some exits. The relative reasoning difficulty of the current frame is predicted by comparing the output results of the adjacent frames on the softmax layer, and the exit information of the previous frame task is used to assist the prediction of the exit of the current frame task. Specifically, first, a first exit is artificially set in the shallow layer of the neural network, which allows simple tasks to exit in advance, and at the same time, the relative position of the exit of the task that does not exit at this exit compared to the exit of the previous frame task is determined. Then, the confidence of the feature map extracted from the first exit of the current frame task i and the output value of the softmax layer is compared with the intermediate features and softmax layer output values ​​extracted from this exit of the previous frame task i-1. If it is determined that the reasoning difficulty of task i is higher and the inter-class distinguishability is lower, task i will skip all exit branches lower than the exit of task i-1 in subsequent reasoning, and task i will try to exit one by one from the exit of task i-1 until the exit requirements are met or the final exit is reached. If the reasoning difficulty of task i cannot be determined to be higher than that of task i-1, task a will follow the traditional exit rules and try to exit one by one at the exit. The relative level of reasoning difficulty of consecutive frames is judged by the relative size of the ratio of the maximum softmax value to the second largest softmax value of the first exit. The larger the ratio, the sharper the output, the higher the inter-class distinguishability of the task, and the lower the reasoning difficulty.

[0087] The formulated exit rules are used in the construction of the optimization problem and are to be followed while solving the optimization problem.

[0088] Step (3): Optimization problem modeling

[0089] Consider a neural network model with L logical layers. The first exit is located at the logical layer 1, which is used to exit some simple tasks and exit prediction. Specifically, the feature map extracted by the task at the first exit is visualized, and the classification results (softmax output) of the task at this exit are collected and compared with the previous frame task. If it is determined that the reasoning difficulty of the current task is higher, the current task will skip all exit branches that are lower than the exit of the previous task except the first exit in subsequent reasoning.

[0090] The goal of the optimization problem is to find an output setting combination X = {x1, x2, ...x j ,...x L |x j ∈{0, 1}, j∈L}, where x jIndicates whether an exit branch is inserted in the jth logical layer. If inserted, it is 1, otherwise it is 0. The logical layer 1 and the logical layer L have the first exit and the final exit respectively. There is x1 = 1, x L =1.

[0091] Let the output performance of the current frame at first-exit be "flat" to the output of the previous frame at first-exit as event A, and the opposite event of event A is event B, that is, the output performance of the current frame at first-exit is not "flat" to the previous frame. Let the probability of the previous frame exiting from the exit set by the logical layer f be P(e2=f), we introduce some variables to characterize the correlation between consecutive frames, the complexity of the task, and the exit distribution of the task.

[0092] Let event A be the softmax distribution of the current frame at the first exit that is "flat" compared to the previous frame, event B be the softmax distribution of the current frame at the first exit that is "unflat" compared to the previous frame, T(π) represents the calculation of the expected consumption of task reasoning when the exit is set to π, e2 represents the exit of the previous frame task when the exit is set to π, μB(π) j It indicates the probability that the task exits from the exit set by logic layer 1~i when event B occurs when the exit is set to π. When the exit is set to π, the expected consumption of task reasoning can be expressed as:

[0093] E[T(π)]=P(A)·E[T(π)|A]+P(B)·E[T(π)|B]

[0094] in,

[0095]

[0096]

[0097] Step (4), the optimization problem is approximated

[0098] Take any three consecutive frames, let e1, e2, e3 represent the actual exits (from first to last) of the three consecutive frames when the exit is set to π, let I, F, K represent the best exits (from first to last) of the three consecutive frames when the exit is set to π. Let event A represent that the softmax distribution of the third frame at the first exit is "flatter" than the second frame, event A1 represent that the softmax distribution of the second frame at the first exit is "flatter" than the first frame, event B represent that the softmax distribution of the third frame at the first exit is "sharper" than the second frame, event B1 represent that the softmax distribution of the second frame at the first exit is "sharper" than the first frame, so:

[0099]

[0100] Among them, P(A, e2=f) is related to the actual exit of the previous frame task and the relative reasoning difficulty, which can be expressed as:

[0101]

[0102] Similarly, the size of P(A1, e1=i) will be related to the exit position of the previous frame task and the relative reasoning difficulty.

[0103] Under the proposed exit rule, when event A occurs, the actual exit of the task may be affected by the exit of the task in the previous frame. Due to the accumulation of correlations between consecutive frames, the probability distribution of the actual exit of the task is difficult to calculate.

[0104] Assuming that the exit is set to π, K is the best exit for the current task i, that is, the exit with the first softmax maximum value greater than the preset threshold. Let the actual exit of the current frame task i be e3, and the exit of the previous frame task be e2. When event B occurs, e3 = K; when event A occurs, there are the following two situations:

[0105] ①e2≤K, task i starts from the exit set by the logical layer e2 and tries to exit layer by layer, and exits after reaching the best exit K, so e3=K.

[0106] ②e2>K, task i skips the best exit K, and tries to exit layer by layer starting from the exit set by the logic layer e2, and exits at the next exit e3 that satisfies the softmax maximum value greater than the preset threshold, and e3>K.

[0107] The distribution of P(K=k|A), k=1, 2...L is only related to the reasoning difficulty of the task and the exit setting π, and has nothing to do with the relevance of the previous frame task. However, P(e3=k|A), k=1, 2...L will be affected by the exit e2 of the previous frame task. Similarly, the exit e2 of the previous frame task may be affected by the exit of the previous frame task. Due to the accumulation of relevance between consecutive frames, the distribution of P(e3=k), k=1, 2...L and the distribution of P(e2=f), f=1, 2...L are difficult to obtain. Therefore, the present invention approximates the actual exit probability distribution P(e3=k|A), k=1, 2...L of the task when event A occurs.

[0108] If it is proved that the probability of ② occurring is small, and P(e3=k|A)≈P(K=k|A), k=1, 2...L, then it can be considered that the task often exits at the optimal exit K or a nearby exit, and P(K=k|A), k=1, 2...L is used instead of P(e3=k|A), k=1, 2...L for subsequent calculations.

[0109] Using VGG16 as the original model, the first exit is set at the logical layer 1, and the exit branch structure is adjusted to predict the number of people in the Caltech dataset. 9900 consecutive frames are taken as the training set, and it is found that 49.64% of the frames allow skipping of some exits. The average number of consecutive frames that continuously skip the exit of the task lower than the previous frame except the first exit is 1.53, and the maximum value is 6. In other words, after an average of 1.53 frames, the task must try to exit each exit one by one, and the error caused by the skipping rule of the task will not continue to accumulate.

[0110] Under the two exit settings, the difference between the exit distribution of the task under the proposed exit rule and the optimal exit distribution is statistically analyzed to evaluate the probability of ② occurring. Figure 3 It can be found that when all candidate logic layers have exits, under the proposed exit rule, the difference between the task exit e2 and the optimal exit F is 3.05%. Figure 4 ,It can be found that the difference rate is 1.82% when randomly selecting logic layers 1, 5, 9 and inserting them into the egress branch.

[0111] It can be found that the difference rate mainly comes from insufficient or over-fitting model training, which causes the maximum softmax output of some tasks at the deep layer exit to be smaller than the maximum softmax output of the shallow layer exit. The size of the difference rate can be controlled by optimizing the model training. Here, it is assumed that under the exit rule proposed by the present invention, the task exit distribution is close to the optimal exit distribution of the task, that is, P(e3=k|A), k=1,2...L≈P(K=k|A), k=1,2...L.

[0112] Under this assumption, no matter whether event A or event B occurs, all tasks will exit from the best exit, regardless of the exit of the task in the previous frame, so P(e3=k)≈P(K=k).

[0113]

[0114] Step (5): Obtain key information

[0115] Let μ j It represents the probability that the best exit of the task is in the logical layers 1-j when all candidate logical layers have exits.

[0116] Let the task pass the complete multi-exit model reasoning, record the softmax output of the task at each exit, and the first exit that satisfies the maximum softmax output value higher than the preset threshold is the best exit for the task. jThe calculation method is to divide the number of tasks whose logical layer of the best exit is logical layer 1~j by the total number of tasks.

[0117] Let v j It indicates the probability that the softmax distribution of the task at the first exit is "flatter" than that of the previous frame, and the previous frame exit f>j when all candidate logical layers have exits. The statistical method is that when the slope between the maximum and the second largest softmax value at the first exit of the current frame is lower than that of the task in the previous frame, the number of skippable tasks at the exits set by logical layers 1 to f-1 is increased by one, v j It is equal to the percentage of tasks that can be skipped at the exit set by logical layer j to the total number of tasks.

[0118] When the logical layer j has an exit, x j =1, μ(π) j =μ j ; When there is no exit at logical layer j, x j =0, μ(π) j =μ(π) j-1 Therefore, μ(π) j It can be expressed as:

[0119]

[0120] The above optimization problem can be expressed as:

[0121]

[0122] st

[0123]

[0124]

[0125]

[0126] Step (6), equivalent optimization problem conversion

[0127] Since it is difficult to solve the above problem directly, the optimal exit setting combination can be found by quantifying the redundant calculation amount and benefit brought by inserting an exit at each candidate logic layer.

[0128] When a task attempts to exit at a certain exit, but fails to exit because the exit criteria are not met, the output result of the logical layer can only be passed to a deeper logical layer. The amount of calculation generated when attempting to exit at the exit is called redundant calculation, because this part of the calculation is invalidated, and does not increase the reasoning accuracy, but only increases the reasoning delay of the task. The amount of redundant calculation introduced by inserting an exit branch is related to the exit rate μ of the exit. j -μj-1 And the task skip rate v j Related.

[0129] The redundant calculation introduced by inserting an exit at logical layer j can be expressed as the probability of event A occurring multiplied by the redundant calculation overhead generated by the exit set by the task at logical layer j under the premise of event A occurring, plus the probability of event B occurring multiplied by the redundant calculation overhead generated by the exit set by the task at logical layer j under the premise of event B occurring. It can also be equivalent to assuming that the probability of all tasks exiting one exit at a time minus the redundant calculation amount saved by the task at layer j under the condition that event A and the event "the exit of the task in the previous frame is higher than j" occur. The task will have 1-μ j-1 The probability of reaching the logical layer j is μ j -μ j-1 The probability task exits at this layer exit, so it will generate However, when event A occurs and the task exit f in the previous frame is higher than the logical layer j, the task will be allowed to skip the layer exit, and the expected reduction in redundant calculation is

[0130] Therefore, the expected redundant computation introduced by setting the export at the logical layer j can be expressed as:

[0131]

[0132] When no exit is set at logical layer j, tasks that originally meet the exit criteria at logical layer j can only exit at the exit of logical layer k. Setting an exit at logical layer j can save some tasks from the reasoning calculation between logical layer j+1 and logical layer k.

[0133] Compared with setting an exit at layer j, the expected benefit of reasoning about a task can be expressed as:

[0134]

[0135] The logical layer k is the logical layer where the next exit after the jth layer exit is located, and μ k-1 =μ j , μ k ≥μ j .

[0136] In this formula, μ j Instead of using the probability μ of the task exiting at the exit set at logical layer j j -μ j-1This is because the amount of computation actually saved by the task that exits at the exit set at logical layer j is from the j+1th logical layer to the Lth logical layer, but only the amount of computation from logical layer j+1 to logical layer k is saved due to inserting an exit at layer j. The amount of computation saved by the task that exits at the exit set at logical layer j from logical layer k to the next exit of logical layer k should be included in the benefit of inserting an exit at logical layer k. Similarly, the amount of computation saved by the task that exits from logical layer 1 to logical layer j-1 from logical layer j+1 to logical layer k should be attributed to the benefit of inserting an exit at logical layer j.

[0137] By setting an exit at the logic layer j, the difference between the revenue generated by a task and the amount of redundant computation generated can be expressed as:

[0138] R (k)j =I (k)j -C j

[0139] When the export is set to X, the difference between the total benefit and the total redundant computation of the entire model for reasoning a task can be expressed as:

[0140] R(X)=I X -C X

[0141] Suppose that the computational cost of a task through the reasoning of the original model is A, where A is a constant, so we have:

[0142] R(X)+O(X)=A

[0143] Since A is a constant, solving X when O(X) is the smallest is solving X when R(X) is the largest. Therefore, the above approximated optimization problem can be transformed into:

[0144] arg max(R(X))

[0145] x j ·R (k)j >=0,j∈{1,2,3...L}

[0146]

[0147]

[0148]

[0149]

[0150] Step (7): export setting method based on ant colony algorithm

[0151] It is not difficult to find that inserting an exit at the logical layer j∈L can reduce the expected redundant computational size C of a task.j Regardless of whether other logic layers insert the exit branch, the benefit I of inserting the exit of logic layer j is (k)j The next exit after logical layer j is located at the logical layer number k. The farther the next exit is from logical layer j, the greater the benefit of inserting the exit in logical layer j. Since logical layer L must have the final exit, it is more convenient to find the logical layer where the exit is inserted one by one from the final exit to the shallower layer. (k)j Inspired by the ant colony algorithm, the present invention applies the idea of ​​the ant colony algorithm to solve the problem of optimal export setting.

[0152] Reference Figure 5 Before the ant colony starts moving, the pheromone concentration between every two logical layers is initialized to τ0, and then m ants are placed in the last layer of the neural network model. This logical layer is used as the starting point of the ants, and the layer where the first exit is artificially specified, 'first', is used as the end point of the ant movement. Starting from the last layer of the neural network, the ants can only move in the direction of the shallower logical layer. Each logical layer they pass through will have an exit branch inserted. A movement path of an ant is a combination of exit settings. When a group of ants completes a complete movement, the pheromone concentration on each path will be updated. Let Q represent the movement of an ant in a unit R (i)j The greater the difference between the selected exit setting benefit and the redundant calculation amount, the greater the pheromone concentration released by the ants on the path.

[0153] The embodiment of the present invention selects the ant cycle model as the pheromone concentration update model. The ant cycle model makes full use of global information and updates the pheromone concentration on the path only after all ants complete a cycle, which can improve the search efficiency and avoid premature convergence to the local optimal solution.

[0154] The update method is:

[0155]

[0156] Where s∈m represents one of the m ants. represents the pheromone concentration released by ant s from logical level i to logical level j, R s (X) represents the difference between the total revenue of the export setting combination generated by the ant movement and the total redundant calculation.

[0157] After each round of movement, the pheromone concentration on each path evaporates with the same probability ρ (0<ρ<1).

[0158]

[0159] Where i, j∈{first, first+1, first+2, first+3, ..., L}, i>j represents the number of logical layers, ρ(0<ρ<1) represents the volatility of pheromone, Δτ ij Represents the pheromone concentration released by all ants from logical level i to logical level j.

[0160] The ants will choose the next logical layer to visit based on the pheromone concentration on the path. The higher the pheromone concentration between the layer where the ant is located and the next candidate accessible logical layer, the higher the probability that the ant will choose to visit that logical layer.

[0161] use The probability of ant s moving from logical level i to logical level j at time t can be expressed as:

[0162]

[0163] where e∈allow s (s=1, 2, ..., L) represents the set of logical layers accessible to ant s candidates, allowing s Initially, there are L-1-'first' elements, excluding the starting point logical level L and logical levels lower than 'first'. It represents the difference between the benefit and the redundant computation when ant s moves from logical layer i to logical layer j, that is, inserting an exit at logical layer j and the next exit of logical layer j is logical layer i. α is the important factor of pheromone concentration. The larger α is, the greater the role of pheromone concentration in path selection. β is the important factor of the difference between benefit and redundant computation. The larger β is, the greater the role of the difference between benefit and redundant computation in path selection. Every time ant s completes a cycle, it needs to calculate the R of this path. s (X), if it is greater than the difference between the benefit of the current optimal solution and the redundant calculation amount, the optimal solution is updated.

[0164] A maximum number of iterations is set for the ant colony in advance. When the number of ant colony cycles reaches the maximum value, the cycle ends.

[0165] The ant colony algorithm can be used to obtain the optimal setting of the remaining exits under the premise that the first exit position has been determined. By enumerating the combination of the first exit position and the remaining exit settings, the combination that can minimize the expected reasoning overhead of the task will be selected as the final exit setting solution.

[0166] Based on the same technical concept as the method embodiment, the present invention also provides an outlet setting device for a multi-outlet neural network, comprising:

[0167] The module for constructing the exit setting optimization problem is used to abstract the neural network into several logical layers and a final exit branch, quantify the reasoning overhead of the logical layers and exit branches, and construct the exit setting optimization problem for multi-exit neural network task reasoning. The goal of this optimization problem is to find an exit setting scheme that minimizes the expected computational consumption of the task and thus minimizes the reasoning latency. Based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated to obtain a form that is easy to solve.

[0168] The key information acquisition module is used to determine the weight of each outlet branch by inserting outlets in the logic layer and training with video tasks based on the abstracted neural network model, reasoning the video task frame by frame on the model with trained weights, determining the relative reasoning difficulty between consecutive frames based on the classification output values ​​of the previous and next frame tasks, and counting the probability of each outlet being skipped by a task;

[0169] The optimization problem conversion and solution module is used to convert the original approximated optimization problem into an equivalent optimization problem based on the difference between the benefit and redundant computation introduced by inserting the exit branch according to the relative reasoning difficulty between consecutive frames and the probability that an exit is skipped by the task, and use the search method based on the ant colony algorithm to solve the new optimization problem, and finally obtain the exit setting scheme with the largest difference between the benefit and the redundant computation as the final exit setting scheme of the multi-exit neural network.

[0170] According to an embodiment of the present invention, the exit setting optimization problem construction module abstracts the neural network into several logic layers and a final exit branch, quantifies the reasoning overhead of the logic layer and the exit branch, and constructs the exit setting optimization problem of multi-exit neural network task reasoning, including:

[0171] Abstract the neural network model: According to the neural network composed of convolution layer, pooling layer, relu function, fully connected layer, soffmax layer, the original model is abstracted into several logical layers and a final output. The fully connected layer and soffmax layer at the end of the model are abstracted as the final output. Each convolution layer and all the pooling layers and relu functions from the convolution layer to the next convolution layer or the final output are abstracted as a logical layer.

[0172] Quantify the inference cost of the logic layer and the export branch: Use floating-point operations to represent the expected computational cost of model inference. The floating-point operations of the j-th logic layer are expressed as Indicates that It represents the expected floating point operations generated by the exit branch reasoning of a task set by the logic layer j;

[0173] Optimization problem modeling: Consider a neural network model with L logical layers. The goal of the optimization problem is to find an exit setting combination X = {x1, x2, ...x j ,...x L |x j ∈{0, 1}, j∈L}, where x j Indicates whether an exit branch is inserted in the jth logical layer. If inserted, it is 1, otherwise it is 0. The logical layer 1 and the logical layer L have the first exit and the final exit respectively. There is x1 = 1, x L =1; let event A be the classification output distribution of the current frame at the first exit being “flat” compared to the previous frame, event B be the classification output distribution of the current frame at the first exit being “unflat” compared to the previous frame, T(π) represents the calculation of the expected consumption of task reasoning when the exit is set to π, e2 represents the exit of the task in the previous frame when the exit is set to π, μB(π) j It represents the probability that the task exits from the exit set by logic layer 1~i when event B occurs when the exit is set to π; then when the exit is set to π, the expected consumption generated by task reasoning is expressed as:

[0174] E[T(π)]=P(A)·E[T(π)|A]+P(B)·E[T(π)|B]

[0175] in,

[0176]

[0177]

[0178] e2=f means that the previous frame exits from the exit set by the logical layer f.

[0179] According to an embodiment of the present invention, the exit setting optimization problem construction module approximates the original optimization problem based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, including:

[0180] Assuming that when event A or B occurs, the task exits from the best exit, the expected approximate value of the consumption expected to be generated by task reasoning is expressed as:

[0181]

[0182] Among them, μ(π) j represents the probability of the task exiting from the exit set at logical layer 1~j when the exit is set to π; v j represents the probability that the task skips the exit set by the logic layer j;

[0183] Let μ jrepresents the probability that the best exit of the task is in the logical layers 1 to j when all candidate logical layers have exits; when the logical layer j has an exit, x j =1, μ(π) j =μ j ; When there is no exit at logical layer j, x j =0, μ(π) j =μ(π) j-1 , then μ(π) j Expressed as: μ(π) j =max i∈{0，1，2，3...j} {x i ·μ i}.

[0184] According to an embodiment of the present invention, the key information acquisition module inserts outlets at different logical layers and determines the weight of each outlet branch by training using video tasks, including:

[0185] For the neural network whose logic layer meets the specified conditions, the export branch is inserted in all the logic layers, and the video task is used for training to obtain the weight coefficient of each export branch;

[0186] For the neural network whose logic layer does not meet the specified conditions, the weight coefficient of each export branch is obtained by training by inserting exports in different logic layers multiple times.

[0187] According to an embodiment of the present invention, the key information acquisition module determines the relative reasoning difficulty between consecutive frames according to the classification output values ​​of the previous and next frame tasks, and counts the probability of each exit being skipped by the task, including:

[0188] Compare the confidence of the feature map and classification output value extracted by the current frame task i at the first exit with the intermediate features and classification output value extracted by the previous frame task i-1 at this exit. If the former is higher than the latter, it is determined that the reasoning difficulty of task i is higher. Task i will skip all exit branches lower than the exit of task i-1 in subsequent reasoning. Task i will try to exit one by one from the exit of task i-1 until the exit requirements are met or the final exit is reached. If the reasoning difficulty of task i cannot be determined to be higher than that of task i-1, task i will follow the traditional exit rules and try to exit one by one at the exit.

[0189] According to the classification output distribution, the optimal exit distribution of the task is obtained and μ is estimated. j size;

[0190] According to the maximum classification output distribution of the task and the maximum output value of the continuous frames at the first exit, estimate v j size.

[0191] According to an embodiment of the present invention, the optimization problem conversion and solution module converts the original approximated optimization problem into an equivalent optimization problem based on the difference between the benefit introduced by inserting the exit branch and the redundant calculation amount according to the relative reasoning difficulty between consecutive frames and the probability that each exit is skipped by the task, including:

[0192] The expected redundant computation introduced by setting the export at the logical layer j is expressed as:

[0193]

[0194] Compared with setting an exit at layer j, the expected benefit of reasoning about a task is expressed as:

[0195]

[0196] The logical layer k is the logical layer where the next exit after the jth layer exit is located, and μ k-1 =μ j , μ k ≥μ j ;

[0197] Then, an exit is set at the logic layer i, and the difference between the revenue generated by a task and the redundant computing amount generated is expressed as:

[0198] R (k)j =I (k)j -C j

[0199] When the export is set to X, the difference between the total benefit and the total redundant computation of the entire model for reasoning a task is expressed as:

[0200] R(X)=I X -C X

[0201] Suppose the computational cost of a task through the reasoning of the original model is A, where A is a constant, then:

[0202] R(X)+O(X)=A

[0203] Since A is a constant, solving X when O(X) is the smallest is solving X when R(X) is the largest. Therefore, the approximated optimization problem is transformed into:

[0204] arg max(R(X))

[0205] x j ·R (k)j >=0,j∈{1,2,3...L}

[0206]

[0207]

[0208] According to an embodiment of the present invention, the optimization problem conversion and solution module uses a search method based on an ant colony algorithm to solve a new optimization problem, including:

[0209] a) Initializing the location of the logical layer where the first exit is located;

[0210] b) Initialize the pheromone concentration important factor α, the difference between the benefit and the redundant calculation amount important factor β, the pheromone volatility ρ, the number of ants m, the maximum number of iterations, and the pheromone concentration of each path;

[0211] c) At the beginning of each iteration, m ants are placed at the final exit of the neural network and move toward the shallow logic layer. The ants are asked to randomly choose the next step according to the pheromone concentration of different paths until they reach the first exit.

[0212] d) A movement path of an ant is a combination of exit settings. When a group of ants completes a complete movement, the pheromone concentration on each path will be updated. The increase in pheromone concentration depends on the difference between the corresponding exit setting method benefit and the redundant calculation amount. At the end of each round of iteration, the pheromone concentration of each path is reduced according to the pheromone volatility ρ;

[0213] e) Repeat c)-d) until the maximum number of iterations is reached;

[0214] f) The path with the largest pheromone concentration is the optimal exit setting method under this first exit setting mode, the position of the first exit is reset, and b)-e) are repeated to obtain the optimal exit setting method under different first exit setting modes;

[0215] g) The exit setting scheme with the largest difference between the benefit and the redundant computation amount is taken as the final exit setting scheme of the multi-exit neural network.

[0216] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the exit setting method for a multi-exit neural network as described above are implemented.

[0217] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for setting an outlet for a multi-outlet neural network as described above are implemented.

[0218] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the specific implementation modes of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for setting an outlet for a multi-outlet neural network, characterized in that: The following steps are involved: Abstract the neural network into several logic layers and a final exit branch, quantify the reasoning overhead of the logic layer and the exit branch, and construct an exit setting optimization problem for multi-exit neural network task reasoning. The goal of this optimization problem is to find an exit setting scheme that minimizes the expected computational consumption of the task and minimizes the reasoning delay. Based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated to obtain a form that is easy to solve. According to the abstracted neural network model, the weight of each exit branch is determined by inserting exits at different logical layers and training with video tasks. The video task is inferred frame by frame on the model with trained weights. The relative inference difficulty between consecutive frames is determined according to the classification output values ​​of the previous and next frame tasks, and the probability of each exit being skipped by a task is counted. According to the relative reasoning difficulty between consecutive frames and the probability that each exit is skipped by the task, the original approximated optimization problem is converted into an equivalent optimization problem based on the difference between the benefit introduced by inserting the exit branch and the redundant computation amount. The new optimization problem is solved by a search method based on the ant colony algorithm, and finally the exit setting scheme with the largest difference between the benefit and the redundant computation amount is obtained as the final exit setting scheme of the multi-exit neural network.

2. The method according to claim 1, characterized in that Abstract the neural network into several logic layers and a final export branch, quantify the reasoning overhead of the logic layer and export branch, and construct the export setting optimization problem of multi-export neural network task reasoning, including: Abstract the neural network model: According to the neural network composed of convolution layer, pooling layer, relu function, fully connected layer, and softmax layer, the original model is abstracted into several logical layers and a final output. The fully connected layer and softmax layer at the end of the model are abstracted as the final output. Each convolution layer and all the pooling layers and relu functions from the convolution layer to the next convolution layer or the final output are abstracted as a logical layer. Quantify the inference cost of the logic layer and the export branch: Use floating-point operations to represent the expected computational cost of model inference. The floating-point operations of the j-th logic layer are expressed as Indicates that It represents the expected floating point operations generated by the exit branch reasoning of a task set by the logic layer j; Optimization problem modeling: Consider a neural network model with L logical layers. The goal of the optimization problem is to find an output setting combination X = {x1, x2, ... x j ,...x L |x j ∈{0,1},j∈L}, where x j Indicates whether an exit branch is inserted in the jth logical layer. If inserted, it is 1, otherwise it is 0. The logical layer 1 and the logical layer L have the first exit and the final exit respectively. There are x1=1, x l =1; let event A be the classification output distribution of the current frame at the first exit being "flat" compared to the previous frame, event B be the classification output distribution of the current frame at the first exit being "unflat" compared to the previous frame, T(π) represents the calculation of the expected consumption of task reasoning when the exit is set to π, e2 represents the exit of the task in the previous frame when the exit is set to π, μB(π) j It represents the probability that the task exits from the exits set by logic layers 1 to j when event B occurs when the exit is set to π; then when the exit is set to π, the expected consumption generated by task reasoning is expressed as: E[T(π)]=P(A)·E[T(π)|A]+P(B)·E[T(π)|B] in, e2=f means that the previous frame exits from the exit set by the logical layer f.

3. The method according to claim 2, characterized in that Based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated, including: Assuming that when event A or B occurs, the task exits from the best exit, the expected approximate value of the consumption expected to be generated by task reasoning is expressed as: Among them, μ(π) j represents the probability of the task exiting from the exit set at logical layer 1~j when the exit is set to π; v j represents the probability that the task skips the exit set by the logic layer j; Let μ j represents the probability that the best exit of the task is in the logical layers 1 to j when all candidate logical layers have exits; when the logical layer j has an exit, x j =1, μ(π) j =μ j ; When there is no exit at logical layer j, x j =0, μ(π) j =μ(π) j-1 , then μ(π) j Expressed as: μ(π) j =max i∈{0,1,2,3...j} {x i ·μ i }.

4. The method according to claim 1, characterized in that: The weight of each export branch is determined by inserting exports at different logical layers and training using video tasks, including: For the neural network whose logic layer meets the specified conditions, the export branch is inserted in all the logic layers, and the video task is used for training to obtain the weight coefficient of each export branch; For the neural network whose logic layer does not meet the specified conditions, the weight coefficient of each export branch is obtained by training by inserting exports in different logic layers multiple times.

5. The method according to claim 2, characterized in that: The relative difficulty of reasoning between consecutive frames is determined based on the classification output values ​​of the previous and next frame tasks, and the probability of each exit being skipped by a task is counted, including: Compare the confidence of the feature map and classification output value extracted by the current frame task i at the first exit with the intermediate features and classification output value extracted by the previous frame task i-1 at this exit. If the former is higher than the latter, it is determined that the reasoning difficulty of task i is higher. Task i will skip all exit branches lower than the exit of task i-1 in subsequent reasoning. Task i will try to exit one by one from the exit of task i-1 until the exit requirements are met or the final exit is reached. If the reasoning difficulty of task i cannot be determined to be higher than that of task i-1, task i will follow the traditional exit rules and try to exit one by one at the exit. According to the classification output distribution, the optimal exit distribution of the task is obtained and μ is estimated. j size; According to the maximum classification output distribution of the task and the maximum output value of the continuous frames at the first exit, estimate v j size.

6. The method according to claim 5, characterized in that According to the relative reasoning difficulty between consecutive frames and the probability of each exit being skipped by a task, the original approximated optimization problem is converted into an equivalent optimization problem based on the difference between the benefit introduced by inserting the exit branch and the redundant computation, including: The expected redundant computation introduced by setting the export at the logical layer j is expressed as: Compared with setting an exit at layer j, the expected benefit of reasoning about a task is expressed as: The logical layer k is the logical layer where the next exit after the jth layer exit is located, and μ k-1 =μ j ,μ k ≥μ j ; Then, an exit is set at the logic layer j, and the difference between the revenue generated by a task and the redundant computing amount generated is expressed as: R (k)j =I (k)j -C j When the export is set to X, the difference between the total benefit and the total redundant computation of the entire model for reasoning a task is expressed as: R(X)=I X -C X Suppose the computational cost of a task through the reasoning of the original model is A, where A is a constant, then: R(X)+O(X)=A Since A is a constant, solving X when O(X) is the smallest is solving X when R(X) is the largest. Therefore, the approximated optimization problem is transformed into: arg max(R(X)) x j ·R (k)j >=0,j∈{1,2,3...L} 7. The method according to claim 2, characterized in that Use ant colony algorithm-based search methods to solve new optimization problems, including: a) Initializing the location of the logical layer where the first exit is located; b) Initialize the pheromone concentration important factor α, the difference between the benefit and the redundant calculation amount important factor β, the pheromone volatility ρ, the number of ants m, the maximum number of iterations, and the pheromone concentration of each path; c) At the beginning of each iteration, m ants are placed at the final exit of the neural network and move toward the shallow logic layer. The ants are asked to randomly choose the next step according to the pheromone concentration of different paths until they reach the first exit. d) A movement path of an ant is a combination of exit settings. When a group of ants completes a complete movement, the pheromone concentration on each path will be updated. The increase in pheromone concentration depends on the difference between the corresponding exit setting method benefit and the redundant calculation amount. At the end of each round of iteration, the pheromone concentration of each path is reduced according to the pheromone volatility ρ; e) Repeat c)-d) until the maximum number of iterations is reached; f) The path with the largest pheromone concentration is the optimal exit setting method under this first exit setting mode, the position of the first exit is reset, and b)-e) are repeated to obtain the optimal exit setting method under different first exit setting modes; g) The exit setting scheme with the largest difference between the benefit and the redundant computation amount is taken as the final exit setting scheme of the multi-exit neural network.

8. An outlet setting device for a multi-outlet neural network, characterized in that: include: The module for constructing the exit setting optimization problem is used to abstract the neural network into several logical layers and a final exit branch, quantify the reasoning overhead of the logical layers and exit branches, and construct the exit setting optimization problem for multi-exit neural network task reasoning. The goal of this optimization problem is to find an exit setting scheme that minimizes the expected computational consumption of the task and thus minimizes the reasoning latency. Based on the assumption that the actual exit distribution of the task is similar to the optimal exit distribution of the task, the original optimization problem is approximated to obtain a form that is easy to solve. The key information acquisition module is used to determine the weight of each exit branch by inserting exits at different logical layers and training with video tasks based on the abstracted neural network model, reasoning the video task frame by frame on the model with trained weights, determining the relative reasoning difficulty between consecutive frames based on the classification output values ​​of the previous and next frame tasks, and counting the probability of each exit being skipped by a task; The optimization problem conversion and solution module is used to convert the original approximated optimization problem into an equivalent optimization problem based on the difference between the benefit and redundant computation introduced by inserting the exit branch according to the relative reasoning difficulty between consecutive frames and the probability that each exit is skipped by the task, and use the search method based on the ant colony algorithm to solve the new optimization problem, and finally obtain the exit setting plan with the largest difference between the benefit and the redundant computation as the final exit setting plan of the multi-exit neural network.

9. A computer device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the outlet setting method for a multi-outlet neural network as described in any one of claims 1 to 7 are implemented.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for setting an outlet for a multi-outlet neural network as described in any one of claims 1 to 7 are implemented.

Images