Multi-user cooperative reasoning delay optimization method and device based on batch processing technology

Abstract

The invention relates to the technical field of delay optimization, in particular to a multi-user cooperative reasoning delay optimization method and device based on the batch processing technology, and the method comprises the steps: obtaining reasoning tasks of a plurality of users, and dividing the reasoning tasks into a plurality of subtasks; traversing the segmentation points of the plurality of subtasks through a target iterative traversal algorithm to find an optimal task segmentation point of the plurality of subtasks, and dividing the plurality of subtasks into a server reasoning subtask and a local reasoning subtask based on the optimal task segmentation point; and calculating the minimum total completion time of the server reasoning sub-task and the local reasoning sub-task based on the actual broadband and the energy constraint condition, and generating a delay optimization strategy of the cooperative reasoning task of the multiple users. According to the method, the task processing delay can be reduced, the system performance of multi-user cooperative reasoning can be effectively improved, the energy consumption of the user equipment can be reduced, the method has relatively high applicability and robustness, and powerful support is provided for application of edge calculation in the fields of Internet of Things and artificial intelligence.

Description

Technical Field

[0001] The present application relates to the technical field of delay optimization, and in particular to a multi-user collaborative reasoning delay optimization method and device based on batch processing technology. Background Art

[0002] In multi-user co-inference scenarios, real-time response and computing power are crucial. With the rapid development of the Internet of Things (IoT) and Artificial Intelligence (AI), edge computing, as a supplement to cloud computing, provides powerful data processing capabilities to meet these needs by performing computing tasks on devices or servers at the edge of the network.

[0003] The related technologies mainly include Deep Neural Networks (DNN), Computation Offloading, and hardware accelerators (such as Graphics Processing Units (GPUs) and Neural Processing Units (NPUs)). These technologies have explored ways to deploy DNN models on distributed edge processors to improve computing efficiency through task offloading and hardware accelerators. Task aggregation technology is also a solution. For example, by segmenting DNN and using edge offloading technology, the latency and throughput of collaborative reasoning can be optimized.

[0004] However, related technologies mainly focus on the reasoning tasks of a single user or a small number of users, and do not deeply explore the batch task optimization problem in multi-user collaborative reasoning scenarios. They are unable to cope with the situation in actual applications where multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time. How to optimize user reasoning task delays through resource allocation and reasonable use of performance is an urgent problem to be solved. Summary of the invention

[0005] The present application provides a multi-user collaborative reasoning delay optimization method and device based on batch processing technology to solve the problem that the related technology mainly focuses on the reasoning tasks of a single user or a small number of users, but does not deeply explore the batch processing task optimization problem in the multi-user collaborative reasoning scenario. It is unable to cope with the situation in actual applications where multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time, and how to optimize user reasoning task delays through resource allocation and reasonable use of performance.

[0006] The first aspect of the present application provides a multi-user collaborative reasoning delay optimization method based on batch processing technology, comprising the following steps: obtaining reasoning tasks of multiple users, and dividing the reasoning tasks into multiple subtasks; traversing the splitting points of the multiple subtasks through a target iterative traversal algorithm to find the optimal task splitting points of the multiple subtasks, and dividing the multiple subtasks into server reasoning subtasks and local reasoning subtasks based on the optimal task splitting points; based on actual bandwidth and energy constraints, calculating the minimum total completion time of the server reasoning subtask and the local reasoning subtask, and generating a delay optimization strategy for the multiple user collaborative reasoning tasks according to the minimum total completion time.

[0007] Optionally, in one embodiment of the present application, the target iterative traversal algorithm traverses the splitting points of the multiple subtasks to find the optimal task splitting point of the multiple subtasks, and divides the multiple subtasks into server reasoning subtasks and local reasoning subtasks based on the optimal task splitting point, including: dividing the subtasks before the optimal task splitting point to the target server for processing to obtain the server reasoning subtask; dividing the subtasks after the optimal task splitting point to the local for processing to obtain the local reasoning subtask.

[0008] Optionally, in one embodiment of the present application, it also includes: when the server reasoning subtask reaches a preset batch processing condition, obtaining a server batch reasoning task; identifying a target batch in the server batch reasoning task through a target maximum batch algorithm, and traversing the server batch reasoning tasks after the target batch through the target iterative traversal algorithm to obtain the minimum total completion time of the server batch reasoning task, and obtaining a delay optimization strategy for the multiple user collaborative reasoning tasks based on the minimum total completion time.

[0009] Optionally, in one embodiment of the present application, the actual bandwidth and energy constraints include: the total energy consumption of any one of the multiple users is constrained to be within a preset energy consumption range; the local computing frequency of any one of the users is constrained to be between the current minimum frequency and the current maximum frequency; the processors of the multiple subtasks are constrained to be local reasoning subtasks; and the initial task start time of the local reasoning subtask is constrained to be 0.

[0010] Optionally, in one embodiment of the present application, the delay optimization strategy of the multiple user collaborative reasoning tasks includes at least one of optimal batch start time, allocation decision, local computing frequency and total completion time.

[0011] The second aspect of the present application provides a multi-user collaborative reasoning delay optimization device based on batch processing technology, including: a segmentation module, used to obtain reasoning tasks of multiple users and divide the reasoning tasks into multiple subtasks; a partitioning module, used to traverse the segmentation points of the multiple subtasks through a target iterative traversal algorithm to find the optimal task segmentation points of the multiple subtasks, and divide the multiple subtasks into server reasoning subtasks and local reasoning subtasks based on the optimal task segmentation points; a generation module, used to calculate the minimum total completion time of the server reasoning subtask and the local reasoning subtask based on actual bandwidth and energy constraints, and generate a delay optimization strategy for the multiple user collaborative reasoning tasks according to the minimum total completion time.

[0012] Optionally, in one embodiment of the present application, the partitioning module includes: a first partitioning unit, used to partition the subtasks before the optimal task partitioning point to the target server for processing to obtain the server reasoning subtask; a second partitioning unit, used to partition the subtasks after the optimal task partitioning point to the local for processing to obtain the local reasoning subtask.

[0013] Optionally, in one embodiment of the present application, it also includes: an acquisition module, used to acquire a server batch reasoning task when the server reasoning subtask reaches a preset batch condition; a traversal module, used to identify a target batch in the server batch reasoning task through a target maximum batch algorithm, and traverse the server batch reasoning tasks after the target batch through a target iterative traversal algorithm to obtain the minimum total completion time of the server batch reasoning task, and generate a delay optimization strategy for the multiple user collaborative reasoning tasks based on the minimum total completion time.

[0014] Optionally, in one embodiment of the present application, the actual bandwidth and energy constraints include: the total energy consumption of any one of the multiple users is constrained to be within a preset energy consumption range; the local computing frequency of any one of the users is constrained to be between the current minimum frequency and the current maximum frequency; the processors of the multiple subtasks are constrained to be local reasoning subtasks; and the initial task start time of the local reasoning subtask is constrained to be 0.

[0015] Optionally, in one embodiment of the present application, the delay optimization strategy of the multiple user collaborative reasoning tasks includes at least one of optimal batch start time, allocation decision, local computing frequency and total completion time.

[0016] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the multi-user collaborative reasoning delay optimization method based on batch processing technology as described in the above embodiment.

[0017] The fourth aspect embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, it implements the above-mentioned multi-user collaborative reasoning delay optimization method based on batch processing technology.

[0018] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned multi-user collaborative reasoning delay optimization method based on batch processing technology.

[0019] The embodiment of the present application can divide the multiple subtasks of the reasoning task into server reasoning subtasks and local reasoning subtasks based on the target iterative traversal algorithm, and generate a delay optimization strategy for multiple user collaborative reasoning tasks according to the minimum total completion time of the task. Thus, a method based on task segmentation and scheduling algorithm is realized, and a traversal algorithm and a maximum batch algorithm are proposed, which significantly reduces the task processing delay. Considering the actual delay of the server, in the scenario where the server bandwidth is limited or the user device capabilities differ significantly, it can effectively improve the system performance of multi-user collaborative reasoning under different conditions and in different scenarios, reduce the energy consumption of user devices, and has high applicability and robustness, providing strong support for the application of edge computing in the fields of Internet of Things and artificial intelligence. Thus, it solves the problem that the related technology mainly focuses on the reasoning tasks of a single user or a few users, and does not deeply explore the optimization problem of batch processing tasks in the multi-user collaborative reasoning scenario. It is impossible to deal with the situation in which multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time in actual applications, and how to optimize the delay of user reasoning tasks through resource allocation and reasonable use of performance.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a multi-user collaborative reasoning delay optimization method based on batch processing technology provided according to an embodiment of the present application;

[0023] Figure 2A schematic diagram of a system framework of a batch task calculation optimization scheduling algorithm according to an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the network structure and layer dimensions of a 3DSSD model according to an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the inference delay of a 3DSSD neural network model on a batch processing server according to an embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of how the average inference delay of each user changes with the number of users under different bandwidths according to an embodiment of the present application;

[0027] Figure 6 This is a schematic diagram showing how the average inference delay of each user changes with the number of users under different computing capabilities according to an embodiment of the present application;

[0028] Figure 7 This is a schematic diagram showing how the average inference delay of each user varies with the number of users under different user performances according to an embodiment of the present application;

[0029] Figure 8 It is a structural schematic diagram of a multi-user collaborative reasoning delay optimization device based on batch processing technology according to an embodiment of the present application;

[0030] Fig. 9 Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0031] Reference numerals:

[0032] 10-Multi-user collaborative reasoning delay optimization device based on batch processing technology: 100-segmentation module, 200-division module and 300-generation module; 901-memory, 902-processor and 903-communication interface. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the multi-user collaborative reasoning delay optimization method and device based on batch processing technology of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the related technologies mentioned in the above background technology mainly focus on the reasoning tasks of a single user or a few users, and do not deeply explore the batch processing task optimization problem in the multi-user collaborative reasoning scenario, it is impossible to deal with the situation in which multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time in actual applications, and how to optimize the user reasoning task delay through resource allocation and reasonable use of performance. The present application provides a multi-user collaborative reasoning delay optimization method based on batch processing technology, in which the multiple subtasks into which the reasoning task is divided can be divided into server reasoning subtasks and local reasoning subtasks based on the target iterative traversal algorithm, and the delay optimization strategy of multiple user collaborative reasoning tasks is generated according to the minimum total completion time of the task. Thus, a method based on task segmentation and scheduling algorithm is realized, and a traversal algorithm and a maximum batch algorithm are proposed, which significantly reduces the task processing delay. Considering the actual delay of the server, in the scenario where the server bandwidth is limited or the user device capabilities differ significantly, the system performance of multi-user collaborative reasoning under different conditions and in different scenarios can be effectively improved, and the energy consumption of user devices can be reduced. It has high applicability and robustness, and provides strong support for the application of edge computing in the fields of Internet of Things and artificial intelligence. This solves the problem that the related technologies mainly focus on the reasoning tasks of a single user or a small number of users, but do not deeply explore the batch task optimization problem in the multi-user collaborative reasoning scenario. It is unable to cope with the situation in actual applications where multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time, and how to optimize user reasoning task delays through resource allocation and reasonable use of performance.

[0035] Before explaining the multi-user collaborative reasoning delay optimization method based on batch processing technology in the embodiment of the present application, the background technology involved in the embodiment of the present application is first explained.

[0036] Assume that a deep neural network (DNN) reasoning task is divided into n subtasks, denoted as {1,2,…,N}. Let the computational workload of the nth subtask be A n , the size of the intermediate data after completing the nth subtask is B n Assume that M users are connected to the same edge server and the local computation frequency of user m is f m When user m processes the nth subtask, the local inference delay calculation formula can be expressed as follows:

[0037]

[0038] in, is the local inference latency, A n is the computational workload of the nth subtask, fm The local calculation frequency of user m.

[0039] The corresponding computational energy consumption can be expressed as:

[0040]

[0041] in, The local frequency calculated for user m is f m The corresponding computing energy consumption in the case of m is the effective value of the switching capacitor.

[0042] Assume that the uplink transmission rate of user m is The transmission power is The upload transmission delay and upload energy consumption can be expressed as:

[0043]

[0044] in, and They are upload transmission delay and upload energy consumption respectively.

[0045] For the subtasks offloaded to the edge server, record x m,n,k represents the decision of user m's nth subtask to be processed in the kth batch. m,n,k = 1, indicating that the corresponding subtask is processed in the kth batch, otherwise x m,n,k = 0. The batch size of the nth subtask in the batch is b n,k ,The computation delay of the edge server can be expressed as follows:

[0046] F n (b n,k ).

[0047] The embodiments of the present application can minimize the total completion time of reasoning tasks for all users under bandwidth and energy constraints.

[0048] Specifically, Figure 1 A flowchart of a multi-user collaborative reasoning delay optimization method based on batch processing technology provided in an embodiment of the present application.

[0049] like Figure 1 As shown, the multi-user collaborative reasoning delay optimization method based on batch processing technology includes the following steps:

[0050] In step S101 , reasoning tasks of multiple users are obtained, and the reasoning tasks are divided into multiple subtasks.

[0051] In some embodiments, multiple users often join a multi-user collaborative reasoning system, where the multi-user collaborative reasoning system can be understood as a complex system in which multiple users or agents jointly participate in the reasoning process to solve complex problems.

[0052] The reasoning task here refers to a complex problem that requires multiple users to participate in solving in a multi-user collaborative reasoning system. Since reasoning tasks are usually highly complex, problems such as reasoning delays often occur. Therefore, after assigning reasoning tasks to different users, the embodiment of the present application can divide the reasoning tasks assigned to each user into multiple subtasks, and process the subtasks separately to facilitate the allocation and scheduling of subsequent tasks. It should be noted that the multiple subtasks here can be understood as subtasks that include all users' reasoning tasks.

[0053] For example, in an edge computing system, where M users are connected to the same edge server with batch processing capabilities, the Deep Neural Networks (DNN) inference task can be split into several subtasks, which can be processed locally or uploaded to the edge server for processing.

[0054] The embodiment of the present application can obtain reasoning tasks of multiple users and divide them into multiple subtasks, so as to simplify the process of task allocation and scheduling based on the calculation of the subtasks.

[0055] Step S102, traversing the splitting points of multiple subtasks through the target iterative traversal algorithm to find the optimal task splitting points of the multiple subtasks, and dividing the multiple subtasks into server reasoning subtasks and local reasoning subtasks based on the optimal task splitting points.

[0056] It can be understood that the target iterative traversal algorithm here can be understood as an algorithm for iterative traversal of partition points (Traversal Algorithm for Partition Point).

[0057] In the actual execution process, the inference task can be divided into several subtasks, some of which are uploaded to the edge server for processing, and some are processed locally. In this division process, the division is performed in an appropriate way so that the subtasks processed on the server and the subtasks processed locally can be combined to achieve the best processing effect.

[0058] Based on this, the embodiment of the present application can propose an iterative traversal algorithm to traverse the split points of the subtasks, and then find an optimal task split point to divide the subtasks into server processing subtasks and local processing subtasks.

[0059] For example, assuming that the computational delay of the edge server is independent of the batch size, the optimal task splitting point can be found by traversing the splitting points through the target iterative traversal algorithm. Based on the optimal task splitting point, multiple subtasks can be split, and some subtasks can be uploaded to the edge server for processing, while the remaining tasks are processed locally.

[0060] The embodiment of the present application can propose an algorithm for iteratively traversing the split points, namely, a target iterative traversal algorithm, and traverse the split points based on the algorithm to find the optimal task split points, thereby optimizing the overall reasoning delay through appropriate task splitting.

[0061] Optionally, in one embodiment of the present application, the splitting points of multiple subtasks are traversed through a target iterative traversal algorithm to find the optimal task splitting point of the multiple subtasks, and the multiple subtasks are divided into server reasoning subtasks and local reasoning subtasks based on the optimal task splitting point, including: dividing the subtasks before the optimal task splitting point to the target server for processing to obtain server reasoning subtasks; dividing the subtasks after the optimal task splitting point to the local for processing to obtain local reasoning subtasks.

[0062] Based on the relevant descriptions of other embodiments, it can be understood that based on the optimal task splitting point, the embodiment of the present application can divide the subtasks into server processing subtasks and local processing subtasks.

[0063] Specifically, after finding the optimal task split point through the target iterative traversal algorithm, all subtasks before the optimal task split point can be divided into the target server for processing, and all subtasks after the optimal task split point can be divided into the local server for processing.

[0064] Step S103, based on the actual bandwidth and energy constraints, the minimum total completion time of the server reasoning subtask and the local reasoning subtask is calculated, and a delay optimization strategy for multiple user collaborative reasoning tasks is generated according to the minimum total completion time. The delay optimization strategy for multiple user collaborative reasoning tasks includes at least one of the optimal batch start time, allocation decision, local computing frequency, and total completion time.

[0065] As a possible implementation method, after dividing the subtasks into server processing subtasks and local processing subtasks, the present application can calculate the minimum total completion time of the two to generate a delay optimization strategy for multiple multi-user collaborative reasoning tasks based on the minimum total completion time.

[0066] For example, assuming that the computation delay of the edge server is independent of the batch size, let the split point be p, and the calculation process of the minimum total completion time can be expressed as follows:

[0067] (1) Derive the batch start time s according to the formula k , the formula can be expressed as follows:

[0068]

[0069] Among them, B0 represents the input data size of the initial subtask.

[0070] (2) Determine the local computation frequency f for each user m and each cutting point n (here it can be understood as cutting a neural network into subtasks, each subtask is numbered n, and the corresponding cutting point is also n) m,n , the formula can be expressed as follows:

[0071]

[0072] Among them, e m,n is the energy constraint of user m when the cutting point is n, and They represent the energy consumed by uploading and downloading intermediate data, κ m Represents the effective switch capacitance value, which is a constant related to the calculated energy consumption.

[0073] (3) Calculate the completion time t for each user m,N and local computational energy consumption The formula can be expressed as follows:

[0074]

[0075] Among them, t m,N,p It indicates the total completion time required for user m to reason about all subtasks when the cut point is p. represents the local inference latency, Indicates the upload delay. Indicates the download delay.

[0076] (4) For each user m, select the optimal split point to minimize the total completion time. The formula can be expressed as follows:

[0077] n * =argmin t m,N (n)

[0078] And calculate the associated and

[0079] For each user m, the total completion time corresponding to each split point n is calculated as t m,N (n), select the shortest t m,N (n) as The corresponding optimal split point is n * , the local optimal calculation frequency can also be calculated based on the optimal segmentation point and optimal unloading decision

[0080] (5) Return the optimal batch start time s k , allocation decision x m,n,k , local calculation frequency f m and the total completion time t * :

[0081]

[0082] Based on the batch start time s k , allocation decision x m,n,k , local calculation frequency f m and the total completion time t * , generating latency optimization strategies for multiple user collaborative reasoning tasks.

[0083] Optionally, in one embodiment of the present application, it also includes: when the server reasoning subtask reaches a preset batch processing condition, obtaining a server batch reasoning task; identifying a target batch in the server batch reasoning task through a target maximum batch algorithm, and traversing the server batch reasoning tasks after the target batch through a target iterative traversal algorithm to obtain the minimum total completion time of the server batch reasoning task, and generating a delay optimization strategy for multiple user collaborative reasoning tasks based on the minimum total completion time.

[0084] It is understandable that the preset batch processing condition here can be understood as the situation where the reasoning task at this time needs to be batch processed. The target maximum batch algorithm here can be understood as an algorithm optimized based on the target iterative traversal algorithm of the embodiment of the present application, which can identify batches of certain target batch processing sizes, and then apply the target iterative traversal algorithm to traverse the batch processing sizes after these batches to find the optimal task allocation strategy. The target batch here can be understood as a batch processing size value determined according to the actual situation.

[0085] In actual application scenarios, the inference delay of the edge server will change with the change of the batch size. Therefore, the embodiment of the present application proposes a maximum batch algorithm (TALB), which can traverse the batch size, traverse the server batch inference tasks after the target batch, and obtain the minimum total completion time of the server batch inference tasks, and generate and further optimize the delay optimization strategy of multiple user collaborative inference tasks based on the minimum total completion time.

[0086] Figure 2Schematic diagram of the system framework of the batch task calculation optimization scheduling algorithm for an embodiment of this application. As Figure 2 shown, the batch task calculation optimization scheduling algorithm can call the target iterative traversal algorithm to calculate the frequency and segmentation points under ideal conditions for the tasks arriving at the terminal, then call the target maximum batch algorithm to replace the ideal conditions and update some actions, execute the updated batch tasks, and jump to the arrival of the next batch task for loop processing.

[0087] For example, the maximum batch algorithm first identifies batches with a batch size of M, and then applies the target iterative traversal algorithm to traverse the batch sizes after these batches to find the optimal task allocation strategy. The specific steps can be expressed as follows:

[0088] (1) Initialization: Set t* = ∞.

[0089] (2) Call the target iterative traversal algorithm (Traversal Algorithm for Partition Point): Derive all Then find the smallest subtask n such that

[0090] (3) Replace the calculation formula: If n ≤ n*, then replace F n (1) with F n (M).

[0091] (4) Traverse the batch size: For b = M, M - 1,..., 1,

[0092] If n > n*, then replace F n (1) with F n (b);

[0093] Use the new F n (·) to calculate

[0094] Call the target iterative traversal algorithm to derive a feasible solution and as well as

[0095] Calculate the actual batch size

[0096] Calculate the actual maximum batch size where n > n*.

[0097] (5) Update the optimal value: If b max ≤ b and t*(b) < t, then update:

[0098] t" ← t"(b)J

[0099] (6) Return result: Return the optimal batch start time s k , allocation decision x m,n,k , local calculation frequency f m and the total completion time t* to generate a latency-optimized strategy for multiple user collaborative reasoning tasks.

[0100] Optionally, in one embodiment of the present application, the actual bandwidth and energy constraints include: the total energy consumption of any one of multiple users is constrained to be within a preset energy consumption range; the local computing frequency of any one user is constrained to be between the current minimum frequency and the current maximum frequency; the processors of multiple subtasks are constrained to be local reasoning subtasks; and the initial task start time of the local reasoning subtask is constrained to be 0.

[0101] As a possible implementation method, the embodiment of the present application can minimize the total completion time of all users' reasoning tasks under bandwidth and energy constraints. The problem can be defined as follows:

[0102]

[0103] The bandwidth and energy constraints include but are not limited to the following three constraints:

[0104] Specifically, the first constraint is that the total energy consumption of each user must be less than or equal to the user's energy constraint e m , which can be expressed as follows:

[0105]

[0106] in, represents the energy consumption of the mth user when processing the nth subtask locally, represents the energy consumption of the nth subtask uploaded by the mth user, It represents the energy consumption of the mth user downloading the nth subtask.

[0107] The second constraint is that the user locally calculates the frequency f m The value range must be within the minimum frequency f m,min and the maximum frequency f m,max The task completion time t m,n and the batch start time s k Must be a non-negative value and can be represented as follows:

[0108] f m,min ≤f m ≤f m,max ,t m,n ≥0,s k ≥0.

[0109] The third constraint is that in the initial state, the initial allocation decision of all subtasks is 1, that is, all subtasks are processed locally at the initial moment, and the initial task start time is 0, which can be expressed as follows:

[0110] x m,0,0 =1,t m,0 =0.

[0111] The present application is described in detail and the results are verified by using a specific example as follows.

[0112] First, select a network model, for example, the 3DSSD (3D Single Stage Detector) model, a DNN model for object detection tasks. Figure 3 Shown is a schematic diagram of the network structure and layer dimensions of a 3DSSD model of an embodiment of the present application.

[0113] Further, the embodiment of the present application can be performed on a batch server, but is not limited to, to evaluate the inference delay. During the transmission process, the users can be distributed within a radius (R) of 100 meters from the edge server, and the wireless bandwidth can be set to 1 MHz.

[0114] Path loss L p (dB) can be set to 128.1+37.6log10R. The standard deviation of shadow fading is X s is 8dB. At this time, the channel gain h m The calculation formula can be expressed as follows:

[0115]

[0116] By applying Shannon's formula, the transfer rate can be calculated. Further, the required upload time can be determined using the intermediate data size and upload power. The formula can be expressed as follows:

[0117]

[0118] Among them, W m represents the transmission bandwidth allocated to user m, N0 represents the Gaussian white noise power, They represent the square of the channel gain respectively.

[0119] In order to adapt to the different hardware capabilities of users, the embodiments of the present application can adopt an alternative parameterization method, which is expressed by assuming that there is a proportional relationship between the server performance and the average performance of the user. Specifically, the parameter α can be used m The ratio between the local inference latency and the edge server inference latency at the maximum processing frequency of user m can be expressed as follows:

[0120]

[0121] Furthermore, based on the user energy consumption constraint, a method of calculating the local inference delay is adopted. Here, represents the energy consumption of user m processing the nth subtask at split point p.

[0122] The local inference delay calculation formula can be expressed as follows:

[0123]

[0124] in, represents the local reasoning delay of user m processing the nth subtask at split point p, Indicates that user m has the maximum processing frequency f m,max The energy consumption of processing the nth subtask under m represents the ratio between the local inference latency and the edge server inference latency at the maximum processing frequency of user m, F n (1) represents the inference latency of the edge server processing the nth subtask.

[0125] The energy consumption calculation formula can be expressed as follows:

[0126]

[0127] Among them, A n represents the computational workload of the nth subtask, ∈ m (f m,max ) represents the energy efficiency of user m at the maximum processing frequency, ∈ e (f e,max ) represents the energy efficiency of the edge server at the maximum processing frequency, e cp,n,o (f e,max ) represents the energy consumption of the edge server processing the nth subtask at the maximum processing frequency, P e Indicates the power consumption of the edge server.

[0128] The user energy allocation calculation formula can be expressed as follows:

[0129]

[0130] in, represents the energy consumption of processing the nth subtask at split point n, e m represents the total energy constraint of user m, F k (1) represents the inference latency of the edge server processing the kth subtask.

[0131] By using the above formula, we can avoid calculating the intermediate variable f m(which represents the processing frequency of the user under a certain energy consumption constraint), but directly calculates the task completion time based on the energy consumption constraint.

[0132] Therefore, when the split point is p, the inference latency for user m to complete all tasks can be expressed as:

[0133]

[0134] Table 1 is a parameter table of the embodiment of the present application, which can be expressed as follows:

[0135] Table 1

[0136]

[0137]

[0138] Figure 4 This is a schematic diagram of the inference delay of the 3DSSD neural network model on a batch server according to an embodiment of the present application. Figure 4 As shown, it shows the inference latency of the 3DSSD neural network model on a batch server.

[0139] Next, we consider the impact of various factors on the algorithm results.

[0140] (1) Implementation results under different server bandwidths

[0141] In simulation, different numbers of users connected to the edge server can usually be simulated by varying the bandwidth of the server. Figure 5 This is a schematic diagram of how the average inference delay of each user changes with the number of users under different bandwidths according to an embodiment of the present application.

[0142] like Figure 5 As shown in FIG. 1 , it shows the case at 1 MHz and 5 MHz. Then, the target maximum batch algorithm is compared with two baseline algorithms, namely the local computation (LC) strategy and the processing sharing offload (PS) algorithm.

[0143] Under the LC strategy, all users process inference tasks locally. PS offloading means that all users evenly share the computing resources of the edge server, that is, the edge computing delay of the n-th subtask becomes MF. n (1).

[0144] Under the LC strategy, all tasks are processed locally, resulting in an average inference latency of 245.5ms. In the scenario where server bandwidth resources are limited (W = 1MHz), TALB improves the inference latency compared to the LC and PS algorithms. However, as the number of users increases, the average inference latency increases rapidly. This is because the server can only connect to one user at a time, and the remaining users need to process inference tasks locally.

[0145] When the bandwidth resources of the edge server are more abundant, especially when W = 5MHz, when the number of users is less than 5, users choose to upload most of the tasks to the server. In this case, the average inference delay (time-alg) of the server is about 50ms. Compared with the PS strategy under W = 5MHz, TALB reduces the inference delay by about 79.5%. Even if the number of users increases, TALB always outperforms the LC and PS algorithms.

[0146] (2) Implementation results under different computing capabilities

[0147] The embodiment of the present application mainly discusses the changes of parameters related to the computing capability of mobile users, namely, α = 1, α = 1.5, and α = 2. A larger α value indicates that the average performance of the mobile device is lower. Figure 6 This is a schematic diagram of how the average inference delay of each user changes with the number of users under different computing powers of an embodiment of the present application, that is, the implementation results of the embodiment of the present application under different computing powers.

[0148] That is, under each offloading strategy, when processing the same task, a larger user capability parameter leads to an increase in the average user inference latency. However, the overall trend remains consistent and TALB performs well in all cases.

[0149] (3) Implementation results of randomly generated computing power

[0150] In order to explore the impact of different user performance on the uninstallation strategy, the embodiment of the present application can randomly generate the computing capacity parameter α of user m. m . Set mean μ = 1 and standard deviation σ = 1 to generate a normal distribution with parameter α. Figure 7 This is a schematic diagram of how the average inference delay of each user changes with the number of users under different user performances for an embodiment of the present application, that is, it is the implementation result of the embodiment of the present application when the computing power is randomly generated.

[0151] By adopting a method of randomly generating a computing power parameter α, the embodiment of the present application can obtain implementation results that are highly similar to real-life scenarios, and can observe various unloading decisions of users with different computing powers.

[0152] When the parameter α varies, users with higher computing power can process more subtasks locally, thereby allocating edge server resources to users with more urgent needs. Figure 7 The curves in denoting random simulations may be better than the curves using the same parameter α.

[0153] According to the multi-user collaborative reasoning delay optimization method based on batch processing technology proposed in the embodiment of the present application, the multiple subtasks into which the reasoning task is divided can be divided into server reasoning subtasks and local reasoning subtasks based on the target iterative traversal algorithm, and the delay optimization strategy of multiple user collaborative reasoning tasks is generated according to the minimum total completion time of the task. Thus, a method based on task segmentation and scheduling algorithm is realized, and a traversal algorithm and a maximum batch algorithm are proposed, which significantly reduces the task processing delay. Considering the actual server delay, in scenarios where the server bandwidth is limited or the user device capabilities differ significantly, the system performance of multi-user collaborative reasoning under different conditions and in different scenarios can be effectively improved, and the energy consumption of user devices can be reduced. It has high applicability and robustness, and provides strong support for the application of edge computing in the fields of the Internet of Things and artificial intelligence. Thus, it solves the problem that the related technology mainly focuses on the reasoning tasks of a single user or a few users, and does not deeply explore the batch task optimization problem in the multi-user collaborative reasoning scenario. It is impossible to deal with the situation in which multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time in actual applications, and how to optimize the user reasoning task delay through resource allocation and reasonable use of performance.

[0154] Next, a multi-user collaborative reasoning delay optimization device based on batch processing technology proposed in an embodiment of the present application is described with reference to the accompanying drawings.

[0155] Figure 8 It is a structural diagram of a multi-user collaborative reasoning delay optimization device based on batch processing technology in an embodiment of the present application.

[0156] like Figure 8 As shown, the multi-user collaborative reasoning delay optimization device 10 based on batch processing technology includes: a segmentation module 100, a division module 200 and a generation module 300.

[0157] The segmentation module 100 is used to obtain reasoning tasks of multiple users and segment the reasoning tasks into multiple subtasks.

[0158] The partitioning module 200 is used to traverse the partitioning points of multiple subtasks through a target iterative traversal algorithm to find the optimal task partitioning points of the multiple subtasks, and to partition the multiple subtasks into server reasoning subtasks and local reasoning subtasks based on the optimal task partitioning points.

[0159] The generation module 300 is used to calculate the minimum total completion time of the server reasoning subtask and the local reasoning subtask based on the actual bandwidth and energy constraints, and generate a delay optimization strategy for multiple user collaborative reasoning tasks according to the minimum total completion time.

[0160] Optionally, in one embodiment of the present application, the division module 200 includes: a first division unit and a second division unit.

[0161] The first division unit is used to divide the subtasks before the optimal task division point to the target server for processing to obtain the server reasoning subtasks;

[0162] The second division unit is used to divide the subtasks after the optimal task division point into local processing to obtain local reasoning subtasks.

[0163] Optionally, in one embodiment of the present application, it also includes: an acquisition module and a traversal module.

[0164] Among them, the acquisition module is used to obtain the server batch reasoning task when the server reasoning subtask reaches the preset batch processing condition.

[0165] The traversal module is used to identify the target batch in the server batch reasoning task through the target maximum batch algorithm, and traverse the server batch reasoning tasks after the target batch through the target iterative traversal algorithm to obtain the minimum total completion time of the server batch reasoning task, and generate a delay optimization strategy for multiple user collaborative reasoning tasks based on the minimum total completion time.

[0166] Optionally, in one embodiment of the present application, the actual bandwidth and energy constraints include: the total energy consumption of any one of multiple users is constrained to be within a preset energy consumption range; the local computing frequency of any one user is constrained to be between the current minimum frequency and the current maximum frequency; the processors of multiple subtasks are constrained to be local reasoning subtasks; and the initial task start time of the local reasoning subtask is constrained to be 0.

[0167] Optionally, in one embodiment of the present application, the delay optimization strategy for multiple user collaborative reasoning tasks includes at least one of optimal batch start time, allocation decision, local computing frequency and total completion time.

[0168] It should be noted that the aforementioned explanation of the embodiment of the multi-user collaborative reasoning delay optimization method based on batch processing technology is also applicable to the multi-user collaborative reasoning delay optimization device based on batch processing technology in this embodiment, and will not be repeated here.

[0169] According to the multi-user collaborative reasoning delay optimization device based on batch processing technology proposed in the embodiment of the present application, the multiple subtasks into which the reasoning task is divided can be divided into server reasoning subtasks and local reasoning subtasks based on the target iterative traversal algorithm, and the delay optimization strategy of multiple user collaborative reasoning tasks is generated according to the minimum total completion time of the task. Thus, a method based on task segmentation and scheduling algorithm is realized, and a traversal algorithm and a maximum batch algorithm are proposed, which significantly reduces the task processing delay. Considering the actual server delay, in scenarios where the server bandwidth is limited or the user device capabilities differ significantly, the system performance of multi-user collaborative reasoning under different conditions and in different scenarios can be effectively improved, and the energy consumption of user devices can be reduced. It has high applicability and robustness, and provides strong support for the application of edge computing in the fields of the Internet of Things and artificial intelligence. Thus, it solves the problem that the related technology mainly focuses on the reasoning tasks of a single user or a few users, and does not deeply explore the batch task optimization problem in the multi-user collaborative reasoning scenario. It is impossible to deal with the situation in which multi-user collaborative reasoning may involve multiple users performing reasoning tasks at the same time in actual applications, and how to optimize the user reasoning task delay through resource allocation and reasonable use of performance.

[0170] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0171] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0172] When the processor 902 executes the program, the multi-user collaborative reasoning delay optimization method based on batch processing technology provided in the above embodiment is implemented.

[0173] Furthermore, the electronic device further comprises:

[0174] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0175] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0176] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0177] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0178] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0179] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0180] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned multi-user collaborative reasoning delay optimization method based on batch processing technology.

[0181] An embodiment of the present application also provides a computer program product, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the multi-user collaborative reasoning delay optimization method based on batch processing technology provided in the embodiment of the present application is implemented.

[0182] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0183] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0184] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0185] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0186] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0187] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0189] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A multi-user collaborative reasoning delay optimization method based on batch processing technology, It is characterized in that The following steps are involved: Acquire reasoning tasks of multiple users, and divide the reasoning tasks into multiple subtasks; Traversing the splitting points of the plurality of subtasks through a target iterative traversal algorithm to find an optimal task splitting point of the plurality of subtasks, and dividing the plurality of subtasks into a server reasoning subtask and a local reasoning subtask based on the optimal task splitting point; Based on actual bandwidth and energy constraints, the minimum total completion time of the server reasoning subtask and the local reasoning subtask is calculated, and a delay optimization strategy for the multiple user collaborative reasoning tasks is generated according to the minimum total completion time.

2. The method according to claim 1, It is characterized in that The step of traversing the split points of the plurality of subtasks through a target iterative traversal algorithm to find an optimal task split point of the plurality of subtasks, and dividing the plurality of subtasks into a server reasoning subtask and a local reasoning subtask based on the optimal task split point comprises: Assigning the subtasks before the optimal task splitting point to the target server for processing to obtain the server reasoning subtask; The subtasks after the optimal task splitting point are divided into local parts for processing to obtain the local reasoning subtasks.

3. The method according to claim 1, It is characterized in that Also includes: When the server reasoning subtask reaches a preset batch processing condition, obtaining a server batch reasoning task; A target batch in a server batch reasoning task is identified by a target maximum batch algorithm, and the server batch reasoning tasks after the target batch are traversed by the target iterative traversal algorithm to obtain a minimum total completion time of the server batch reasoning task, and a delay optimization strategy for the multiple user collaborative reasoning tasks is generated according to the minimum total completion time.

4. The method according to claim 1, It is characterized in that The practical bandwidth and energy constraints include: The total energy consumption of any user among the multiple users is constrained to be within a preset energy consumption range; The local computing frequency of any user is constrained to be between the current minimum frequency and the current maximum frequency; The processor constraints of the plurality of subtasks are local reasoning subtasks; The initial task start time constraint of the local reasoning subtask is 0.

5. The method according to claim 1, It is characterized in that The delay optimization strategy for the multiple user collaborative reasoning tasks includes at least one of optimal batch start time, allocation decision, local calculation frequency and total completion time.

6. A multi-user collaborative reasoning delay optimization device based on batch processing technology, It is characterized in that include: A segmentation module, used to obtain reasoning tasks of multiple users and segment the reasoning tasks into multiple subtasks; A partitioning module, configured to traverse the partitioning points of the plurality of subtasks through a target iterative traversal algorithm to find an optimal task partitioning point of the plurality of subtasks, and to partition the plurality of subtasks into a server reasoning subtask and a local reasoning subtask based on the optimal task partitioning point; A generation module is used to calculate the minimum total completion time of the server reasoning subtask and the local reasoning subtask based on actual bandwidth and energy constraints, and generate a delay optimization strategy for the multiple user collaborative reasoning tasks according to the minimum total completion time.

7. The device according to claim 6, It is characterized in that The division module comprises: A first division unit, configured to divide the subtasks before the optimal task division point to a target server for processing, so as to obtain the server reasoning subtask; The second division unit is used to divide the subtasks after the optimal task division point into local processing to obtain the local reasoning subtasks.

8. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-user collaborative reasoning delay optimization method based on batch processing technology as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that The program is executed by a processor to implement the multi-user collaborative reasoning delay optimization method based on batch processing technology as described in any one of claims 1-5.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed, it is used to implement the multi-user collaborative reasoning delay optimization method based on batch processing technology as described in any one of claims 1-5.

Images