A multi-objective DNN inference task deployment method and system oriented to operation reliability and latency

Abstract

The application provides a multi-target DNN inference task deployment method and system facing operation reliability and time delay, and the method comprises the following steps: receiving a DNN inference task proposed by a mobile device; acquiring a physical network topology graph containing a central cloud server, an edge server and the mobile device; acquiring a preset population quantity and a maximum iteration number for multi-target DNN inference task deployment optimization; finding an optimized DNN inference task deployment by using a multi-target genetic model, generating population individuals according to the population quantity, performing multi-time iteration optimization on all population individuals, and outputting the found optimized DNN inference task deployment result after the iteration reaches the maximum iteration number. Wherein, the physical network topology graph and the DNN inference task are taken as inputs of the multi-target genetic model, the multi-target optimization direction of the multi-target genetic model contains high operation reliability and low delay, and the constraint condition of the multi-target genetic model contains inference total time delay constraint.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of DNN inference task deployment, and particularly relates to a multi-target DNN inference task deployment method and system for operation reliability and time delay. BACKGROUND

[0002] Thanks to the rapid development of Internet of Things (IoT) and machine learning, a large number of intelligent mobile applications have been developed, such as VR, monitoring, etc. Deep learning technology, as the core technology of artificial intelligence, especially deep neural networks (DNN), has been widely applied in various fields. The requirements of emerging intelligent mobile applications for communication systems mainly include four aspects: low response time, low energy consumption, low monetary cost and uniform distribution of computing resources. However, some intelligent mobile applications require strong computing power, such as face recognition. Considering the limited resources of mobile devices, it is difficult to process intensive deep learning applications on local mobile devices. The main processing method at present is to perform DNN inference to meet the real-time requirements of intelligent applications.

[0003] Among them, intelligent mobile applications are application programs developed by using local mobile devices and intelligent technologies. These applications aim to provide user-friendly, convenient and efficient services to meet the various needs of users on mobile devices. Intensive deep learning applications mainly refer to intelligent application programs that involve a large amount of data reading and writing and network request response requirements.

[0004] DNN inference refers to the process of obtaining a calculation result by performing a calculation task tightly coupled with a DNN model. The implementation of DNN inference needs to go through two stages of model training and online inference.

[0005] (1) Model training stage: the DNN model needs to be trained iteratively to complete weight update and is stored in the cloud server as a callable service function model.

[0006] (2) Online inference stage: the trained DNN model is deployed centrally or distributively on terminals, edge nodes or cloud servers, and performs DNN inference tasks to provide users with an intelligent service with real-time inference and rapid decision-making capabilities.

[0007] For the online inference phase, the traditional method is to directly offload part of the DNN model to the remote cloud server (or can be called the central cloud server) to reduce the pressure of the local device, that is, to transfer the inference task of the Internet of Things device to the remote cloud server for execution, but this method will cause heavy load and network congestion, thereby affecting the execution of DNN inference tasks sensitive to time delay. With the development of mobile edge computing technology, it brings new paradigms to the deployment of DNN inference tasks. In the edge computing environment, the typical deployment mode of DNN inference tasks mainly includes the following four modes: 1) edge node deployment mode; 2) edge-end collaborative deployment mode; 3) cloud-edge-end collaborative deployment mode; 4) terminal collaborative deployment mode.

[0008] Document scheme 1: Cost-Driven Off-Loading for DNN-Based Applications Over Cloud, Edge, and End Devices The author studies the distributed deployment of DNN inference tasks in cloud servers, edge nodes and devices, and proposes an adaptive discrete particle swarm optimization algorithm to optimize resource utilization and reduce system cost consumption under the premise of ensuring inference delay constraints.

[0009] Document scheme 2: Energy-aware inference offloading for DNN-driven applications in mobile edge clouds The author studies the DNN inference task offloading in MEC network, assuming that each requested DNN inference task has been partitioned. Then they provide a random algorithm and an online algorithm to minimize the total energy consumption and process real-time applications, respectively.

[0010] Document scheme 3: Throughput Maximization of Delay-Aware DNN Inference in Edge Computing by Exploring DNN Model Partitioning and Inference Parallelism The author divides the DNN inference task into two parts, one part is executed in the local Internet of Things device, and the other part is executed in the cloudlet in the MEC network, and multiple threads are allocated to realize the accelerated inference of the DNN inference task. Under the premise of meeting the inference delay of the DNN inference task, the throughput of the DNN inference task is maximized.

[0011] The literature scheme 1 and the literature scheme 2 focus on the optimization of resources and the cost overhead. The literature scheme 3 mainly focuses on maximizing the throughput of the DNN inference task. It can be seen that the current research schemes are mostly concentrated on the optimization of system cost or the efficiency of DNN inference.

[0012] The above three schemes do not consider the influence of reliability on the execution of the DNN inference task, and the reliability is more and more valued in the current network environment. When the execution of the DNN inference task results in inaccurate calculation due to server failure or other unreliable conditions, it may cause waste of computing resources, or even safety problems. In addition, the above research only considers one main optimization target to calculate the optimal solution. With the increasing complexity of network structure and business, a single optimization target is more and more difficult to adapt to the flexible and variable network environment. SUMMARY

[0013] In view of this, the embodiments of the present application provide a multi-objective DNN inference task deployment method and system oriented to operation reliability and latency to eliminate or improve one or more defects in the prior art.

[0014] One aspect of the present application provides a multi-objective DNN inference task deployment method oriented to operation reliability and latency, the method comprising the following steps: receiving a DNN inference task proposed by a mobile device; obtaining a physical network topology graph comprising a center cloud server, an edge server and a mobile device; obtaining a preset population size and a maximum iteration number for multi-objective DNN inference task deployment optimization; using a multi-objective genetic model to find an optimized DNN inference task deployment, generating population individuals according to the population size, and performing multiple iteration optimization on all population individuals, and outputting the found optimized DNN inference task deployment result after iteration reaches the maximum iteration number; wherein the physical network topology graph and the DNN inference task are input to the multi-objective genetic model, the multi-objective optimization direction of the multi-objective genetic model comprises high operation reliability and low latency, and the constraint condition of the multi-objective genetic model comprises inference total latency constraint.

[0015] In some embodiments of the present application, the data structure of the physical network topology graph comprises the total number of the center cloud server, the edge server and the mobile device, and the link connected between the center cloud server, the edge server and the mobile device.

[0016] In some embodiments of the present application, the multi-objective genetic model searches for an optimized DNN inference task deployment according to a multi-objective genetic algorithm. In each round of iteration, the method comprises: randomly generating a parent population, obtaining a child population by performing genetic operations on the parent population; merging the child population and the parent population; calculating the fitness values in each target optimization direction according to the objective function, and calculating the fast non-dominated sorting of the merged population according to the fitness values; calculating the crowding degree of each individual in the merged population in each target optimization direction; selecting a better population as the next generation parent population according to the calculated fast non-dominated sorting and crowding degree; checking whether the maximum number of iterations is reached, and if so, exiting the iteration process, and if not, continuing the next round of iteration.

[0017] In some embodiments of the present application, the high operation reliability includes service reliability and inference accuracy reliability, and the low latency is targeted at a function of minimizing the total latency of the DNN inference task as an objective function.

[0018] In some embodiments of the present application, the DNN inference task comprises a plurality of DNN inference subtasks, and the DNN inference task deployment result is a scheme of offloading the DNN inference subtasks to different servers; the DNN inference task is accompanied by a maximum inference total latency, which is used as the inference total latency constraint; the inference total latency includes the processing delay of the DNN inference subtasks on the server and the data transmission delay of the data transmission between the DNN inference subtasks on the link.

[0019] In some embodiments of the present application, the method uses the number of CPU cores of the server to represent the computing resources, and the constraint condition of the multi-objective genetic model further includes a server computing resource constraint. Each DNN inference subtask offloaded to the server needs to consume the computing resources on the server. The server computing resource constraint means that the total computing resources required by the DNN inference subtasks deployed on each server cannot exceed the maximum computing resources that the server can provide.

[0020] In some embodiments of the present application, the constraint condition of the multi-objective genetic model further includes a deployment constraint. The deployment constraint means that each DNN inference subtask needs to be deployed on a server, and each DNN inference subtask can only be deployed on one server for execution, and the server can only serve the mobile device, edge server or central cloud server that initiates the inference task.

[0021] Another aspect of the present application provides a multi-objective DNN inference task deployment system oriented to operation reliability and latency, comprising a processor, a memory and computer programs / instructions stored on the memory, the processor being configured to execute the computer programs / instructions, and the system implementing the steps of the method according to any one of the above embodiments when the computer programs / instructions are executed.

[0022] Another aspect of the present application provides a computer readable storage medium having computer programs / instructions stored thereon, the computer programs / instructions being executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0023] Another aspect of the present application provides a computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0024] The multi-objective DNN inference task deployment method and system oriented to operation reliability and latency provided by the present application can be used for DNN inference tasks proposed by mobile devices, and can be used for multi-objective DNN inference task deployment optimization in a physical network topology environment not cultivated by the mobile devices, wherein a multi-objective genetic model is used for multi-objective DNN inference task deployment optimization, the multi-objective genetic model taking high operation reliability and low latency as the multi-objective optimization direction, and taking inference total latency constraint as a constraint condition, so as to comprehensively consider the influence of operation reliability and latency, and find an optimal DNN inference task deployment result, i.e., a Pareto optimal solution.

[0025] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following figures and detailed description thereof or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0026] It will be understood by those skilled in the art that the objects and advantages of the present application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0028] Figure 1 A flow chart of a multi-objective DNN inference task deployment method oriented to operation reliability and latency in an embodiment of the present application.

[0029] Figure 2 A schematic diagram of a physical network.

[0030] Figure 3 A multi-objective optimization direction graph for an embodiment of the present application.

[0031] Figure 4 A multi-objective DNN inference task deployment method flowchart for an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed description will be given to the present application in combination with embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but not as a limitation to the present application.

[0033] It should be noted that, in order to avoid the present application being obscured by unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0034] It should be emphasized that the term “comprises / comprising” is used herein to indicate the presence of a feature, element, step or component, but not to exclude the presence or addition of one or more other features, elements, steps or components.

[0035] It should be noted that, if not specifically stated, the term “connected” herein can not only mean direct connection, but also indirect connection with an intermediate.

[0036] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0037] The problems of the existing DNN inference task deployment method include: (1) the existing DNN inference task deployment does not consider the influence of reliability on DNN inference task execution; (2) the existing DNN inference task deployment only considers one main optimization target, but a single optimization target is increasingly difficult to adapt to flexible network environment.

[0038] The purpose of the present application is to propose a multi-objective DNN inference task deployment method for operation reliability and time delay in a cloud edge network scenario, which can obtain a Pareto solution set of DNN inference task deployment within an effective time, and the method is not limited to a specific network structure and has good scalability. The present application mainly studies the deployment method of DNN inference task in the cloud edge network scenario, and the main purpose is to obtain the Pareto front of the DNN inference task deployment strategy for multi-objective optimization of operation reliability and time delay, so as to quickly select the most suitable deployment strategy in a flexible and variable network scenario. Nonlinear programming is used to establish the constraint condition, and the objective function is established for the optimization direction of each target, and a multi-objective optimization algorithm is determined based on the genetic evolution strategy. In a limited time, a better Pareto solution set (Pareto solution set) is obtained, and the solutions in the Pareto solution set are ensured to meet the various constraints in the DNN inference task deployment. Quickly obtaining the Pareto front (or Pareto solution set) of the DNN inference task deployment strategy to realize multi-objective optimization is the main problem to be solved in the present application.

[0039] Figure 1 For the multi-objective DNN inference task deployment method for operation reliability and time delay in an embodiment of the present application, the method comprises the following steps:

[0040] Step S110: receiving a DNN inference task proposed by a mobile device.

[0041] Step S120: obtaining a physical network topology graph containing a central cloud server, an edge server and a mobile device.

[0042] Step S130: obtaining a preset population size and maximum iteration number for multi-objective DNN inference task deployment optimization.

[0043] In the specific implementation process, the population size and the maximum iteration number can be a fixed value, or can be a parameter artificially adjusted according to different situations.

[0044] Step S140: using a multi-objective genetic model to find an optimized DNN inference task deployment, generating population individuals according to the population size, and performing multiple iteration optimizations on all population individuals, and outputting the found optimized DNN inference task deployment result after the iteration reaches the maximum iteration number; wherein the physical network topology graph and the DNN inference task are used as inputs of the multi-objective genetic model, the multi-objective optimization direction of the multi-objective genetic model includes high operation reliability and low delay, and the constraint condition of the multi-objective genetic model includes inference total time delay constraint.

[0045] In the implementation process, the execution subject of the method can be a mobile device, and after step S140, the mobile device deploys DNN inference sub-tasks included in the DNN inference task to different server sides according to the DNN inference task deployment result, in which the server sides can include a mobile device, an edge server (ES) and a central server, or only include the edge server and the central server.

[0046] The multi-objective DNN inference task deployment method for operation reliability and time delay can deploy the DNN inference task proposed by the mobile device in the physical network topology environment not cultivated by the mobile device, and can use the multi-objective genetic model to perform multi-objective DNN inference task deployment optimization, in which high operation reliability and low delay are used as the multi-objective optimization direction, and the inference total time delay constraint is used as the constraint condition, so as to comprehensively consider the influence of operation reliability and time delay, and find the optimal DNN inference task deployment result, i.e., the Pareto optimal solution.

[0047] Figure 2 For the schematic diagram of the physical network, the local device is connected to the edge server, the edge server is connected to the central server, and the topology relationship shown in the figure is presented. Figure 2

[0048] In some embodiments of the present application, the data structure of the physical network topology graph includes the total number of central cloud servers, edge servers and mobile devices, and the links connected between the central cloud servers, the edge servers and the mobile devices.

[0049] By using the embodiments of the present application, the DNN inference task deployment can be performed within the range of the physical network topology graph, and the network range for performing the DNN inference task deployment is effectively limited.

[0050] In some embodiments of the present application, the multi-objective genetic model finds the optimized DNN inference task deployment according to the multi-objective genetic algorithm, and in each round of iterative optimization process, the method includes:

[0051] (1) randomly generating a parent population, and obtaining a child population by performing genetic operations on the parent population, in which the genetic operations include selection, crossover and mutation operations, etc.

[0052] (2) merging the child population and the parent population;

[0053] (3) calculating the fitness values in the target optimization direction according to the objective function, and calculating the fast non-dominated sorting of the merged population according to the fitness values;

[0054] (4) calculating the crowding degree of the individuals in the merged population in each target optimization direction.​

[0055] (5) according to the calculated fast non-dominated sorting and crowding degree, select a better population as the next generation of parent population;

[0056] (6) check whether the maximum number of iterations is reached, if reached, exit the iteration process, if not reached, continue the next round of iteration.

[0057] By adopting the embodiments of the application, the multi-objective genetic model optimization can be performed according to the specific NSGA-II algorithm.

[0058] In some embodiments of the application, the high operation reliability includes service reliability and reasoning accuracy reliability, and the low latency is taken as a target function in the function of minimizing the total delay of the DNN reasoning task. Figure 3 The multi-objective optimization direction graph is used in an embodiment of the application.

[0059] By adopting the embodiments of the application, the target function can be further enriched, so that the final multi-objective DNN reasoning task deployment result has better low latency and high reliability characteristics.

[0060] In some embodiments of the application, the DNN reasoning task includes a plurality of DNN reasoning subtasks, and the DNN reasoning task deployment result is a scheme of offloading the DNN reasoning subtasks to different servers; the DNN reasoning task is accompanied by a maximum reasoning total delay, which is taken as the reasoning total delay constraint; the reasoning total delay includes the processing delay of the DNN reasoning subtasks on the server and the data transmission delay of the data transmission between the DNN reasoning subtasks on the link.

[0061] In some embodiments of the application, the method uses the number of CPU cores of the server to represent the computing resources, and the constraint condition of the multi-objective genetic model further includes a server computing resource constraint; each DNN reasoning subtask offloaded to the server needs to consume the computing resources on the server; the server computing resource constraint means that the integrated computing resources required by the DNN reasoning subtasks deployed on each server cannot exceed the maximum computing resources that the server can provide.

[0062] By adopting the above two embodiments of the application, the server computing resources and the total delay of the DNN reasoning task deployment can be constrained, so as to guarantee better DNN reasoning task execution efficiency.

[0063] In yet some embodiments of the present application, the constraint conditions of the multi-objective genetic model further include deployment constraints, which means that each DNN inference sub-task needs to be deployed on a server, and each DNN inference sub-task can only be deployed on one server for execution, and the server can only serve the mobile device, edge server or central cloud server that initiates the inference task.

[0064] In the induction process of the method proposed in the present application, there are four steps: 1. parameter description; 2. constraint condition summary; 3. objective function summary; and 4. DNN inference task deployment method calculation. Next, each step will be analyzed one by one.

[0065] 1. Parameter Description

[0066] In the present application, the cloud-edge-end network topology environment is represented by a graph G=(N,L). Where N represents the mobile device, edge server and central cloud server mentioned above, i.e. N={n mobile ∪n edge ∪n cloud}, |N| represents the total number of servers, where each server is represented by n a . L represents the link between the above-mentioned devices. For each link l∈L, there is a server node pair (n1,n2), where n1,n2∈N and n1≠n2. The undirected link between server nodes n1,n2 is represented by E , and the maximum bandwidth of link l is represented by B l . In the present application, it is considered that the bandwidth resource of the link is always sufficient, i.e. B l is the transmission rate.

[0067] Each server has a certain amount of computing resources. For computing resources, the present scheme considers that for each DNN inference task, the server can provide stable computing efficiency services to it as long as the computing resources required by the DNN inference task do not exceed the remaining computing resources on the server. Since DNN inference tasks are different from DNN training tasks, they often require less CPU and GPU resources, so it is more inclined to use CPU to process DNN inference services than to use expensive hardware accelerators for DNN inference. Therefore, in this paper, we use the number of CPU cores to represent the computing resources, i.e. represents the number of CPU cores on server n. In addition to computing resources, we also consider the service reliability R n and inference accuracy reliability A n of each server, and assume that server n processes DNN inference tasks at a certain processing rate η n .

[0068] where the DNN inference task is initiated by mobile device nmobile is generated, with a reasoning delay requirement T SLA , which represents the maximum acceptable total latency of reasoning. The DNN reasoning task can be modeled as a directed acyclic graph G' = (V, E). Where V represents the set of sub-tasks of the DNN reasoning task, |V| represents the total number of sub-tasks. E represents the set of data dependency relationships between sub-tasks. And for each v i ∈V,i∈|V|, the offloading of each sub-task on the server needs to occupy the computing resources of the server, using to represent. At the same time, F i is used to represent the computing workload required by sub-task i of the DNN reasoning task. For data dependencies, for each dependency relationship e∈E, there is a pair of sub-task nodes, (v i ,v j ) where v i ,v j ∈V and v i ! = v j . Since certain data transmission is required between different sub-tasks, F' is used to represent the data transmission volume on dependency e. e

[0069] Under the actual network topology, the relationship between the DNN reasoning task sub-task and the server can be represented by , if , then it means that the DNN reasoning task sub-task i is deployed on server a, otherwise , and after the DNN reasoning task is deployed in the actual network topology, F is used to represent whether the DNN reasoning task dependency e passes through the link l, if , then it means that it passes through the link l.

[0070] II. Constraint summary

[0071] Since the deployment problem of the DNN reasoning task is an NP-hard problem, the main solution idea is to find a better feasible solution that satisfies the constraints in the optimization target direction. In the solving process, the provided deployment strategy needs to be judged to determine whether the strategy satisfies the relevant constraint conditions:

[0072] (1) Server computing resource constraint: The offloading of each sub-task of the DNN reasoning task needs to consume the computing resources on the server to execute the task. The comprehensive required computing resources of the tasks deployed on each server must not exceed the maximum computing resources that the server can provide, that is, formula 1.

[0073]

[0074] ​(2) Deployment constraints: for the deployment of each DNN inference task, let the initiator node of the DNN inference task be denoted as n initiate In order to implement the DNN inference task, each subtask of the DNN inference task needs to be deployed on a server for implementation, and each subtask can only be deployed on one server for execution, and the server can only serve the mobile device, edge server, central cloud server that initiates the inference task, that is, formula 2.

[0075]

[0076] (3) Inference total delay constraint: the total delay of the DNN inference task is usually composed of: 1. The processing delay of the DNN subtask on the server; 2. The data transmission delay of the dependent data transmission on the link. Use η n to represent the processing rate of the server n. Use to represent the processing delay of the DNN inference task subtask i. Then:

[0077]

[0078] Then the total processing delay of the inference task is:

[0079]

[0080] Use F′ e to represent the amount of data that needs to be transmitted in the dependency relationship e, and the transmission delay of the dependency relationship e is where The value of is determined in the present application using Dijkstra algorithm:

[0081]

[0082] Then the total transmission delay of the inference task is:

[0083]

[0084] From the above formula, the total execution delay of the DNN inference task after offloading is:

[0085] T=t proc +t tran ; (7)

[0086] In order to meet the QoS of the DNN inference request, the total delay needs to be less than the delay requirement of the DNN inference task, where QoS (Quality of Service) refers to a capability provided by the network when transmitting data packets, that is:

[0087] T≤T SLA ; (8)

[0088] III. Objective function summary

[0089] The present application attempts to obtain a Pareto solution set for the deployment of DNN inference tasks through a multi-objective optimization method, in order to obtain more accurate and adaptable solutions for more demand scenarios. The Pareto solution set refers to a set of solutions that cannot be improved in a multi-objective optimization problem, and these solutions are called Pareto optimal solutions or non-dominated solutions. In order to determine such a set of Pareto solutions, the fitness of each solution is calculated by the value of the objective function. The multi-objective optimization direction determined by the present application is high operational reliability and low latency, where high operational reliability is divided into service reliability and inference accuracy reliability.

[0090] (1) Service reliability: Due to the universality of system failures in hardware and software, 100% reliability of the server is not easy to achieve. For example: when encountering high-priority tasks, it may interrupt the processing of low-priority inference tasks, resulting in timeout, or some ESs do not transmit inference results due to failure. The above situations all indicate that mobile devices, edge servers or central cloud servers may be unreliable. In this paper, R n represents the operational reliability of the server, in particular, the operational reliability of the central cloud server is the highest, followed by the edge server, and finally the mobile device. Although a dynamic variable of operational reliability, in order to simplify non-critical issues, we believe that the operational reliability of the server is constant during the deployment of DNN inference tasks. Therefore, the operational reliability of the DNN inference request is represented as:

[0091]

[0092] The objective function for maximizing service reliability is:

[0093]

[0094] (2) Inference accuracy reliability: Inference accuracy reliability measures the consistency and accuracy of DNN model inference results under different input conditions. If the inference results of the model are unstable or unreliable, it may lead to incorrect system decisions or outputs, thereby affecting the performance and reliability of the entire application. Therefore, accuracy reliability is also one of the important indicators for measuring the reliability of DNN inference task deployment. It is not difficult to find that if the server is not reliable, the accuracy is also not reliable, therefore, the accuracy reliability of the DNN inference request after deployment is represented as:

[0095]

[0096] The objective function for maximizing inference accuracy reliability is:

[0097]

[0098] (3) DNN inference total latency: the objective function of the total latency is the same as the calculation mentioned in the constraint condition, that is, formula 7.

[0099] Then the objective function of minimizing the DNN inference total latency is:

[0100] min T = min t proc + t tran ; (13)

[0101] Four, DNN inference task deployment method calculation.

[0102] The multi-objective DNN inference task deployment method for operation reliability and latency provided by the application is mainly aimed at the cloud edge network scene, when the mobile device proposes the DNN inference task demand, each subtask is deployed on the server to realize online inference execution and provide intelligent service. The algorithm adopts the genetic evolution idea, is improved and adapted based on the NSGA-II algorithm, and finds the Pareto solution set of the DNN inference task deployment strategy.

[0103] In an embodiment of the application, the NSGA-II algorithm is used for calculation of the multi-objective DNN inference task deployment result, and the NSGA-II algorithm is a multi-objective genetic algorithm. The multi-objective DNN inference task deployment method for operation reliability and latency, Figure 4 The flow chart of the multi-objective DNN inference task deployment method for operation reliability and latency in the embodiment of the application is as follows:

[0104] Input: physical network topology G=(N, L), a DNN inference task G=(V, E).

[0105] Output: a set of Pareto solutions of DNN inference task deployment and the accurate reliability A m , operation reliability R m , and latency T m corresponding to each solution set.

[0106] Initial value: population size: popSize=20; iteration number: gen=1000, len and i are two temporary variables, and the content after % is a pseudo code comment.

[0107]

[0108]

[0109] The input of the method is: actual mobile device, edge server, physical network structure of central cloud server, and single DNN inference task to be deployed. The output is a set of Pareto solution set of deployment of the DNN inference task, and the fitness value of each optimization objective corresponding to each solution. The initial condition is the population size popSize of each generation and the total number of iterations gen.

[0110] Steps 1-7 are the process of generating an initial population, which is limited by the resource constraint condition, i.e., the generation of solutions is limited by the deployment constraint condition, and when a solution that meets the constraint condition is generated, it is put into the population.

[0111] Steps 8-18 are the operations of crossover, mutation, selection, etc. of the genetic algorithm for the initial population and iteration.

[0112] Step 10 is to obtain a sub-population by crossover and mutation operation, the main idea of which is to select parent populations by binary selection method for crossover and mutation operation, and two offspring are generated each time until the population size of the sub-population is also popSize.

[0113] Step 11 combines the parent and child populations, and there are 2*popSize solution vectors in total.

[0114] Step 12 calculates the fitness value of the total population, which is the function value of the corresponding objective function.

[0115] Step 13 calculates the non-dominated sorting of the total population, i.e., the Pareto level of the population individuals.

[0116] Step 14 calculates the crowding degree value of the individuals in the population on each objective, and the crowding degree value of the population is the sum of the crowding degree values of each objective.

[0117] Step 15 selects the next generation parent population from the population, the selection method is to traverse the non-dominated sorting by level, if the population set of the Pareto level is accumulated to not more than the upper limit of the next generation population, it is selected into the next generation population; if it is accumulated to more than, the population is selected according to the crowding degree sorting in the Pareto level population and added to the next generation population.

[0118] Step 16 means that when the number of iterations reaches the maximum number of iterations, the solution set of the parent population at this time is the Pareto solution set of the deployment of the DNN inference task proposed in this paper, and the value set is the function value of each optimization objective function.

[0119] By using the NSGA-II algorithm, the multi-objective DNN inference task deployment proposed in this application can be realized.

[0120] The multi-objective DNN inference task deployment method and system for operation reliability and time delay provided by the application can deploy and optimize the multi-objective DNN inference task of a mobile device in a physical network topology environment that is not cultivated by the mobile device, and the multi-objective genetic model is used for multi-objective DNN inference task deployment optimization, the multi-objective genetic model takes high operation reliability and low delay as the multi-objective optimization direction, and takes the inference total time delay constraint as the constraint condition, so as to comprehensively consider the influence of operation reliability and time delay, and find the optimal DNN inference task deployment result, that is, the Pareto optimal solution.

[0121] Specifically, the method provided by the application has the following advantages and beneficial effects:

[0122] (1) The method combines edge computing and cloud computing scenarios, considers the DNN inference task deployment decision problem in a cloud edge end network scenario with three types of servers, that is, mobile devices, edge servers and cloud servers, and establishes a multi-objective DNN inference task deployment problem model for operation reliability and time delay. Unlike some existing researches, the application does not simply divide the DNN inference task into local and cloud two parts for execution, but freely deploys the subtasks of the DNN inference task on each server while setting certain constraint conditions, so as to better model the structural characteristics of the actual network scenario.

[0123] (2) In view of the deficiency that current researches do not consider reliability, the application considers the influence of reliability, and considers the deployment strategy of the DNN inference task as one of the optimization targets. By taking reliability as an optimization target, the deficiency that current researches do not consider or only take reliability as a constraint condition is solved. The application considers reliability as an optimization target to better realize the deployment of the DNN inference task with high reliability in the actual scenario.

[0124] (3) In view of the deficiency of single optimization target in current researches, the application considers multi-objective optimization to adapt to flexible and variable network scenarios. When making a deployment strategy decision, the multi-objective optimization method is used to make a decision, which can solve the problem of performance decline of a single optimization target in a flexible and variable network scenario. The Pareto solution set obtained by the multi-objective optimization of the application can better adapt to complex and variable network environments and business demands.

[0125] (4) The optimization of the deployment strategy is converted into a mathematical model, and a multi-objective DNN inference task deployment method for operation reliability and time delay is provided.

[0126] Corresponding to the above method, the application further provides a multi-objective DNN inference task deployment system facing operation reliability and time delay, which comprises a computer device, the computer device comprises a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory, and the system realizes the steps of the method as described above when the computer instructions are executed by the processor.

[0127] The embodiment of the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the foregoing edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0128] The embodiment of the application further provides a computer program product, which comprises a computer program / instruction, and the computer program / instruction is executed by a processor to realize the steps of the foregoing method. Moreover, the computer program product can run on a mobile device.

[0129] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination thereof. The choice of hardware or software implementation is dependent on the particular application and design constraints imposed on the solution. Skilled persons can use different methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of the application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine readable medium or transmitted through a data signal carried in a carrier wave in a transmission medium or communication link.

[0130] It should be clear that the application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted herein. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the application.

[0131] Features described and / or illustrated with respect to one implementation can be used in the same manner or in a similar manner in one or more other implementations and / or in combination with or in place of features of other implementations.

[0132] The preferred embodiments of the present application are described above with reference to the flowchart and / or block diagram of the method, apparatus, system, uses and / or computer program according to this application. It will be understood that each block of the flowchart and / or block diagram, and / or combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. Such computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A multi-objective DNN inference task deployment method oriented to operation reliability and latency, characterized in that, The method comprises the following steps: receiving a DNN inference task proposed by a mobile device; obtaining a physical network topology graph comprising a central cloud server, an edge server and the mobile device; obtaining a preset population size and a maximum iteration number for multi-objective DNN inference task deployment optimization; finding an optimized DNN inference task deployment using a multi-objective genetic model, generating a population individual according to the population size, performing multiple iteration optimization on all population individuals, and outputting the found optimized DNN inference task deployment result after iteration reaches the maximum iteration number; wherein the physical network topology graph and the DNN inference task are input to the multi-objective genetic model, the multi-objective optimization direction of the multi-objective genetic model comprises high operation reliability and low latency, and the constraint condition of the multi-objective genetic model comprises inference total time delay constraint; deploying DNN inference sub-tasks contained in the DNN inference task to different server sides according to the DNN inference task deployment result, wherein the server sides comprise the mobile device, the edge server and the central cloud server, or the server sides only comprise the edge server and the central cloud server; The multi-objective genetic model finds an optimized DNN inference task deployment according to a multi-objective genetic algorithm. In each round of iteration optimization, the method comprises: randomly generating a parent population, obtaining a child population by performing genetic operations on the parent population; merging the child population and the parent population; calculating the fitness value in each target optimization direction according to the objective function, and calculating the fast non-dominated sorting of the merged population according to the fitness value; calculating the crowding degree of individuals in the merged population in each target optimization direction; selecting a better population as the next generation parent population according to the calculated fast non-dominated sorting and crowding degree; checking whether the maximum iteration number is reached, and if so, exiting the iteration process, and if not, continuing the next round of iteration; the high operation reliability comprises service reliability and inference accuracy reliability, and the low latency takes the function of minimizing the total time delay of the DNN inference task as the objective function; The objective function for maximizing service reliability is: ; The objective function for maximizing the inference accuracy reliability is: ; where n represents a server n, representing a subtask , representing a subtask deployed on a server n, representing service reliability of the server n, representing inference accuracy reliability, representing a mobile device, an edge server, and a central cloud server, representing a total number of servers, representing a subtask set of a DNN inference task, representing a total number of subtasks.

2. The method of claim 1, wherein, The data structure of the physical network topology graph comprises the total number of central cloud servers, edge servers and mobile devices, and the links connected between the central cloud servers, the edge servers and the mobile devices.

3. The method of claim 1, wherein, The DNN inference task comprises multiple DNN inference sub-tasks, and the DNN inference task deployment result is a scheme of offloading DNN inference sub-tasks to different servers; The DNN inference task is accompanied by a maximum inference total time delay, which is used as the inference total time delay constraint, and the inference total time delay comprises the processing delay of DNN inference sub-tasks on the server and the data transmission time delay of data transmission between DNN inference sub-tasks on the link.

4. The method of claim 3, wherein, The method uses the number of CPU cores of the server to represent the computing resources, and the constraint condition of the multi-objective genetic model further comprises a server computing resource constraint. Each DNN inference subtask unloaded to the server consumes computing resources on the server. The server computing resource constraint refers to the total computing resources required by the DNN inference subtasks deployed on each server cannot exceed the maximum computing resources that the server can provide.

5. The method of claim 3, wherein, The constraint condition of the multi-objective genetic model further comprises a deployment constraint. The deployment constraint refers to each DNN inference subtask needs to be deployed on a server, and each DNN inference subtask can only be deployed on one server for execution, and the server can only serve the mobile device, edge server or central cloud server that initiates the inference task.

6. A multi-objective DNN inference task deployment system oriented to operation reliability and latency, comprising a processor, a memory and computer programs / instructions stored on the memory, characterized in that, The processor is configured to execute the computer program / instruction, and when the computer program / instruction is executed, the system implements the steps of the method according to any one of claims 1 to 5.

7. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

8. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

Images