Model training task allocation method and system in heterogeneous computing power environment

Abstract

The invention belongs to the field of distributed training, and provides a model training task allocation method and system in a heterogeneous computing power environment, which is used for constructing a layered deep reinforcement learning model for selecting optimized splitting points and allocating resources among a terminal layer, an edge layer and a cloud computing layer. Therefore, efficient calculation task allocation and model training are realized. Tasks with different computing power requirements are coped with by designing a plurality of sub-strategy networks. A resource sensing module is combined to measure the states of computing resources, network bandwidth, delay and the like of each layer. The heuristic algorithm is used for preliminarily estimating a splitting point range, reducing exploration of irrelevant positions and optimizing search efficiency. Resources required by neural network model calculation are measured by the number of floating point operations. In the selection of the splitting points, a deep reinforcement learning method is mainly adopted to be combined with a resource sensing module, and an optimal splitting point is searched through structure optimization, so that effective splitting of the model is realized, and efficient joint training is completed at two end sides.

Description

Technical Field

[0001] The present invention belongs to the field of distributed training technology, and specifically relates to a method and system for allocating model training tasks in a heterogeneous computing power environment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the era of big models, model parameters and data scale grow exponentially. Although the total amount of computing resources is developing rapidly, the computing power that each computing provider can provide is unevenly distributed. Under this unbalanced computing power distribution condition, how to better support big model training and give full play to the current computing power resource advantages has very important research significance and research value.

[0004] Split computing is a new computing mode that splits large models and deploys them in distributed clusters for parallel training. It can improve the efficiency of large model training and make full use of fragmented computing resources. It is the current technical development trend for large model training under distributed computing conditions. However, the existing methods have different choices for split points, resulting in uneven distribution of computing resources, which leads to low training efficiency. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a method and system for allocating model training tasks in a heterogeneous computing environment. The present invention establishes a hierarchical deep reinforcement learning (DRL) model. Taking the terminal-edge layer as an example, different sub-strategy networks learn and optimize for different computing resource requirements, and realize the selection of split points between layers by combining deep reinforcement learning, realizing overall serialization and inter-layer parallelism, and updating the global strategy after each training cycle to achieve real-time task allocation and optimization. The key to the research is how to effectively combine deep reinforcement learning and resource perception modules to optimize the split point selection and task allocation strategy between the terminal and edge layers and achieve overall optimization.

[0006] According to some embodiments, a first solution of the present invention provides a method for allocating model training tasks in a heterogeneous computing environment, using the following technical solutions: A method for allocating model training tasks in a heterogeneous computing environment, comprising: Under the computing architecture consisting of the terminal layer, edge layer, and cloud computing layer, multiple model training requests from the terminal layer are obtained, and multiple terminals are grouped according to the size relationship between the data volume of the model training requests and the set threshold; Based on the computing power information of the edge layer and the computing power requirements of the model training request, a comprehensive evaluation function is used to find the best edge matching node for each terminal; Determine the initial selection range of the split point in the model according to the grouping of the model training request. Within the determined initial selection range, iterate and loop to find the local optimal split point based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point. According to the grouping of model training requests, a preliminary selection range of split points is determined in the model. Within the determined preliminary selection range, the optimal edge-cloud split point is iteratively cycled to find the minimum delay and energy consumption between the edge matching node and the cloud computing layer at the split point. Coordinate the local optimal split point and the edge-cloud optimal split point to allocate training tasks.

[0007] Furthermore, the comprehensive evaluation function is specifically: ; in, It is an edge node of computing resources, is the computing requirement of the current terminal task on the node, is the network bandwidth from the terminal to the edge node, It's network latency. It is a tuning parameter used to balance the weights of computing power, bandwidth, latency, and energy consumption for current user requests. ， There are edge nodes within its search range ; Load balancing factor , is the energy consumption generated by selecting the current node, and the node selection frequency ; The larger the value of the comprehensive evaluation function is, the higher the matching degree between the terminal and the edge node is.

[0008] Further, a hierarchical deep reinforcement learning model is used to solve the iterative loop search process, wherein the hierarchical deep reinforcement learning model includes a state space, an action space, and an overall reward function; The state space includes computing resource status, network bandwidth and latency of the terminal layer; computing resource status of the edge layer and connection status of the cloud server, and computing resource status of the cloud computing layer; The action space includes corresponding splitting schemes for various conditional divisions; The overall reward function includes a reward function of the edge and cloud computing layers and a reward function of the terminal and edge layers.

[0009] Furthermore, the multiple terminals are grouped according to the size relationship between the data volume of the model training request and the set threshold, specifically: Calculate the data volume of the model training request of each terminal in turn; If the amount of data requested by the model training is greater than the set threshold, it is considered a high computing power requirement level task and is classified into the high computing power requirement set; If the amount of data requested for model training is less than the set threshold, it is a low computing power requirement level task and is classified into the low computing power requirement set; Therefore, the model training requests for all terminals in the terminal layer are divided into two groups: a high computing power requirement set and a low computing power requirement set.

[0010] Furthermore, the preliminary selection range of the split point is determined in the model according to the grouping of the model training request. Within the determined preliminary selection range, based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point, the local optimal split point is iteratively cycled to find the specific: If the amount of data requested for model training is less than the set threshold, it is a task with low computing power requirements, and the initial selection range of split points is after the split point in the middle layer of the model structure. Otherwise, it is a task with high computing power requirements, and the initial selection range of split points is before the split point in the middle layer of the model structure. Within the initial selection range, take a split point as an example and calculate the delay and energy consumption between the terminal and the edge node at the current split point; The iterative loop preliminarily selects all split points within the range to determine the local optimal split point based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point.

[0011] Furthermore, the initial selection range of the split point is determined in the model according to the grouping of the model training request. Within the determined initial selection range, based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the split point, the optimal edge-cloud split point is iteratively cycled, specifically: If the amount of data requested for model training is less than the set threshold, it is a task with low computing power requirements, and the initial selection range of split points is after the split point in the middle layer of the model structure. Otherwise, it is a task with high computing power requirements, and the initial selection range of split points is after the split point in the middle layer of the model structure. Within the initial selection range, take a split point as an example to calculate the delay and energy consumption between the cloud computing layer and the edge node at the current split point; The iterative loop preliminarily selects all split points within the range to determine the local optimal split point based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the split point.

[0012] Furthermore, the local optimal split point and the edge-cloud optimal split point are coordinated to achieve the allocation of training tasks, specifically: Based on the local optimal splitting point, the corresponding inter-layer positions of the structural layer of the model are split; Based on the edge-cloud optimal split point, the corresponding inter-layer positions of the structural layer of the model are split; The structural layer of the model is divided into three structural blocks, and then the same model training request task is split into three tasks for parallel training to achieve the distribution of training tasks.

[0013] According to some embodiments, a second solution of the present invention provides a model training task allocation system in a heterogeneous computing environment, which adopts the following technical solutions: A model training task allocation system in a heterogeneous computing environment, comprising: The task computing power identification module is configured to obtain multiple model training requests from the terminal layer under the computing architecture consisting of the terminal layer, the edge layer, and the cloud computing layer, and group the multiple terminals according to the size relationship between the data volume of the model training request and the set threshold; A matching node determination module is configured to find the best edge matching node for each terminal using a comprehensive evaluation function based on the computing power information of the edge layer and the computing power requirements of the model training request; The terminal-edge splitting module is configured to determine a preliminary selection range of split points in the model according to the grouping of the model training request, and iteratively loop to find a local optimal split point within the determined preliminary selection range based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point; The edge-cloud splitting module is configured to determine a preliminary selection range of splitting points in the model according to the grouping of model training requests, and iteratively loop to find the best edge-cloud splitting point based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the splitting point within the determined preliminary selection range; The training task allocation module is configured to coordinate the local optimal splitting point and the edge-cloud optimal splitting point to allocate training tasks.

[0014] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0015] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in a method for allocating model training tasks in a heterogeneous computing environment as described in the first aspect above.

[0016] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0017] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for allocating model training tasks in a heterogeneous computing environment as described in the first aspect above are implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention constructs a layered deep reinforcement learning architecture for optimizing the selection of split points and resource allocation between the terminal layer, edge layer and cloud computing layer, thereby achieving efficient computing task allocation and model training. Multiple sub-strategy networks are designed to cope with tasks with different computing power requirements. A resource perception module is combined to measure the computing resources, network bandwidth and latency of each layer. The heuristic algorithm is used to preliminarily estimate the range of split points, reduce the exploration of irrelevant locations, and optimize the search efficiency. In the first stage, the split points are roughly selected to screen out the better areas; in the second stage, these areas are finely optimized to gradually improve the training accuracy, and the rewards of high-computing power and low-computing power tasks are summarized through the reward signal integration mechanism, and the global strategy is updated after each control cycle. Real-time task allocation and optimization are achieved through joint strategies, so that the system can achieve overall optimization in a multi-level architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 Schematic diagram of the layered DRL model calculation architecture in an embodiment of the present invention; Figure 2 This is a flow chart of selecting the best split point in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0023] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0025] Terminology explanation: Heterogeneous computing power: There are differences in the computing power that computing power providers can provide, and there are differences in the model training tasks requested by users.

[0026] Embodiment 1 This embodiment provides a method for allocating model training tasks in a heterogeneous computing environment. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps: Under the computing architecture consisting of the terminal layer, edge layer, and cloud computing layer, multiple model training requests from the terminal layer are obtained, and multiple terminals are grouped according to the size relationship between the data volume of the model training requests and the set threshold; Based on the computing power information of the edge layer and the computing power requirements of the model training request, a comprehensive evaluation function is used to find the best edge matching node for each terminal; Determine the initial selection range of the split point in the model according to the grouping of the model training request. Within the determined initial selection range, iterate and loop to find the local optimal split point based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point. According to the grouping of model training requests, a preliminary selection range of split points is determined in the model. Within the determined preliminary selection range, the optimal edge-cloud split point is iteratively cycled to find the minimum delay and energy consumption between the edge matching node and the cloud computing layer at the split point. Coordinate the local optimal split point and the edge-cloud optimal split point to allocate training tasks.

[0027] The computing delays of the terminal layer, edge layer, and cloud computing layer are affected by the parameters and quantity of the deep neural network model and their respective computing capabilities. The resources required for the neural network model calculation are measured by the number of floating-point operations. In the selection of split points, the deep reinforcement learning method is mainly used in combination with the resource perception module to find the optimal split point through structural optimization to achieve effective splitting of the model and complete efficient joint training on both ends.

[0028] This embodiment is described from the following three aspects, specifically including: 1. Construction of hierarchical deep reinforcement learning model.

[0029] Take the terminal layer and edge layer as an example: First, define its state space, including the computing resource status of each layer (such as CPU usage, memory utilization), network bandwidth, latency, and current computing task requirements. Represents the computing power (number of CPU cores, utilization), network bandwidth, and latency of the terminal layer. Represents the computing resource status of the edge server and the connection status of the cloud server, Represents the computing resource status of the cloud computing layer.

[0030] Secondly, define its action space or search space, covering the corresponding splitting schemes divided according to various conditions, and explore every possible splitting point as much as possible according to the diversity of its structure and the characteristics of global search. Specifically, the current state is three layers, and it is necessary to find the best splitting points of the two splitting points of "terminal-edge" and "edge-cloud". Here, based on resource perception, select the appropriate search method, and reasonably select the splitting point according to the characteristics of its unbalanced computing power distribution to achieve real-time global optimization, and temporarily determine the current best splitting point.

[0031] In each model training cycle, the model is split according to the split point set in the cycle, and the split performance is evaluated at the split point, including the system performance, communication overhead, computing efficiency, etc. after the split. The reward function is set to evaluate the value of each action generated by the current action, and the parameters of the model are continuously updated. After the control cycle consisting of multiple training cycles is completed, its policy function is updated.

[0032] like Figure 1 As shown, this embodiment specifically designs a multi-level DRL agent architecture (deep reinforcement learning (DRL)). Specifically, using multi-agent or hierarchical DRL methods in DRL can achieve segmentation point and resource allocation optimization in a multi-layer architecture. In measuring the resources consumed by model calculation, the number of floating point operations (FLOPS) is used as a unit.

[0033] Similarly, taking the terminal-edge layer as an example, we first need to set a split point at this layer , indicating that this split point is set between the terminal (UE) and the edge layer A user device is in its The split point of the model selected by the control cycle. , here is a set of levels. When the split point is selected at the extreme of the terminal layer of the splittable position, that is, In this case, all computing tasks are handled by the edge layer after the split point. Taking the DNN model training task as an example, the terminal layer does not perform any calculations for the DNN model training task. At this time, the computing delay of the terminal layer is 0, and all computing tasks are handled by the edge layer. , the split point is placed at the end, that is, all computing tasks are handed over to the terminal layer.

[0034] Before distinguishing the split point by delay, the split point selection can be determined based on the current request volume of the terminal layer, that is, a preliminary estimate is made using a heuristic algorithm before training to quickly find a preliminary split point range. This heuristic method narrows the search space and allows the model training to focus on the possible best split point range, thereby reducing the exploration time of irrelevant locations.

[0035] 2. Training process: Phase 1 By dividing the training process into multiple stages, each stage focuses on different split point accuracy. In the first stage, the split point position is roughly selected to screen out the better area.

[0036] If the amount of data requested for model training is less than the set threshold, it is a task with low computing power requirements, and the initial selection range of split points is after the split point in the middle layer of the model structure; On the contrary, for tasks with high computing power requirements, the initial selection range of split points is before the split point in the middle layer of the model structure.

[0037] What this means is that multiple terminals are first grouped according to the size relationship between the amount of data requested by the model training and the set threshold, and divided into a high computing power requirement set and a low computing power requirement set. Then, based on this grouping situation, a preliminary selection range of the split point is divided before or after the split point of the middle layer of the entire structural layer of the model.

[0038] It is understandable that in the model training request sent by the terminal, each structural layer of the model is used as a split point. By comparing the latency and energy consumption of different split points, it is possible to measure whether the current split point is better than other selected split points.

[0039] Take one split point as a training cycle, and all split points within this initial selection range constitute a control cycle, that is, one control cycle contains multiple training cycles.

[0040] Phase II (1) Calculation delay If the split point is chosen between the terminal and edge layers, the transmission delay at the terminal layer can be expressed as: (1); in, Is a user device In the model structure The amount of data in the layer, That is, the total amount of data to be processed at the terminal layer. Indicates the number of FLPOS that each core in the terminal layer can process. Indicated in The number of cores in this control cycle table. That is, the computational delay is obtained by dividing the total number of FLOPS required to complete the task by the number of FLOPS that can be processed simultaneously per second in the current control cycle.

[0041] The computing task is partially performed by the edge layer. The computing delay of the edge layer can be expressed as: (2); in, is the total amount of data to be processed in the edge layer. Indicates the number of FLPOS that each core in the edge layer can process. Indicated in This controls the number of kernels in the periodic table. It is worth noting that the model structure layer involved in the split is only used for splitting, not for calculation, in order to avoid repeated calculations.

[0042] Similarly, the computational delay of the edge-to-cloud computing layer is also related to the choice of its split point. Therefore, the computational delay of the cloud computing layer is specifically: (3); in, It is the total amount of data to be processed by the cloud servers in the cloud computing layer. It is the number of FLPOS that can be processed by each core of the cloud server providing computing in the current cloud computing layer. It means in The number of cores provided during this control period according to its policy function.

[0043] (2) Transmission delay The transmission delay between layers is the second point that needs to be considered, also starting from the terminal-edge layer.

[0044] The delay caused by the terminal-edge layer uplink transmission is: (4); in, Is a user device The uplink transmission delay generated at the terminal-edge, Indicates that on this terminal side based on The amount of data transmitted at this split point, Indicates the uplink transmission rate on the terminal side. It is the inherent delay in the uplink transmission process.

[0045] In each training cycle, in addition to the previous layer transmitting data forward to the next layer, the next layer also needs to return certain feedback to the previous layer to optimize the model parameters. The layers here refer to the terminal layer, edge layer, and cloud computing layer.

[0046] (5); in, It is used Downlink transmission delay caused by terminal-edge, yes The amount of data sent from the edge node to the terminal layer at this split point, is the downlink transmission rate of the edge layer, It is the inherent delay of downlink transmission between the edge layers.

[0047] Similarly, the uplink transmission delay of the edge-to-cloud computing layer is: (6); in, Is a user The uplink transmission delay caused by the edge-terminal, Indicates that on this edge side based on The amount of data transmitted at this split point, Indicates the uplink transmission rate of the edge side. It is the inherent delay in the uplink transmission process; The downlink transmission delay of the edge-to-cloud computing layer is: (7); in, It is used Downlink transmission delay caused by edge layer-cloud computing layer, yes The amount of data sent from the cloud computing layer to the edge layer at this split point, is the downlink transmission rate of the cloud computing layer, It is the inherent delay of the downlink transmission in the cloud computing layer.

[0048] In a complete training cycle (epoch), the model will perform a forward and backward transmission on the entire dataset, and the above four formulas will be called once for each transmission. It is worth noting that if the dataset is divided into multiple batches for transmission during the training cycle, then each transmission needs to call the uplink and downlink formulas once for the transmission of model data.

[0049] After a control cycle training consisting of multiple training cycles is completed, the results of multiple training tasks need to be summarized and evaluated, and the final updated model parameters are transmitted to the end user device at the terminal layer so that the user device has the latest training model and parameter synchronization is achieved.

[0050] (8); in, It is the total amount of data to be processed by the cloud servers in the cloud computing layer. This means that the split point between the edge and the cloud is not at the end but between layers. It is the update delay of the control cycle of the model between the edge-cloud computing layer; The above formula indicates that if the split point of the edge-cloud computing layer is before the last layer of the DNN model, that is, the cloud computing layer needs to be transmitted, the model strategy that needs to be updated in the cloud computing layer is transmitted to the edge layer, which is consistent with the current downlink transmission rate. Divide by, and take into account the current inherent transmission delay Otherwise, there is no need for transmission, and the data transmission delay is 0. Different from the previous downlink transmission, which is to find the local optimum of a single training cycle, the purpose here is to coordinate multiple training results to find the overall optimum.

[0051] Similarly, when the edge layer transmits content to each user device in each terminal layer, specifically: (9); in, This means that the split point in the terminal layer and edge layer is not at the end but between layers. It is the update delay of the control cycle of the model between the terminal and the edge layer; Parameter synchronization is achieved by updating the model parameters after DNN model training.

[0052] User Equipment The total delay incurred in one training cycle is: (10); in, Indicates that a control cycle contains In a training cycle, no matter it is to find the split point between the terminal layer and the edge layer, or the split point between the edge layer and the cloud computing layer, the goal is to add up the corresponding energy consumption and then minimize the sum of the total delays generated in a training cycle.

[0053] (3) Energy consumption If latency is the efficiency that is constantly being pursued, energy consumption is a constraint that needs to be considered in the pursuit of efficiency. The energy consumption generated by the system is also a focus of attention. Energy consumption mainly consists of two aspects, namely computing energy consumption and transmission energy consumption. This embodiment only considers the dynamic power consumption of the device during execution, taking the terminal layer as an example: , after sorting, we can get: (11); in, Indicates the energy consumption calculated based on the split point. Specifically, is the capacitive load of the processor, is the operating voltage of the processor, is the processing frequency of the processor. It is the ratio of data volume to uplink transmission, that is, the time consumed for data transmission. is the transmission power between the current terminal layer and the edge layer, Represents the energy consumption of the terminal layer in one training cycle.

[0054] Similarly, the energy consumption of the edge layer can be expressed as: (12); Different from the terminal layer, which only needs to consider its own computing energy consumption and the uplink transmission energy consumption, Indicates the energy consumption of the edge layer in a training cycle. The edge layer includes its own computing energy consumption, the downlink transmission energy consumption with the terminal layer, the uplink transmission energy consumption with the cloud computing layer, and the synchronization of the current training cycle. , Indicates that the energy consumption of the current training cycle is the energy consumption of the entire control cycle. , It is the transmission power between the current edge layer and the cloud computing layer.

[0055] Similarly, the energy consumption of the cloud computing layer can also be expressed as: (13); It represents the energy consumption of the cloud computing layer in one training cycle. No matter it is to find the split point between the terminal layer and the edge layer, or the split point between the edge layer and the cloud computing layer, the goal is to minimize the sum of the corresponding energy consumption.

[0056] 3. Node and split point selection: like Figure 2 As shown in the figure, the computing power requirement level of the task is identified by the amount of data (FLOPS) through the request of the terminal layer. That is, before the DRL model starts training, the input tasks are preprocessed and classified into two categories: high computing power requirement and low computing power requirement.

[0057] (14); in, Included The collection of all user requests, Included The collection of all user requests, Set a threshold for the amount of data.

[0058] Based on formula (14), multiple terminals are grouped according to the relationship between the amount of data requested for model training and the set threshold, specifically: Calculate the data volume of the model training request of each terminal in turn; If the amount of data requested by the model training is greater than the set threshold, it is considered a high computing power requirement level task and is classified into the high computing power requirement set; If the amount of data requested for model training is less than the set threshold, it is a low computing power requirement level task and is classified into the low computing power requirement set; Therefore, the model training requests for all terminals in the terminal layer are divided into two groups: a high computing power requirement set and a low computing power requirement set.

[0059] Unlike the delay caused by calculation, it is necessary to find edge nodes that match the current terminal computing power requirements before transmission. That is, in response to the uneven distribution of computing power, a resource perception model is established to enable terminal devices to evaluate the current computing power, network bandwidth, and delay of each edge node, and to help find edge nodes within the range by defining a function.

[0060] (15); in, It is an edge node of computing resources, is the computing demand (task volume) of the current terminal task on the node, is the network bandwidth from the terminal to the edge node, It's network latency. It is a tuning parameter used to balance the weights of computing power, bandwidth, latency, and energy consumption for current user requests. , which has In order to avoid always selecting nodes with high computing resources instead of suitable ones, a load balancing factor is added to the evaluation function. Prevent individual nodes from being overloaded, Representation Node The ratio of the total computational workload of all currently assigned tasks to the amount of tasks that can be processed. is the energy consumption generated by selecting the current node. In addition, to avoid an edge node Selected by multiple users, introduced , based on the real-time load of the current edge node With maximum load The ratio of the node load and the weighted Add them together and you get, , Focus on resource utilization, which is used to measure the proportion of the current load of a certain type of resource (such as bandwidth or computing). A higher value indicates that it is close to its maximum load capacity and the penalty is greater. Indicates the proportion of the current request volume. A higher value indicates that the server is close to its maximum concurrent processing capacity and the penalty is larger. Through real-time updates, less used nodes are given priority, so that the system tends to more evenly select edge nodes to participate in the calculation.

[0061] The idea of ​​this embodiment is as follows: through comprehensive evaluation function To select the terminal device to match the appropriate edge node, that is, after grouping requests with different computing power requirements, they are trained separately to avoid disordered search in a large range. For high-computing power requests, the model directly learns to split near the front end of the hierarchy; for low-computing power requests, the split point can be kept at the back to reduce transmission delay. This makes the split point location clearer and more certain, which helps to quickly find the optimal location for each type of request.

[0062] For multiple user requests in the terminal layer Also try to select the best set of edge nodes , considering the proposed The function performs the matching of the current selection. In order to achieve more flexibility, Weights are assigned to different devices to reflect their priorities. (18); in, All terminal devices The sum of functions.

[0063] When maximizing the comprehensive evaluation function, the following constraints must be met: (16); (17); (18); in, The tolerance range of load balancing is set, and the differences of all nodes are limited to this range to achieve the purpose of load balancing.

[0064] Construct a hierarchical DRL model, including two sub-strategy networks, which are respectively designed for tasks with high computing power requirements and tasks with low computing power requirements, and learn with different focuses. Specifically, taking the terminal-edge layer as an example, the high computing power requirement sub-network will prioritize learning to position the split point forward, focusing on computing at the edge nodes, while the low computing power requirement sub-network will tend to position the split point backward, using edge and cloud computing resources, and learn according to this focus. In each training cycle, the corresponding sub-strategy is selected for training based on the task classification results. Through division, the location of the optimal split point for the splitting training of the two is also emphasized. The former is more focused on This range, the latter This range, by reducing the search range, can achieve higher training accuracy with the same number of training times.

[0065] According to the final The value provides a return value and is constantly updated parameters to form a closed loop.

[0066] By setting up a reward function (19); Likewise, (20); Among them, m and n are high computing power request sets Task requests and low computing power request collection in Task request in is the computational latency of the task requests in the high-computing request set based on the split point, is the transmission delay of the task request in the high computing power request set based on the split point, is the task request transmission delay in the low computing power request set based on the split point, is the computational latency of the task requests in the low-computing request set based on the split point, and They are weight factors; A reward-based integration mechanism is used to aggregate the reward signals of high-computing power and low-computing power tasks, and the global strategy is updated after each control cycle, using different weight parameters to balance the importance of the two types of tasks.

[0067] (twenty one); in, is the reward function of the terminal layer and the edge layer, is the weight of high computing power tasks, is the weight of the current terminal-edge link in the overall link in the high computing power set, It is the weight of the current terminal-edge link in the overall link in the low computing power set.

[0068] It can be seen as learning and finding the best split point in the current training cycle separately in the early stage of training, coordinating the two types of computing tasks in the later stage of training, and using a joint strategy for real-time task allocation and split point selection. and It is the weight of the current task, which allows more flexible task handling. is the weight of high-computing tasks, reflecting the impact of high-computing tasks in the reward function. If the value is greater than , the model will be more inclined to choose to optimize the split point position of high computing power tasks, and vice versa, it will pay more attention to optimizing the split point position of low computing power tasks. After that, each learning process is only related to the previous one, reflecting the Markov property.

[0069] Combine it with the edge-cloud computing layer to achieve overall optimization. The reward function of the edge and cloud computing layers is designed first. (twenty two); in, is the reward function between the edge layer and the cloud computing layer, , and is the weight factor, It is the weight of the link between the current best edge node in the best edge node set and the cloud computing layer in the overall link.

[0070] Redesign the overall reward function: (twenty three); in, is the weight factor of the reward function between the terminal and the edge layer, is the weight factor between the edge-cloud computing layer.

[0071] Through this multi-dimensional reward mechanism, combined with the reward signals of high and low computing power tasks, the iterative update optimization of the system is achieved. The reward function provides feedback signals for strategy updates. The decision function here uses the PPO algorithm, which limits the strategy update range so that the new strategy will not deviate significantly from the old strategy.

[0072] The main idea of ​​this method is to build a hierarchical deep reinforcement learning architecture for optimizing the selection of split points and resource allocation between the terminal layer, edge layer, and cloud computing layer, so as to achieve efficient computing task allocation and model training. Multiple sub-strategy networks are designed to cope with tasks with different computing power requirements. A resource-aware module is combined to measure the computing resources, network bandwidth, and latency of each layer. A heuristic algorithm is used to preliminarily estimate the range of split points, reduce the exploration of irrelevant locations, and optimize the search efficiency. In the first stage, the split points are roughly selected to screen out the better areas; in the second stage, these areas are finely optimized to gradually improve the training accuracy. Through the reward signal integration mechanism, the rewards of high-computing power and low-computing power tasks are summarized, and the global strategy is updated after each control cycle. Real-time task allocation and optimization are achieved through joint strategies, so that the system can achieve overall optimization in a multi-level architecture.

[0073] Based on the local optimal splitting point, the corresponding inter-layer positions of the structural layer of the model are split; Based on the edge-cloud optimal split point, the corresponding inter-layer positions of the structural layer of the model are split; The structural layer of the model is divided into three structural blocks, and then the same model training request task is split into three tasks for parallel training to achieve the distribution of training tasks.

[0074] Embodiment 2 This embodiment provides a model training task allocation system in a heterogeneous computing environment, including: The task computing power identification module is configured to obtain multiple model training requests from the terminal layer under the computing architecture consisting of the terminal layer, the edge layer, and the cloud computing layer, and group the multiple terminals according to the size relationship between the data volume of the model training request and the set threshold; A matching node determination module is configured to find the best edge matching node for each terminal using a comprehensive evaluation function based on the computing power information of the edge layer and the computing power requirements of the model training request; The terminal-edge splitting module is configured to determine a preliminary selection range of split points in the model according to the grouping of the model training request, and iteratively loop to find a local optimal split point within the determined preliminary selection range based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point; The edge-cloud splitting module is configured to determine a preliminary selection range of splitting points in the model according to the grouping of model training requests, and iteratively loop to find the best edge-cloud splitting point based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the splitting point within the determined preliminary selection range; The training task allocation module is configured to coordinate the local optimal splitting point and the edge-cloud optimal splitting point to allocate training tasks.

[0075] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0076] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0077] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0078] Embodiment 3 This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the method for allocating model training tasks in a heterogeneous computing power environment as described in the first embodiment above are implemented.

[0079] Embodiment 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a method for allocating model training tasks in a heterogeneous computing environment as described in the first embodiment above are implemented.

[0080] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0081] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0084] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0085] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for allocating model training tasks in a heterogeneous computing environment, characterized in that: include: Under the computing architecture consisting of the terminal layer, edge layer, and cloud computing layer, multiple model training requests from the terminal layer are obtained, and multiple terminals are grouped according to the size relationship between the data volume of the model training requests and the set threshold; Based on the computing power information of the edge layer and the computing power requirements of the model training request, a comprehensive evaluation function is used to find the best edge matching node for each terminal; Determine the initial selection range of the split point in the model according to the grouping of the model training request. Within the determined initial selection range, iterate and loop to find the local optimal split point based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point. According to the grouping of model training requests, a preliminary selection range of split points is determined in the model. Within the determined preliminary selection range, the optimal edge-cloud split point is iteratively cycled to find the minimum delay and energy consumption between the edge matching node and the cloud computing layer at the split point. Coordinate the local optimal split point and the edge-cloud optimal split point to allocate training tasks.

2. A method for allocating model training tasks in a heterogeneous computing environment as claimed in claim 1, characterized in that: The comprehensive evaluation function is specifically: ； in, It is an edge node of computing resources, is the computing requirement of the current terminal task on the node, is the network bandwidth from the terminal to the edge node, It's network latency. It is a tuning parameter used to balance the weights of computing power, bandwidth, latency, and energy consumption for current user requests. ， Within its search range ; Load balancing factor , is the energy consumption generated by selecting the current node, and the node selection frequency ; The larger the value of the comprehensive evaluation function is, the higher the matching degree between the terminal and the edge node is.

3. The method for allocating model training tasks in a heterogeneous computing environment according to claim 1, characterized in that: A hierarchical deep reinforcement learning model is used to solve the iterative loop search process, wherein the hierarchical deep reinforcement learning model includes a state space, an action space, and an overall reward function; The state space includes computing resource status, network bandwidth and latency of the terminal layer; computing resource status of the edge layer and connection status of the cloud server, and computing resource status of the cloud computing layer; The action space includes corresponding splitting schemes for various conditional divisions; The overall reward function includes a reward function of the edge and cloud computing layers and a reward function of the terminal and edge layers.

4. The method for allocating model training tasks in a heterogeneous computing environment according to claim 1, characterized in that: The multiple terminals are grouped according to the size relationship between the data volume of the model training request and the set threshold, specifically: Calculate the data volume of the model training request of each terminal in turn; If the amount of data requested by the model training is greater than the set threshold, it is considered a high computing power requirement level task and is classified into the high computing power requirement set; If the amount of data requested by the model training is less than the set threshold, it is a task with low computing power requirements and is classified into the low computing power requirements set; Therefore, the model training requests for all terminals in the terminal layer are divided into two groups: a high computing power requirement set and a low computing power requirement set.

5. The method for allocating model training tasks in a heterogeneous computing environment according to claim 1, characterized in that: The method determines a preliminary selection range of split points in the model according to the grouping of model training requests, and iterates and loops to find the local optimal split point within the determined preliminary selection range based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point, specifically: If the amount of data requested for model training is less than the set threshold, it is a task with low computing power requirements, and the initial selection range of split points is after the split point in the middle layer of the model structure. Otherwise, it is a task with high computing power requirements, and the initial selection range of split points is before the split point in the middle layer of the model structure. Within the initial selection range, take a split point as an example and calculate the delay and energy consumption between the terminal and the edge node at the current split point; The iterative loop preliminarily selects all split points within the range to determine the local optimal split point based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point.

6. The method for allocating model training tasks in a heterogeneous computing environment according to claim 1, characterized in that: The method determines a preliminary selection range of split points in the model according to the grouping of model training requests. Within the determined preliminary selection range, based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the split point, iterates and loops to find the best edge-cloud split point, specifically: If the amount of data requested for model training is less than the set threshold, it is a task with low computing power requirements, and the initial selection range of split points is after the split point in the middle layer of the model structure. Otherwise, it is a task with high computing power requirements, and the initial selection range of split points is before the split point in the middle layer of the model structure. Within the initial selection range, take a split point as an example to calculate the delay and energy consumption between the cloud computing layer and the edge node at the current split point; The iterative loop preliminarily selects all split points within the range to determine the local optimal split point based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the split point.

7. The method for allocating model training tasks in a heterogeneous computing environment according to claim 1, characterized in that: The coordination of the local optimal split point and the edge-cloud optimal split point is used to allocate the training tasks, specifically: Based on the local optimal splitting point, the corresponding inter-layer positions of the structural layer of the model are split; Based on the edge-cloud optimal split point, the corresponding inter-layer positions of the structural layer of the model are split; The structural layer of the model is divided into three structural blocks, and then the same model training request task is split into three tasks for parallel training to achieve the distribution of training tasks.

8. A model training task allocation system in a heterogeneous computing environment, characterized in that: include: The task computing power identification module is configured to obtain multiple model training requests from the terminal layer under the computing architecture consisting of the terminal layer, the edge layer, and the cloud computing layer, and group the multiple terminals according to the size relationship between the data volume of the model training request and the set threshold; A matching node determination module is configured to find the best edge matching node for each terminal using a comprehensive evaluation function based on the computing power information of the edge layer and the computing power requirements of the model training request; The terminal-edge splitting module is configured to determine a preliminary selection range of split points in the model according to the grouping of the model training request, and iteratively loop to find a local optimal split point within the determined preliminary selection range based on the goal of minimizing the delay and energy consumption between the terminal and the edge matching node at the split point; The edge-cloud splitting module is configured to determine a preliminary selection range of splitting points in the model according to the grouping of model training requests, and iteratively loop to find the best edge-cloud splitting point based on the goal of minimizing the delay and energy consumption between the edge matching node and the cloud computing layer at the splitting point within the determined preliminary selection range; The training task allocation module is configured to coordinate the local optimal splitting point and the edge-cloud optimal splitting point to allocate training tasks.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements a method and steps of allocating model training tasks in a heterogeneous computing environment as described in any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the steps in a method for allocating model training tasks in a heterogeneous computing environment as described in any one of claims 1-7.

Images