Task migration method, device and storage medium

Abstract

The present application discloses a task migration method, device and storage medium, which relates to the field of artificial intelligence technology, including: obtaining a computational graph corresponding to a computational task of a source device; assigning a target device to each computational node in the computational graph through a linear programming model, wherein the target device is used to execute each of the computational nodes; optimizing the operation of each computational node according to the device resource vector of the target device and the computational requirements of each computational node. The present application reconstructs the computational graph to ensure that the task can be efficiently executed on the target device after the task migration is completed, thereby improving the execution efficiency of the task.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a task migration method, device and storage medium. Background Art

[0002] With the rapid development of AI (Artificial Intelligence) technology, the resource demand for AI computing tasks is increasing. Modern AI systems generally adopt a multi-device parallel computing architecture, which reasonably allocates tasks among multiple heterogeneous devices to fully utilize the computing advantages of each device and improve overall computing efficiency.

[0003] Currently, most existing systems use a static allocation strategy, that is, before the task starts executing, the task is fixedly allocated to a specific device according to preset rules or algorithms. This strategy can optimize resource utilization and improve computing efficiency to a certain extent. However, the static allocation strategy lacks the ability to dynamically adjust at runtime. Once the initial allocation fails to accurately match the actual task requirements and device performance characteristics, or faces sudden changes in computing load, it may lead to resource mismatch, device overload or idleness during task execution, resulting in a waste of computing resources and reduced task execution efficiency. Summary of the invention

[0004] The main purpose of this application is to provide a task migration method, device and storage medium, aiming to improve the execution efficiency of tasks.

[0005] To achieve the above objectives, the present application proposes a task migration method, which includes:

[0006] Obtain the computation graph corresponding to the computation task of the source device;

[0007] Allocating a target device to each computing node in the computing graph through a linear programming model, wherein the target device is used to execute each computing node;

[0008] The operation of each of the computing nodes is optimized according to the device resource vector of the target device and the computing requirements of each of the computing nodes.

[0009] In one embodiment, the step of optimizing the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node includes:

[0010] Determining a degree of match between the device resource vector and each of the computing requirements;

[0011] In the case where the matching degree does not reach a preset matching degree threshold, determining whether the computing node has an operator that is not supported by the target device;

[0012] In a case where the computing node does not have an operator that is not supported by the target device, reconstructing the computing node;

[0013] In the case that at least one operator not supported by the target device exists in the computing node, the operator not supported by the target device is replaced with a dedicated operator of the target device.

[0014] In one embodiment, before the step of optimizing the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node, the step further includes:

[0015] The device context of the target device is configured for each of the computing nodes according to the device resource vector.

[0016] In one embodiment, after the step of optimizing the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node, the step further includes:

[0017] An instruction sequence for the target device is generated according to the instruction set architecture of the target device and the operator of each of the computing nodes.

[0018] In one embodiment, the instruction set architecture includes an accelerated instruction set, and the step of generating an instruction sequence of the target device according to the instruction set architecture of the target device and the operator of each of the computing nodes includes:

[0019] An instruction sequence for the target device is generated according to the execution efficiency of each operator on the target device and the probability of using the accelerated instruction set of the target device.

[0020] In one embodiment, before the step of generating the instruction sequence of the target device according to the instruction set architecture of the target device and the operator of each of the computing nodes, the step further includes:

[0021] According to the computing tasks of each of the operators, a hardware acceleration library supported by the target device is bound to each of the operators.

[0022] In one embodiment, after the step of assigning a target device to each computing node in the computing graph, the method further includes:

[0023] Applying a unified memory management model, adjusting the memory space of the source device and the target device so that the source device and the target device access the same physical address space;

[0024] Mapping the physical address space into a virtual address space;

[0025] Data is transferred between the source device and the target device by accessing the virtual address space.

[0026] In one embodiment, after the step of transmitting data between the source device and the target device by accessing the virtual address space, the method further includes:

[0027] Determining a data transmission time for each of the computing nodes;

[0028] The migration order of each computing node is adjusted according to the data transmission time.

[0029] In addition, to achieve the above-mentioned purpose, the present application also proposes a task migration device, which includes:

[0030] An acquisition module, used to acquire a computational graph corresponding to a computational task of a source device;

[0031] An allocation module, used to allocate a target device to each computing node in the computing graph through a linear programming model, wherein the target device is used to execute each computing node;

[0032] The optimization module is used to optimize the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node.

[0033] In addition, to achieve the above objectives, the present application also proposes a task migration device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the task migration method described above.

[0034] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, the steps of the task migration method described above are implemented.

[0035] One or more technical solutions proposed in this application have at least the following technical effects: first, the calculation graph corresponding to the calculation task of the source device is obtained. Through the calculation graph, the various calculation operations and data dependencies of the calculation task can be clearly displayed, which is convenient for the subsequent calculation node allocation; then, through the linear programming model, the target device is allocated to each calculation node in the calculation graph. Through the linear programming model, various factors that may affect the execution of the calculation node can be comprehensively considered, so as to allocate the most suitable target device for execution to each calculation node, which helps to balance the load of each device and avoid overloading or idleness of some devices, thereby improving the execution efficiency of the task; finally, according to the device resource vector of the target device and the calculation requirements of each calculation node, the operation of each calculation node is optimized, and the resources of the target device are further optimized to ensure that the calculation node can run in the most efficient way under the given device resources. Through such optimization, the node calculation time is shortened and the task execution speed is improved. This application realizes the dynamic migration of calculation tasks and the efficient use of resources through reasonable device allocation and optimization adjustment based on device resource vectors and calculation requirements, thereby improving the execution efficiency of tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A flowchart of the first embodiment of the task migration method of this application is provided;

[0039] Figure 2 A schematic diagram of the judgment flow of operator replacement provided in Embodiment 2 of the present application;

[0040] Figure 3 A schematic diagram of the structure of memory management provided in Embodiment 4 of the present application;

[0041] Figure 4 This is a schematic diagram of the module structure of the task migration device according to an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the task migration method in the embodiment of the present application. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0044] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0045] In this embodiment, for the convenience of description, the following description is made with the task migration system as the execution subject.

[0046] Modern AI computing tasks are usually distributed and executed on multiple heterogeneous devices, such as CPU (Central Processing Unit), GPU (Graphic Processing Unit) and NPU (Neural Network Processing Unit). Static resource allocation strategies are usually adopted for task allocation and execution between heterogeneous devices, which simplifies the complexity of device management to a certain extent. However, in actual operation, it is difficult to make dynamic adjustments based on the real-time status of the device and task requirements. However, the current dynamic device resource allocation technology faces problems such as reconstruction of the computational graph, data transmission efficiency, context management, and instruction set compatibility in actual applications. These problems make it difficult to switch tasks between different devices, making it impossible to fully utilize the advantages of each device to improve the execution efficiency of tasks.

[0047] The present application provides a solution, which obtains the computational graph corresponding to the computational task of the source device, understands the computational logic and dependency of the task, and facilitates the subsequent task allocation; and optimizes the distribution of tasks among different devices through a linear programming model, determines the target device corresponding to each computational node in the computational graph, and ensures that the computational node can be reasonably allocated to the most suitable device for execution, thereby making full use of the computational advantages of different devices; further, according to the device resource vector of the target device, configures the device context of the target device for each computational node, and converts the resource information required for executing the computational node from the corresponding source device to the target device, so that each computational node understands the environment in which it is about to run, that is, the environment of the target device, makes correct resource usage decisions, and ensures that the target device can correctly execute each computational node; further, according to the device resource vector of the target device and the computational requirements of each computational node, optimizes each computational node, perceives and adapts to the resource status of the target device, reduces the computational overhead of the computational node on the basis of realizing basic functions, and improves the execution efficiency of the computational node on the target device; finally, according to the instruction set architecture of the target device, maps the operator of each computational node to the instruction sequence of the target device, and takes into account the specific optimization and limitations of the hardware, ensuring that the operator can be executed with the highest efficiency on the target device, thereby jointly improving the execution efficiency of the task.

[0048] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes the task migration system as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0049] Based on this, the present application embodiment provides a task migration method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the task migration method of the present application.

[0050] In this embodiment, the task migration method includes steps S10 to S30:

[0051] Step S10, obtaining a computation graph corresponding to the computation task of the source device;

[0052] It should be noted that a computational graph refers to a data structure used to represent the execution order of computing tasks and the direction of data flow. It is composed of computing nodes (computing operations) and edges (data dependencies) to form a DAG (Directed Acyclic Graph) to ensure the order of computing and the certainty of the results.

[0053] For example, an undirected graph may be used to record the computing operations of a task, but additional rules (such as topological sorting) are required to generate a directed graph to ensure the order and dependency of task execution.

[0054] Step S20, assigning a target device to each computing node in the computing graph through a linear programming model, wherein the target device is used to execute each computing node;

[0055] It should be noted that the linear programming model is used to optimize (maximize or minimize) the linear objective function under a set of linear constraints. In this embodiment, the goal of the optimization problem is to minimize the task migration cost, such as data transmission time, computing resource switching time, etc.

[0056] Exemplarily, in order to migrate computing tasks to more efficient devices for execution, obtain the computing graph of the source device. The computing graph is a graphical representation of the computing task, which decomposes the computing task into multiple executable nodes (computing nodes). Each node represents a computing operation or function, and these nodes are connected by edges (data dependencies). Through the computing graph, computing tasks can be more clearly assigned to different target devices for execution. When using a linear programming model to assign target devices to each computing node, the optimization goal can be set to minimize the overhead during migration. The defined objective function can refer to the following formula:

[0057]

[0058] in, is the execution time of the task on device i, that is, the sum of the execution times of the computing nodes on the target device i; is the weight, which can be adjusted according to the priority of the task or the load of the device. During the task migration process, the execution time of the task on the target device It directly reflects the resource usage of the target device on the task. By optimizing the task execution time, the resource utilization of each device can be better balanced, thereby indirectly reducing the migration cost. When the execution time is minimized, each computing node is assigned to the corresponding target device according to the target device where each computing node is located at this time.

[0059] Illustratively, the target device includes the source device.

[0060] For example, the linear programming model can be further optimized by comprehensively considering the energy consumption and temperature of the equipment. The objective function is defined as the comprehensive weighted value of the total execution time, total energy consumption and maximum temperature of the computing task. At the same time, constraints are added - setting the upper limit of energy consumption and temperature of each device. In this way, a more optimized task scheduling and resource allocation strategy can be achieved while ensuring the normal operation of the equipment.

[0061] It is understandable that through reasonable task scheduling and resource allocation, the computing power of each device can be fully utilized, the computing waiting time and data transmission overhead can be reduced, thereby improving the execution efficiency of the task.

[0062] Step S30, optimizing each computing node according to the device resource vector of the target device and the computing requirements of each computing node;

[0063] It should be noted that the device resource vector is used to describe the hardware resources of the device, such as memory occupancy, computing cores, remaining storage space, etc.; computing demand refers to the resources and conditions required to complete the tasks of the computing node, such as memory, bandwidth, etc.

[0064] In one feasible embodiment, the device resource vector of the target device is analyzed to identify available resources and resource limitations; for the current computing node, the resources and conditions required to complete the computing task are determined; in the case where the device resource vector cannot meet the computing requirements, the consumption of some resources can be appropriately reduced according to the importance of the required resources to the computing task of the computing node, the related operations of the computing nodes can be merged or deleted, and each computing node is optimized so that the computing requirements of each computing node meet the range of the device resource vector of the target device, so that each target device can correctly execute each computing node.

[0065] For example, methods such as cosine similarity, Jaccard similarity coefficient and Manhattan distance can be used to indicate the degree of match between resources and requirements. If the degree of match does not meet the standard, the computing node can be reconstructed or replaced, for example, the algorithm implementation of the current computing node is changed, or the operator of the current computing node is replaced, so that the current computing node meets the computing capacity of the corresponding target device.

[0066] Exemplarily, other factors, such as cost, efficiency, etc., may be further considered to further optimize each computing node.

[0067] It is understandable that by rationally optimizing the computing nodes, it is possible to ensure that each computing node is executed efficiently, reduce unnecessary waste of resources, and at the same time speed up the execution of tasks and improve the efficiency of task execution.

[0068] This embodiment provides a task migration method, which optimizes the distribution of computing nodes among different devices through a linear programming model to ensure the highest overall execution efficiency of the task; and then optimizes the computing nodes to further improve the execution speed of each computing node on the corresponding target device while ensuring that each computing node can be correctly executed by the target device, thereby jointly improving the execution efficiency of the task.

[0069] Based on the above first embodiment, a second embodiment of the task migration method of the present application is proposed. In this embodiment, step S30 includes:

[0070] Step S31, determining the matching degree between the device resource vector and the computing requirement;

[0071] For example, according to the device resource vector and computing requirements The matching degree between them is used to determine whether each computing node needs to be optimized. The matching degree can be calculated by referring to the following formula:

[0072]

[0073] The numerator represents the inner product of the device resource vector and the node computing demand, which is used to indicate the matching degree between the computing node and the device resources. and The norms of the device resource vector and computing requirements are respectively represented (usually the Euclidean norm, which is the square root of the sum of the squares of the vector elements), which is used to remove the scale differences between the vectors. By dividing the inner product by the product of the norms of the two vectors, a dimensionless matching value can be obtained, which can more intuitively represent the degree of matching between resources and requirements.

[0074] Step S32, when the matching degree does not reach the preset matching degree threshold, determining whether the computing node has an operator that is not supported by the target device;

[0075] It should be noted that the matching threshold refers to a preset value used to determine whether the matching degree meets the requirements for executing the task on a specific device. It is usually set according to the task requirements. For example, it is set to 0.8 in high-performance scenarios and 0.6 for daily use. It can also be adjusted through experiments to balance performance and task migration costs. This embodiment does not specifically limit this.

[0076] It is understandable that this step is to ensure that the task can be executed correctly on compatible devices and avoid task failure due to operator incompatibility.

[0077] For example, if an operator is found that the target device does not support it, the computing node can be reallocated; the target device can also be temporarily blacklisted or the user can be prompted to check or adjust the target device to avoid allocating a similar computing node to the target device again.

[0078] For example, when it is found that there are operators that are not supported by the target device, but can be compensated by using additional software libraries or other tools, the matching degree can be improved by automatically installing the missing libraries or tools.

[0079] Step S33, in the case that the computing node does not have an operator that is not supported by the target device, reconstructing the computing node;

[0080] It should be noted that reconstruction refers to the process of adjusting and optimizing the internal operations of the computing node without changing the functions of the computing node.

[0081] For example, resource overhead can be reduced by operator fusion, that is, merging multiple operations into a more efficient operator. For example, consecutive convolutional layers, activation functions, and pooling layers can be combined into a single convolution operation. The complexity of the overall task can also be reduced by pruning, that is, removing unnecessary computing nodes. Both of the above methods can further improve the overall efficiency of the task.

[0082] It can be understood that by reconstructing the computing node, it is ensured that the target device can execute the computing node normally, and at the same time more efficient computing node deployment and execution are achieved, thereby improving the overall task execution efficiency.

[0083] Step S34: When there is at least one operator in the computing node that is not supported by the target device, the operator that is not supported by the target device is replaced with a dedicated operator for the target device.

[0084] It should be noted that a dedicated operator refers to an operator specially designed for a target device to implement a specific function, and can utilize specific hardware features or software functions of the target device.

[0085] For example, when the target device does not support the Swish function included in the current computing node, the Swish function can be replaced with ReLU (Rectified Linear Unit) or other similar activation functions supported by the target device. Moreover, if the target device supports custom operators, the Swish function can be implemented as an automated operation and deployed to the target device.

[0086] For example, a general matrix multiplication operator on a GPU may be replaced by a dedicated acceleration instruction on an NPU.

[0087] For example, please refer to Figure 2 , Figure 2 A schematic diagram of a judgment process for operator replacement is provided, wherein step S101 is first executed to determine the matching degree between a device resource vector and a computing requirement; then, step S102 is entered to determine whether the matching degree reaches a preset matching degree threshold; if the matching degree threshold is reached, step S103 is executed to normally execute subsequent processes and normally complete the computing task on the target device; if the matching degree threshold is not reached, step S104 is entered to determine whether there is an illegal operator (i.e., an operator not supported by the target device) in the computing node; if there is no illegal operator in the computing node, step S106 is executed to reconstruct the computing node; if there is an illegal operator in the computing node, step S105 is executed to replace the illegal operator with a dedicated operator in the target device that can implement the function.

[0088] In this implementation, by reconstructing the computing nodes and replacing the operators that are not supported by the target device, it is possible to ensure that the computing tasks can run correctly on the target device, thereby improving the compatibility of the system.

[0089] In a feasible implementation manner, before step S30, the method further includes:

[0090] Step S301, configuring a device context of the target device for each computing node according to the device resource vector of the target device;

[0091] It should be noted that the device context refers to the environment and status information required to perform computing tasks on a specific device, including the memory location, data format, and hardware resource information of the current node.

[0092] For example, since the types of resources contained in each device are different, during the migration process, the device context of the computing node needs to be updated from the source device to the target device accordingly. The device resource vector of the target device of the computing node determined by the above linear programming model is Map and update the device context, where: Indicates specific resources on the device, such as memory size. For example, when migrating convolution operations from the GPU to the NPU, the memory information needs to be updated from the GPU's 24GB video memory to the NPU's 8GB memory.

[0093] For example, during the device context mapping process, the device context of the computing node is mapped from the source device Migrate to the target device , you need to meet at least the following migration conditions:

[0094]

[0095] This formula means that the target device must have at least enough resources to run the node migrated from the source device. If the target device does not have enough resources to execute the computing node, the target device of the computing node can be reallocated, or the target devices of all computing nodes in the computing graph can be reallocated.

[0096] For example, in order to adapt to the hardware architecture of the target device, the data alignment and storage format are readjusted according to the device resource vector of the target device to ensure that the target device can efficiently read and execute the data. For example, the NPU's memory access requirements may be different from those of the GPU. The task migration system will repackage the data to ensure data consistency and readability.

[0097] In this implementation, by reallocating the device context, that is, rebinding the resource information required for the computing node to execute on the target device, it is ensured that each computing node can execute correctly on the target device.

[0098] Based on the above first and / or second embodiments, a third embodiment of the task migration method of the present application is proposed. In this embodiment, after step S30, the method further includes:

[0099] Step S40, generating an instruction sequence for the target device according to the instruction set architecture of the target device and the operators of each computing node.

[0100] In one feasible embodiment, the instruction set type and function supported by the target device are determined, each operator is mapped to the corresponding instruction supported by the instruction set architecture of the target device, the execution order of each instruction is determined taking into account the data dependency between the operators, and an instruction sequence is generated; in addition, the instruction sequence can be optimized to improve execution efficiency based on the architectural characteristics of the target device, such as instruction pipeline, single instruction multiple data flow, etc.

[0101] It should be noted that the instruction set architecture refers to a set of specifications and standards in the computer architecture that define how the processor executes instructions. It stipulates the instruction format, instruction type, addressing mode, register structure, etc. that the processor can recognize. Common instruction set architectures include x86, ARM (Advanced RISC Machine), MIPS (Microprocessor without interlocked pipelined stages), etc.

[0102] In addition, it should be noted that an operator is one of the basic units of a computing node, representing an instruction to perform a certain computing operation or data processing, such as a matrix operation, a convolution operation, and the like.

[0103] Exemplarily, identify the operators in the computation graph, and find the corresponding instructions or instruction sequences in the instruction set architecture to implement the same function; optimize the layout of data in memory according to the characteristics of ARM, and use the parallel execution capabilities supported by ARM to map multiple operators to multiple execution units; finally, arrange the execution order of instructions reasonably according to the data dependencies between operators (edges in the computation graph) to obtain the instruction sequence of the target device.

[0104] Exemplarily, operators may be replaced by using discrete optimization problems, continuous optimization problems, heuristic searches, and the like. For example, the optimal operator configuration may be found by using a gradient descent optimization algorithm. The replaceable operator space may be determined by combining device performance and operating characteristics.

[0105] In this implementation, through the operator mapping process, the computing resources of the target device can be fully utilized to improve the execution efficiency of the task.

[0106] In a feasible implementation, the instruction set architecture includes an accelerated instruction set, and step S40 includes:

[0107] Step S41, generating an instruction sequence for the target device according to the execution efficiency of each operator on the target device and the probability of using the accelerated instruction set of the target device.

[0108] It should be noted that execution efficiency refers to the speed or performance of the operator's execution on the target device, which is usually measured by indicators such as computational time complexity, space complexity or throughput, and may depend on the complexity of the operator, the characteristics of the device, and the degree of match between the operator and the device instruction set; accelerated instruction set refers to instructions specifically designed to improve the performance of specific types of computing tasks, such as SIMD (Single Instruction Multiple Data) instructions, vector instructions, etc.

[0109] Exemplarily, the discrete optimization problem is used to replace the operator, and the goal is set to find the best mapping strategy to maximize the efficiency of the operator when it is executed on the target device. The following formula can be referred to:

[0110]

[0111] in, Representation Operator Execution efficiency on the target device, Indicates the target device pair operator The support degree of the acceleration instruction set of the target device is calculated by multiplying the execution efficiency of each operator by the probability of using its acceleration instruction set to obtain the weighted efficiency. The weighted efficiencies of all operators (from 1 to m) are summed up to find the mapping strategy that maximizes this sum. According to the obtained mapping strategy, the operator is mapped to the operation instruction on the target device, including the acceleration instruction, and the instruction sequence is formed according to the dependency relationship.

[0112] In this implementation, using the acceleration instructions of the target device can reduce the execution time of the computing node and improve the execution efficiency of the task.

[0113] In a feasible implementation manner, before step S40, the method further includes:

[0114] Step S401 : Bind a hardware acceleration library supported by a target device to each operator according to the computing task of each operator.

[0115] It should be noted that a hardware acceleration library refers to a function library optimized for specific hardware, which is used to accelerate computationally intensive tasks and usually utilizes the characteristics of hardware (such as the parallel computing capability of GPU) to improve computing efficiency. Binding refers to associating an operator with a specific hardware acceleration library so that the optimization functions provided by the hardware acceleration library can be utilized when executing computing tasks.

[0116] For example, on the NPU, operators including computing tasks such as convolution calculations and matrix operations can be bound to the hardware acceleration library of the NPU, such as a convolution acceleration library, a matrix operation acceleration library, and the like.

[0117] In this implementation, the underlying hardware resources are directly utilized through the hardware acceleration library, which significantly improves the computing performance and thus improves the execution efficiency of the task.

[0118] Based on the above first, second and / or third embodiments, a fourth embodiment of the task migration method of the present application is proposed. In this embodiment, after step S10, the task migration method further includes:

[0119] Step A11, applying a unified memory management model to adjust the memory space of the source device and the target device so that the source device and the target device access the same physical address space;

[0120] It should be noted that the unified memory management model is a memory management mechanism that allows different devices (such as NPU, GPU, other accelerators, etc.) to access shared physical memory space through a unified interface.

[0121] Exemplarily, a unified memory management model is applied so that the CPU and GPU can share the same memory address space, where the CPU is responsible for generating rendering instructions and data, and the GPU is responsible for executing instructions and rendering images. The GPU can then directly read the instructions and data generated by the CPU by accessing the memory address space, without the CPU having to copy the rendering instructions and data to the GPU's memory. This reduces both memory overhead and data copying between devices, thereby reducing data transmission overhead.

[0122] Step A12, mapping the physical address space to the virtual address space.

[0123] For example, please refer to Figure 3 , Figure 3 A schematic diagram of the structure of memory management is provided. The source device and the target device have a shared address space, that is, the source device and the target device can directly read and write the same memory location, but this has problems with isolation and address management. Therefore, it is necessary to map the shared address space to a virtual address space. When the device needs to access the physical memory, the virtual address is converted into a physical address (shared address) according to the mapping table to achieve memory access. Through virtual address mapping, malicious interference between devices is reduced, and address management across devices is simplified.

[0124] Step A13, transferring data between the source device and the target device by accessing the virtual address space.

[0125] In a feasible embodiment, when the source device needs to transfer data to the target device, it accesses the virtual address space to determine the virtual address of the data to be transferred, converts it into a physical address (shared address) through an address mapping mechanism, reads the data from the physical memory, and transfers the data to the shared address; the target device can read the data by accessing the same virtual address and store it in local non-shared physical memory.

[0126] For example, through the asynchronous task execution mechanism, computing tasks can continue to be executed in the foreground, and data transfer and task switching can be performed in the background, ensuring that the task migration process does not affect the overall execution efficiency of the computing tasks. For example, when data is migrated from the GPU to the NPU, the data transfer process will be carried out in the background, and the computing tasks in the foreground can still continue to be executed.

[0127] For example, the event-driven mechanism is used to manage device context updates and data synchronization during task migration, and the event flow is defined. , where each event represents a task migration step, such as data transfer, context switching, etc. That is, when the data transfer is completed, the context update event will be triggered to rebind the computing nodes in the computation graph to the context of the target device, thereby ensuring that the task continues to be executed on the target device.

[0128] In this implementation, by sharing physical memory, multiple copies of data when transmitted between devices are reduced, thereby improving the efficiency of data transmission.

[0129] In a feasible implementation manner, after step A13, the method further includes:

[0130] Step E10, determining the data transmission time of each computing node;

[0131] It should be noted that data transmission time refers to the time required to transmit data from one computing node to another computing node, including network communication delay, data serialization or deserialization time, and possible queuing waiting time.

[0132] For example, Represents the DMA transfer time. The data transfer time of each calculated node can be determined by referring to the following formula:

[0133]

[0134] The data size divided by the bandwidth represents the transmission delay on the network, which includes the data propagation time in the network, the queuing waiting time at each network node, and the actual transmission time.

[0135] Step E20, adjusting the migration order of each computing node according to each data transmission time.

[0136] For example, computing nodes with shorter data transmission time can complete migration faster, freeing up resources for subsequent subtasks, thereby reducing the overall migration delay. Therefore, on the premise of satisfying the dependency relationship, the migration order of each computing node can be arranged according to the length of data transmission time. The shorter the transmission data, the earlier the migration order.

[0137] Exemplarily, the execution time of each computing node on the target device may also be considered, and the data transmission time and execution time may be comprehensively considered to schedule the tasks through a heuristic algorithm (such as shortest remaining time first).

[0138] Exemplarily, the memory access strategy of each computing node may be adjusted according to the data transmission time to reduce high-latency paths and map frequently accessed data to faster storage devices or use a cache strategy.

[0139] In this implementation, the migration order of the computing nodes is adjusted according to the data transmission time, thereby optimizing the task migration process, reducing the migration overhead, and improving the execution efficiency of the tasks after the migration.

[0140] In a feasible implementation manner, after step S10, the method further includes:

[0141] Step A10, mapping the memory addresses of the source device and the target device to the same virtual address space;

[0142] It should be noted that the memory address is the location number of the storage unit in the internal memory, which is used to identify and locate the data in the memory; the virtual address space refers to the logical address space, which is used to manage memory in the operating system or application, and converts the virtual address into a physical address through the address mapping mechanism.

[0143] Step A20: using direct memory access technology to transfer data from the source device to the target device.

[0144] It should be noted that DMA (Direct Memory Access) technology is a technology that allows hardware devices (such as peripherals or processors) to directly access main memory (memory) for data read and write operations without the need for CPU intervention. The DMA controller is responsible for coordinating data transfers and can independently complete data transfers while the CPU is busy performing other tasks.

[0145] Exemplarily, by configuring an MMU (Memory Management Unit), the physical memory addresses of the source device and the target device are mapped to a unified virtual address space; by configuring a DMA controller, data is efficiently and quickly transmitted between the source device and the target device.

[0146] For example, if multiple DMA channels are supported, multiple data transfer tasks can be processed simultaneously, thereby further calculating the execution efficiency of the tasks.

[0147] In this embodiment, through memory address mapping technology, the source device and the target device can share data in the same virtual address space, thereby avoiding repeated transmission and storage of data and improving the efficiency of data transmission; through DMA technology, the memory can be directly accessed without the need for CPU intervention, thereby improving the efficiency of data transmission, thereby jointly reducing the data waiting time of the computing node and improving the execution efficiency of the task.

[0148] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the task migration method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0149] The present application also provides a task migration device, please refer to Figure 4 , the task migration device comprises:

[0150] An acquisition module 10 is used to acquire a computation graph corresponding to a computation task of a source device;

[0151] An allocation module 20, configured to allocate a target device to each computing node in the computation graph through a linear programming model, wherein the target device is used to execute each computing node;

[0152] The optimization module 30 is used to optimize the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node.

[0153] The task migration device provided in the embodiment of the present application adopts the task migration method in the above embodiment to improve the execution efficiency of the task. Compared with the prior art, the beneficial effects of the task migration device provided in the present application are the same as the beneficial effects of the task migration method provided in the above embodiment, and the other technical features in the task migration device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0154] An embodiment of the present application provides a task migration device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the task migration method in the above-mentioned embodiment one.

[0155] Reference below Figure 5, which shows a schematic diagram of the structure of a task migration device suitable for implementing an embodiment of the present application. The task migration device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The task migration device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0156] like Figure 5 As shown, the task migration device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the task migration device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the task migration device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a task migration device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0157] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0158] The task migration device provided in the embodiment of the present application adopts the task migration method in the above embodiment to improve the execution efficiency of the task. Compared with the prior art, the beneficial effects of the task migration device provided in the present application are the same as the beneficial effects of the task migration method provided in the above embodiment, and the other technical features in the task migration device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0159] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0160] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0161] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the task migration method in the above-mentioned embodiment.

[0162] The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0163] The computer-readable storage medium may be included in the task migration device; or may exist independently without being assembled into the task migration device.

[0164] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the task migration device, the task migration device: obtains the calculation graph corresponding to the calculation task of the source device; allocates a target device to each calculation node in the calculation graph through a linear programming model, wherein the target device is used to execute each calculation node; optimizes the operation of each calculation node according to the device resource vector of the target device and the calculation requirements of each calculation node.

[0165] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0166] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0167] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0168] The readable storage medium provided in the embodiment of the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned task migration method, and can improve the execution efficiency of tasks. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the task migration method provided in the above-mentioned embodiment, and will not be described in detail here.

[0169] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the task migration method as described above when executed by a processor.

[0170] The computer program product provided in the embodiment of the present application can improve the execution efficiency of tasks. Compared with the prior art, the beneficial effects of the computer program product provided in the present application are the same as the beneficial effects of the task migration method provided in the above embodiment, which will not be described in detail here.

[0171] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A task migration method, characterized in that: The task migration method comprises: Obtaining a computation graph corresponding to a computation task of a source device, wherein the computation graph includes computation nodes and data dependencies; Allocate a target device to each computing node in the computing graph through a linear programming model, wherein the objective function of the linear programming model is a comprehensive weighted value of the total execution time, total energy consumption and maximum temperature of the computing task, and the constraint conditions added to the linear programming model are: setting an upper limit of energy consumption and an upper limit of temperature for each device, and the target device is used to execute each of the computing nodes; Optimizing the operation of each of the computing nodes according to the device resource vector of the target device and the computing requirements of each of the computing nodes; Generate an instruction sequence for the target device according to the instruction set architecture of the target device and the operator of each of the computing nodes; The step of optimizing the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node includes: Determining a degree of match between the device resource vector and each of the computing requirements; In the case where the matching degree does not reach a preset matching degree threshold, determining whether the computing node has an operator that is not supported by the target device, wherein the matching degree threshold refers to a preset value used to determine whether the matching degree meets the requirement for executing the task on the device, and is set according to the task requirements; In a case where the computing node does not have an operator that is not supported by the target device, performing an operator fusion or pruning operation on the computing node; In the case where the computing node has at least one operator that is not supported by the target device, replacing the operator that is not supported by the target device with a dedicated operator for the target device, wherein the dedicated operator implements the same function as the operator that is not supported by the target device; The instruction set architecture includes an accelerated instruction set, and the step of generating an instruction sequence of the target device according to the instruction set architecture of the target device and the operator of each of the computing nodes includes: When the sum of the products of the execution efficiency of each operator on the target device and the probability of the target device using the acceleration instruction set of each operator reaches a maximum, each operator is mapped to an operation instruction on the target device, and the instruction sequence is generated based on the data dependency.

2. The task migration method according to claim 1, characterized in that: Before the step of optimizing the operation of each computing node according to the device resource vector of the target device and the computing requirements of each computing node, the step further includes: The device context of the target device is configured for each of the computing nodes according to the device resource vector.

3. The task migration method according to claim 1, characterized in that: Before the step of generating the instruction sequence of the target device according to the instruction set architecture of the target device and the operators of each of the computing nodes, the step further includes: According to the computing tasks of each of the operators, a hardware acceleration library supported by the target device is bound to each of the operators.

4. The task migration method according to claim 1, characterized in that: After the step of allocating a target device to each computing node in the computing graph, the method further includes: Applying a unified memory management model, adjusting the memory space of the source device and the target device so that the source device and the target device access the same physical address space; Mapping the physical address space into a virtual address space; Data is transferred between the source device and the target device by accessing the virtual address space.

5. The task migration method according to claim 4, characterized in that: After the step of transmitting data between the source device and the target device by accessing the virtual address space, the method further comprises: Determining a data transmission time for each of the computing nodes; The migration order of each computing node is adjusted according to the data transmission time.

6. A task migration device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the task migration method according to any one of claims 1 to 5.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the task migration method according to any one of claims 1 to 5 are implemented.

Images