Computing task allocation method and system, electronic equipment, medium and product

Abstract

The invention discloses a calculation task distribution method and system, electronic equipment, a medium and a product, and relates to the technical field of artificial intelligence calculation and hardware architecture, the calculation task distribution method and system are applied to a calculation task distribution system, the calculation task distribution system comprises an equipment abstraction layer and all equipment, and the equipment abstraction layer comprises standardized interfaces of all the equipment. Comprising the following steps: when a new device is accessed, distributing a calculation task through a device abstraction layer, if the resource load of the new device exceeds the limit, screening idle devices with low resource load, selecting a target device with high overlap ratio with a hardware resource unit of the new device and highest matching degree with the calculation task from the idle devices, migrating the calculation task to the target device, and transferring the target device to the new device. And matching the instruction set. The technical problem that calculation task allocation is not intelligent is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence computing and hardware architecture technology, and in particular to a computing task allocation method, system, electronic device, storage medium and computer program product. Background Art

[0002] In the current AI application development and deployment, developers need to deal with a variety of heterogeneous hardware devices (such as GPU, NPU, CPU, etc.). The diversity of these devices poses a great challenge to the reasonable allocation of computing tasks. Existing technologies need to handle a large number of computing tasks when developing AI applications. Due to the differences between devices, different devices cannot be flexibly called, and the incompatibility between devices needs to be considered, which increases the difficulty of development. In addition, traditional computing task allocation methods often rely on static rules to allocate tasks and cannot adapt to the dynamic changes of devices, resulting in unintelligent task allocation and increased development complexity. Therefore, there is a problem of unintelligent computing task allocation in the current AI application development and deployment. Summary of the invention

[0003] The main purpose of this application is to provide a computing task allocation method, system, electronic device, storage medium and computer program product, aiming to solve the technical problem of unintelligent computing task allocation.

[0004] To achieve the above object, the present application proposes a computing task allocation method, characterized in that it is applied to a computing task allocation system, the computing task allocation system includes a device abstraction layer, the device abstraction layer defines a standardized interface for each device in the task allocation system, and the computing task allocation method includes: When a new device is detected to be connected to the computing task allocation system, the computing task is allocated to the new device for execution through the device abstraction layer; In the case where the resource load of the new device exceeds a preset resource load upper limit, screening out idle devices whose resource load is lower than a preset resource load lower limit from the devices in the computing task allocation system; The idle device whose hardware resource unit overlaps with each other higher than a preset overlap threshold and whose matching degree with the computing task is the highest is recorded as the target device, the computing task is migrated to the target device for execution, and the computing task is matched with the instruction set of the target device.

[0005] In one embodiment, the step of allocating the computing task to the new device for execution through the device abstraction layer includes: Acquire device characteristics of the new device through a resource set of the new device, and detect task requirements of each computing task in the computing task allocation system; Matching each detected task requirement with the device feature to obtain each matching degree; The computing task with the highest matching degree is allocated to the new device through the device abstraction layer, and the hardware resource unit that meets the computing requirements of the computing task is driven by the corresponding driver module of the new device to execute the computing task.

[0006] In one embodiment, before the step of allocating the computing task to the new device for execution through the device abstraction layer, the step further includes: Monitoring whether new devices are connected to the computing task allocation system; When a new device is detected to be connected to the computing task allocation system, the device type of the new device is identified, and a driver module corresponding to the new device is loaded according to the device type, wherein the driver module is used to drive a hardware resource unit on the new device; A hardware resource unit on the new device is identified to obtain a resource set of the new device.

[0007] In one embodiment, the step of recording the idle device whose hardware resource unit overlap degree among the idle devices is higher than a preset overlap degree threshold and whose matching degree with the computing task is the highest as the target device comprises: Calculating the overlap of hardware resource units between the idle devices and the new device; Filtering the idle devices whose overlap is lower than a preset overlap threshold, and calculating the matching degree between each of the filtered idle devices and the computing task according to the task requirements of the computing task and the device characteristics of each of the filtered idle devices; The idle device with the highest matching degree is recorded as the target device.

[0008] In one embodiment, the step of migrating the computing task to the target device for execution includes: Acquire a source memory space of the new device and a target memory space of the target device in a unified virtual address space, wherein the memory space of each device in the computing task allocation system is mapped in the unified virtual address space; The execution state of the computing task is saved, and the computing task is transferred from the source memory space to the target memory space through direct memory access, and computing resources are allocated to the computing task after the transfer is completed, so as to restore the execution state of the computing task.

[0009] In one embodiment, the step of matching the computing task with the instruction set of the target device includes: Determine a hardware acceleration library of the target device according to the device characteristics of the target device, and optimize the acceleration path of each operator in the hardware acceleration library; The high-level operators in the computing task are mapped to the hardware acceleration instruction set of the target device according to the hardware acceleration library.

[0010] In addition, to achieve the above purpose, the present application also proposes a computing task allocation system, the computing task allocation system includes a device abstraction layer, the device abstraction layer defines a standardized interface for each device in the task allocation system, and the computing task allocation device includes: A new device access module is used to allocate computing tasks to the new device for execution through the device abstraction layer when detecting that a new device is connected to the computing task allocation system; A resource reallocation module, configured to select idle devices having resource loads lower than a preset resource load lower limit from among the devices in the computing task allocation system when the resource load of the new device exceeds a preset resource load upper limit; The task migration module is used to record the idle device whose hardware resource unit overlaps with each idle device higher than a preset overlap threshold and has the highest matching degree with the computing task as the target device, migrate the computing task to the target device for execution, and match the computing task with the instruction set of the target device.

[0011] In addition, to achieve the above objectives, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the computing task allocation method described above.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the computing task allocation method described above are implemented.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the computing task allocation method described above are implemented.

[0014] The present application provides a computing task allocation method, which is applied to a computing task allocation system. The computing task allocation system includes a device abstraction layer and various devices, and the device abstraction layer includes standardized interfaces of the various devices. The computing task allocation method includes: when a new device is detected to be connected to the computing task allocation system, the computing task is allocated to the new device for execution through the device abstraction layer; when the resource load of the new device exceeds a preset resource load upper limit, each idle device whose resource load is lower than a preset resource load lower limit is screened out from the various devices in the computing task allocation system; the idle device whose overlap of hardware resource units in each idle device is higher than a preset overlap threshold and has the highest match with the computing task is recorded as a target device, the computing task is migrated to the target device for execution, and the computing task and the instruction set of the target device are matched.

[0015] This application introduces a device abstraction layer to define a unified operation interface for each device, hide the underlying hardware differences, and enable the system to flexibly respond to computing devices of different brands and models. When a new device is connected, it can quickly identify and assign tasks to the new device, ensuring the effective use of computing resources while reducing manual intervention and improving the system's automation level. By real-time monitoring of device resource load and screening out idle devices, the system can ensure stable operation under high load conditions and avoid overall performance degradation caused by overloading of a single device. By comprehensively considering the overlap of hardware resources and the matching of tasks, it can ensure that computing tasks can be executed on the most suitable device, improving the efficiency and accuracy of task execution. At the same time, the matching of instruction sets further improves the execution performance of tasks. Compared with related solutions that are often based on simple load balancing strategies and lack comprehensive consideration of dynamic changes in resource load and device compatibility, this application provides a unified management method for different devices through a device abstraction layer, shielding the hardware implementation differences of the underlying devices, so that computing tasks can be switched freely between different devices. At the same time, dynamic adjustments are made according to resource load and resource overlap, making full use of the hardware resources of each device, reducing task waiting time and execution delays, avoiding excessive waste and idleness of device resources, achieving efficient use of resources, and improving resource utilization and task execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart of the first embodiment of the method for allocating computing tasks in this application is provided; Figure 2 A flowchart of the second embodiment of the method for allocating computing tasks in this application is provided; Figure 3 A flowchart of the third embodiment of the method for allocating computing tasks in this application is provided; Figure 4 This is a schematic diagram of the module structure of the computing task allocation device according to an embodiment of the present application; Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the computing task allocation method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The embodiment of the present application is applied to a computing task allocation system, and is applied to a computing task allocation system. The computing task allocation system includes a device abstraction layer and various devices, and the device abstraction layer includes standardized interfaces of the various devices. The main solution is: when a new device is detected to be connected to the computing task allocation system, the computing task is allocated to the new device for execution through the device abstraction layer; when the resource load of the new device exceeds a preset resource load upper limit, the idle devices whose resource load is lower than the preset resource load lower limit are screened out from the various devices in the computing task allocation system; the idle devices whose hardware resource unit overlaps higher than the preset overlap threshold and have the highest matching degree with the computing task are recorded as target devices, the computing task is migrated to the target device for execution, and the computing task and the instruction set of the target device are matched.

[0023] In this embodiment, for ease of description, the following description is made with the computing task allocation system as the execution subject.

[0024] In the current AI application development and deployment environment, developers are faced with the challenge of dealing with diverse heterogeneous hardware devices (including GPU, NPU, CPU, etc.) and numerous AI computing frameworks (such as TensorFlow, PyTorch, etc.). The diversity of these devices and frameworks significantly increases the complexity of the reasonable allocation of computing tasks. Existing technologies often require developers to write specific code for each device and framework. This practice not only increases the complexity of the development process, but also increases the economic cost of hardware expansion. Therefore, there is a lack of intelligence in the allocation of computing tasks in the current AI application development and deployment.

[0025] The present application provides a solution by introducing a device abstraction layer, defining a unified operation interface for each device, hiding the underlying hardware differences, and enabling the system to flexibly respond to computing devices of different brands and models. When a new device is connected, it automatically identifies and loads the driver for the new device, and dynamically adjusts and filters and migrates tasks according to the real-time resource situation of the device, ensuring that tasks can be executed on devices with sufficient resources, thereby improving the response speed and stability of the system. At the same time, it dynamically allocates resources according to task requirements, thereby improving resource utilization and task execution efficiency.

[0026] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a computing task allocation system, etc. The following takes the computing task allocation system as an example to illustrate this embodiment and the following embodiments.

[0027] Based on this, the present application embodiment provides a computing task allocation method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the computing task allocation method of the present application.

[0028] In this embodiment, the computing task allocation system is applied. The computing task allocation system includes a device abstraction layer and various devices. The device abstraction layer defines a standardized interface of each device. The computing task allocation method includes steps S01 to S03: Step S01, when a new device is detected to be connected to the computing task allocation system, the computing task is allocated to the new device for execution through the device abstraction layer; It should be noted that the device abstraction layer is an intermediate layer that defines the standardized interfaces of each device, so that upper-layer applications do not need to care about the specific implementation of the underlying devices, and can achieve unified management and control of devices. The standardized interface refers to the unified interface provided by the device abstraction layer for each device, which is used to achieve communication and data exchange between devices. The tasks of each device are managed through a unified standardized interface. For example, whether it is a GPU or an NPU, developers can initiate convolution calculation or matrix operation requests through the same interface, and the system automatically adapts these requests at the bottom layer.

[0029] In addition, it should be noted that when the system detects that a new device is connected, the new device is identified through the device abstraction layer. The new device refers to the hardware device that has just been connected to the computing task allocation system. Once the new device is identified, the system will assign the matching computing tasks to the new device for execution based on the current task queue and the performance characteristics of the device. The computing task refers to the computing work that needs to be performed on the hardware device.

[0030] It is understandable that since existing solutions usually require manual configuration of new device drivers and communication interfaces, step S01 is performed. Through the standardized interface defined by the device abstraction layer, the new device can be seamlessly connected to the computing task allocation system without the need for additional modification or configuration of the system. The system can automatically detect the access of the new device and immediately assign the computing task to the new device for execution, thereby improving the flexibility and scalability of the system.

[0031] Step S02, when the resource load of the new device exceeds the preset resource load upper limit, screen out idle devices whose resource load is lower than the preset resource load lower limit from the devices in the computing task allocation system; It should be noted that during the task execution process, the system will continuously monitor the resource load of each device. Resource load refers to the computing tasks currently being processed by the device and the amount of resources occupied. When the resource load of a device exceeds the preset resource load upper limit, the system triggers the resource load monitoring mechanism and screens out idle devices whose resource load is lower than the preset resource load lower limit from all devices. The preset resource load upper / lower limit refers to the resource load threshold set by the system, which is used to determine whether the device is overloaded or idle. Idle devices refer to devices whose resource load is lower than the preset resource load lower limit.

[0032] It is understandable that, since existing solutions usually adopt static or experience-based allocation strategies, step S02 is performed to monitor the resource load of each device in real time to ensure that timely measures are taken when the resource load exceeds the preset upper limit, and to screen idle devices with resource loads below the preset lower limit, thereby providing alternative targets for the migration of computing tasks, realizing dynamic monitoring and intelligent adjustment of device resource loads, and avoiding task execution failure or performance degradation due to resource overload.

[0033] Step S03, record the idle device whose hardware resource unit overlap is higher than the preset overlap threshold and has the highest matching degree with the computing task as the target device, migrate the computing task to the target device for execution, and match the computing task with the instruction set of the target device.

[0034] It should be noted that the hardware resource unit refers to the hardware resources on the device, such as CPU, GPU, NPU, etc., the overlap refers to the degree of similarity of hardware resources between different devices, and the preset overlap threshold refers to the threshold of hardware resource overlap set by the system, which is used to screen devices that match the target task. The target device refers to the device selected to perform the computing task after screening and matching. The instruction set refers to the set of instructions supported by the device, which is used to guide the device to perform specific computing tasks.

[0035] In addition, it should be noted that after screening out idle devices, the system needs to further evaluate the matching degree between these devices and the current computing tasks, and select the idle device with the highest matching degree as the target device based on the overlap of hardware resource units (such as processor type, memory size, etc.) and task requirements (such as computing intensive, memory intensive, etc.), migrate the computing tasks to the target devices for execution, and match the computing tasks with the instruction sets of the target devices to ensure that the tasks can be executed efficiently.

[0036] It is understandable that since existing solutions often rely on static rules or simple load balancing algorithms to allocate tasks, they are unable to adapt to the dynamic changes of devices and the diversity of tasks. Therefore, step S03 is performed. By comprehensively considering the overlap of hardware resource units and the matching degree of computing tasks, the system can select the target device that best matches the task. After the task is migrated, the system can automatically match the computing task with the instruction set of the target device to ensure that the task can be efficiently executed on the target device.

[0037] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 In step S01, the step of assigning the computing task to the new device for execution through the device abstraction layer includes steps S11 to S13: Step S11, obtaining device characteristics of the new device through a resource set of the new device, and detecting task requirements of each computing task in the computing task allocation system; It should be noted that the system obtains the device characteristics of the device through the resource set of the new device. The resource set refers to the sum of various hardware resource units possessed by the new device, including but not limited to CPU, GPU, memory, storage and other hardware resources, represented by device vectors, for example:

[0038] in, The hardware resource units on the device, such as computing cores and memory blocks, are extracted based on these resource sets and can reflect the key attributes of the device performance and applicable scenarios, such as computing power, memory size, storage speed, etc. For example, a device with rich GPU resources may have characteristics including high parallel computing power and large video memory. At the same time, the system also needs to detect the task requirements of each computing task in the computing task allocation system. Task requirements refer to the hardware and software conditions that each computing task needs to meet during execution, including but not limited to computing complexity, memory usage, and dependence on specific hardware accelerators. For example, some AI tasks may require high-performance GPUs to accelerate the training of deep learning models, while other tasks may focus more on CPU processing speed and memory size.

[0039] Step S12, matching the detected task requirements with the device features to obtain the matching degrees; It should be noted that the system matches the detected task requirements with the device characteristics to obtain the matching degree. The matching degree refers to the degree of matching between each computing task and the new device in terms of resource requirements and capabilities. The matching degree calculation formula is:

[0040] in, Indicates device characteristics. Indicates task requirements.

[0041] For example, if the task is mainly compute-intensive (such as convolution calculations, matrix multiplication), the system will give priority to devices with strong computing power and support for large-scale parallel computing (such as GPUs or NPUs). If the task has high requirements for memory bandwidth or requires large-scale data storage, the system will give priority to devices with high-bandwidth video memory (such as GPUs) or devices with high memory capacity (such as the CPUs of some high-performance servers). For tasks with low latency requirements (such as real-time video processing, real-time reasoning, etc.), the system will choose devices with faster response speeds, such as low-latency NPUs.

[0042] Step S13, the computing task with the highest matching degree is allocated to the new device through the device abstraction layer, and the hardware resource unit that meets the computing requirements of the computing task is driven by the corresponding driver module of the new device to execute the computing task.

[0043] It should be noted that the system assigns the most matching computing tasks to the new device through the device abstraction layer, and drives the hardware resource units that meet the computing requirements of the computing tasks through the driver module corresponding to the new device to perform the computing tasks. The driver module is software specifically used to control and manage specific hardware devices. It is responsible for translating the instructions of the upper-level application into signals that the hardware can understand, thereby driving the hardware device to perform the corresponding operations. For example, a GPU driver module can receive computing instructions issued by the upper-level application and convert them into parallel computing tasks that the GPU can execute.

[0044] For example, assuming that the current computing task is a convolutional neural network (CNN) reasoning task, the task requirement of this computing task is computationally intensive, mainly performing short-array operations and requiring high parallel computing capabilities. If there are two available devices A and B in the system, device A (GPU) has 1000 computing cores, supports high parallelism, has 16GB of video memory, a memory bandwidth of 512GB / s, and a floating-point computing capability of 30 TFLOPS, and device B (CPU) has 16 computing cores, strong single-core performance, but limited parallel computing capabilities, and a memory bandwidth of 50GB / S, then the system will determine that device A is most suitable for this task because of its powerful parallel computing capabilities and high floating-point computing capabilities, which can efficiently perform convolution computing tasks; assuming that the current computing task is a large-scale data processing task, the task requirement of this computing task is memory intensive, requiring high bandwidth and large memory capacity. If there are two available devices A and B in the system, device A (GPU) has 16GB of video memory, a memory bandwidth of 512GB / S, and device B (CPU) has 128GB RAM, memory bandwidth is 200GB / S, the system will determine that device B (CPU) is most suitable for this task, because it has a larger memory capacity and can process larger data sets; assuming that the current computing task is a real-time video inference task, the task requirements of this computing task are low latency, high throughput, and fast response. If there are two available devices A and B in the system, device A (GPU) has powerful computing power, but the response time is relatively long, which is suitable for batch processing tasks. Device B (NPU) is specially optimized for low-latency inference tasks and has a fast response speed. The system will determine that device B (NPU) is most suitable for real-time video inference tasks, because it optimizes low latency and fast response and can process video streams in real time.

[0045] In this embodiment, by extracting resource collection and device characteristics, the system can clearly understand the performance characteristics and scope of application of the new device, and by detecting task requirements, the system can clarify the specific requirements of each computing task, providing necessary data support for subsequent computing task allocation, which helps to achieve more intelligent task allocation. By calculating the matching degree, the system can accurately evaluate the execution effect of each computing task on the new device, thereby avoiding allocating unsuitable tasks to the new device, improving the utilization of computing resources and the efficiency of task execution. Through the collaborative work of the device abstraction layer and the driver module, the system can accurately allocate the computing tasks with the highest matching degree to the new device, and make full use of the hardware resources of the new device to perform computing tasks, which not only improves the execution efficiency of computing tasks, but also reduces the complexity and cost of developing and deploying AI applications.

[0046] In a feasible implementation manner, in step S01, before the step of assigning the computing task to the new device for execution through the device abstraction layer, steps A01 to A03 are also included: Step A01, monitoring whether there is a new device connected to the computing task allocation system; It should be noted that in the computing task allocation system, a device file subsystem monitoring module is deployed. This module is responsible for real-time monitoring of the insertion and removal events of devices in the system. When a new device is connected to the system, the monitoring module will immediately detect this event and trigger the corresponding processing flow. The monitoring module obtains and identifies the unique identifier of the new device (such as device ID, serial number, etc.) by reading the device file of the new device or using other identification mechanisms.

[0047] Step A02, when a new device is detected to be connected to the computing task allocation system, the device type of the new device is identified, and a driver module corresponding to the new device is loaded according to the device type, where the driver module is used to drive a hardware resource unit on the new device; It should be noted that when the monitoring module detects that a new device is connected, the system determines the device type of the new device through the device identification module (based on the device's hardware ID, model information or specific communication protocol). Once the device type is identified, the system searches for a driver module that matches the device type from the pre-configured driver library and loads it into the system. The driver module then communicates with the new device to drive the hardware resource unit on it.

[0048] In addition, it should be noted that the system will dynamically optimize the driver loading of new devices to ensure that the time for the device to load the corresponding driver is minimized. The optimization model for dynamic device loading is:

[0049] in, Indicates the device The time overhead of loading the corresponding device driver.

[0050] Step A03, identifying the hardware resource units on the new device, and obtaining the resource set of the new device.

[0051] It should be noted that after the driver module is loaded and successfully communicates with the new device, the system uses the resource identification module to identify the hardware resource units (such as CPU core, GPU, memory, storage, etc.) on the new device. The information of these resource units is collected and integrated into a resource set for subsequent computing task allocation.

[0052] In this embodiment, through the real-time monitoring mechanism, the system can instantly detect the access of new devices, providing a basis for subsequent device identification and driver loading, ensuring that the computing task allocation system can flexibly respond to dynamic changes in hardware devices, and improving the scalability and flexibility of the system. Through the device type identification and driver module dynamic loading mechanism, the system can automatically load the correct driver module for the new device, thereby driving the hardware resource unit on the new device, ensuring that the new device can immediately participate in task calculations, improving the utilization of computing resources and the overall performance of the system. By identifying the hardware resource units on the new device and obtaining its resource collection, the system can more accurately understand the computing power, memory layout and other hardware characteristics of the new device, providing a basis for realizing intelligent allocation of computing tasks, ensuring that tasks can run efficiently on the new device, and at the same time, improving the resource utilization and overall performance of the system.

[0053] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those of the first and second embodiments can be referred to the above description, and will not be described in detail later. Figure 3 In step S03, the step of recording the idle device whose hardware resource unit overlap degree among the idle devices is higher than the preset overlap degree threshold and has the highest matching degree with the computing task as the target device includes steps S21 to S23: Step S21, calculating the overlap of hardware resource units between each idle device and the new device; It should be noted that the hardware resource unit information of the new device and all currently idle devices is obtained, including but not limited to CPU model, GPU model, memory capacity, storage type and capacity, etc. Then, the system uses the resource overlap algorithm to compare the hardware resource units of each pair of idle devices and new devices and calculate the overlap between them. The overlap is usually expressed as a percentage, reflecting the similarity between the two in hardware resources. The overlap algorithm model is expressed as:

[0054] in, Indicates the resource collection corresponding to the new device. Indicates the resource set corresponding to the idle device.

[0055] Step S22, filtering idle devices whose overlap is lower than a preset overlap threshold, and calculating the matching degree between each filtered idle device and the computing task according to the task requirements of the computing task and the device characteristics of each filtered idle device; It should be noted that, based on step S21, the system will filter out all idle devices whose overlap is lower than the preset overlap threshold, and only retain devices with higher overlap as candidates. The preset overlap threshold is determined based on the system's requirements for resource matching accuracy and the needs of actual application scenarios. The system calculates the matching degree of each candidate device with the task based on the computing requirements of the computing task (such as computing volume, memory requirements, dependence on specific hardware, etc.) and the device characteristics of each idle device after filtering (such as CPU performance, GPU model, memory capacity, etc.). The calculation formula is:

[0056] in, is the device characteristics of the target device, To calculate the computational requirements of the task, the matching degree is also expressed as a numerical value, reflecting the degree of adaptation between the device and the task.

[0057] Step S23: Record the idle device with the highest matching degree as the target device.

[0058] It should be noted that, based on step S22, the system will determine the idle device with the highest matching degree as the target device according to the calculated matching degree, and this target device will be responsible for executing the current computing task.

[0059] In this embodiment, by calculating the degree of overlap, the system can more accurately evaluate the degree of match between idle devices and new devices in hardware resources, and provide more accurate information for subsequent task allocation. By identifying devices with high resource overlap, the system can more effectively utilize existing resources and avoid assigning tasks to devices with mismatched resources, thereby reducing resource waste. By calculating the degree of match, the system can intelligently assign tasks to the most suitable devices, thereby improving the efficiency and accuracy of task execution. By optimizing task allocation, the system can reduce the migration and waiting time of tasks between devices, thereby speeding up task execution. By selecting the device with the highest matching degree as the target device, the system can ensure that the task is executed on the optimal resources, thereby improving the efficiency and accuracy of task execution. Through intelligent task allocation, the system can avoid multiple tasks competing for the same resources at the same time, thereby reducing resource conflicts and waiting time.

[0060] In a feasible implementation manner, in step S03, the step of migrating the computing task to the target device for execution includes steps B01-B02: Step B01, obtaining a source memory space of the new device and a target memory space of the target device in a unified virtual address space, wherein the memory space of each device in the computing task allocation system is mapped in the unified virtual address space; It should be noted that the system uses memory mapping technology to convert physical memory space Mapped to a unified virtual address space , to ensure data sharing between different devices, the optimization goal of memory mapping is to minimize the memory copy time during data transmission:

[0061] in, To minimize memory copy time during data transfer, is the time to transmit data, The time required for memory mapping.

[0062] In addition, it should be noted that the system accesses a unified virtual address space, which is a logical address space that maps the memory space of all devices in the computing task allocation system. Through the unified virtual address space, the system can access and operate these memory spaces in a unified and consistent manner. According to the identification information of the new device and the target device, the system can find and obtain their source memory space (that is, the memory area where the new device currently stores the computing tasks) and target memory space (that is, the memory area where the target device plans to receive and process the computing tasks) in the unified virtual address space.

[0063] Step B02, save the execution status of the computing task, transfer the computing task from the source memory space to the target memory space through direct memory access, and allocate computing resources to the computing task after the transfer is completed, and restore the execution status of the computing task.

[0064] It should be noted that, based on step B01, the system saves the execution status of the current computing task, which includes the current progress of the task, the data used, and any context information related to the task execution. The computing task is transferred from the source memory space to the target memory space using the Direct Memory Access (DMA) technology. The DMA technology allows devices (such as the new device and the target device) to exchange data directly without the intervention of the CPU, thereby avoiding CPU intervention and reducing transmission delays:

[0065] in, is the data transmission time, is the DMA transfer time, The DMA technology is used to reduce the time required to pass through the CPU, which greatly improves the efficiency of data transmission. After the transmission is completed, the system will allocate appropriate computing resources (such as CPU, GPU, etc.) to the target device according to the needs of the computing task and the resources of the target device, and restore the execution status of the computing task, and continue to execute the task on the target device. The task migration model is:

[0066] in, For computing nodes, A collection of resources corresponding to the target device. To meet the computing needs of computing tasks, when tasks are migrated, computing nodes From the source device Rebind to the target device , ensuring that tasks can be performed efficiently on new devices.

[0067] In this implementation, a unified virtual address space is provided to provide a unified memory access interface, so that the system can access the memory space of any device without distinction, manage and utilize the memory space more effectively, and reduce the occurrence of memory fragmentation. Through direct memory access, the system can greatly reduce the delay of data transmission and improve the efficiency of data transmission. At the same time, the system can intelligently allocate computing resources to it according to the requirements of the computing task and the resource situation of the target device, thus solving the problem of unintelligent allocation of computing resources.

[0068] In a feasible implementation manner, in step S03, the step of matching the computing task with the instruction set of the target device includes steps B11 to B12: Step B11, determining the hardware acceleration library of the target device according to the device characteristics of the target device, and optimizing the acceleration path of each operator in the hardware acceleration library; It should be noted that the hardware acceleration library refers to a collection of algorithms and operations optimized for specific hardware devices (such as GPU, FPGA, ASIC, etc.). These algorithms and operations can utilize the parallel processing capabilities of the hardware to accelerate the computing tasks in the AI ​​model. The acceleration path means that in the hardware acceleration library, each operator (such as matrix multiplication, convolution operation, etc.) has one or more execution paths. These paths are optimized according to hardware characteristics and task requirements to achieve optimal performance.

[0069] In addition, it should be noted that the system collects detailed device characteristics of the target device, including but not limited to the device's processor type (such as CPU, GPU, NPU, etc.), memory size, processing speed, power consumption limit, etc. Based on the collected device characteristics, the system selects the hardware acceleration library that best matches the target device from the predefined hardware acceleration library set, and optimizes the acceleration path of each operator in the library, including adjusting the execution order of operators, reducing unnecessary memory access, and utilizing parallel processing capabilities, etc., to maximize the execution efficiency of computing tasks. The optimization model is:

[0070] in, is the hardware instruction set on the target device, The operator that needs to be executed for the task.

[0071] Step B12: Mapping high-level operators in the computing task to the hardware acceleration instruction set of the target device according to the hardware acceleration library.

[0072] It should be noted that high-level operators refer to computing tasks defined in AI applications, usually using high-level programming languages ​​​​(such as Python) and deep learning frameworks (such as TensorFlow, PyTorch). These tasks are composed of a series of high-level operators, such as addition, multiplication, activation functions, etc. The hardware acceleration instruction set is the underlying instruction set supported by the target device hardware. These instructions are directly executed by the hardware to implement various computing functions.

[0073] In addition, it should be noted that the high-level operators in the computing tasks (such as matrix multiplication, convolution operations, etc.) are mapped to the instruction sets supported by the hardware acceleration library of the target device. These instruction sets are instruction sets that can be directly understood and executed by the underlying hardware. The mapping process involves converting the semantics of the high-level operators into the specific implementation of the underlying hardware instructions, including determining the input and output of the operator, selecting appropriate hardware instructions, setting the parameters of the instruction, etc.

[0074] In addition, it should be noted that the system automatically replaces high-level operators in computing tasks with acceleration library operators of the target device through the device abstraction layer. For example, convolution operations will use a dedicated acceleration library on the NPU, and the cuDNN library will be used on the GPU. The system automatically replaces operators according to device characteristics to ensure that tasks can maintain efficient execution during the migration process.

[0075] In addition, it should be noted that the system will replace the computing operators according to the hardware acceleration library of the target device. For example, the convolution operation may call the cuDNN acceleration library on the GPU and call the dedicated NPU convolution acceleration library on the NPU, which ensures the optimal performance when executed on different devices.

[0076] In this implementation, by automatically determining the most suitable hardware acceleration library based on the device characteristics of the target device and further optimizing the acceleration path of each operator in the library, intelligent allocation of computing tasks is achieved, which not only improves the execution efficiency of AI applications, but also reduces the complexity of development and deployment. By automatically mapping high-level operators to the hardware acceleration instruction set of the target device, accurate execution of computing tasks and efficient use of hardware resources are achieved, which not only improves the execution efficiency of AI applications, but also reduces the complexity of development and deployment, allowing AI applications to adapt to different target devices more quickly.

[0077] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the computing task allocation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0078] This application also provides a computing task allocation system, please refer to Figure 4 The computing task allocation system includes a device abstraction layer and various devices. The device abstraction layer defines the standardized interface of each device. The computing task allocation device includes: A new device access module 10 is used to allocate computing tasks to the new device for execution through a device abstraction layer when a new device is detected to be connected to the computing task allocation system; The resource reallocation module 20 is used to select idle devices whose resource load is lower than the preset resource load lower limit from the devices in the computing task allocation system when the resource load of the new device exceeds the preset resource load upper limit; The task migration module 30 is used to record the idle device whose hardware resource unit overlap is higher than the preset overlap threshold and has the highest matching degree with the computing task as the target device, migrate the computing task to the target device for execution, and match the computing task with the instruction set of the target device.

[0079] Optionally, the new device access module 10 is further used for: Acquire device characteristics of the new device through a resource set of the new device, and detect task requirements of each computing task in the computing task allocation system; Matching each detected task requirement with the device characteristics to obtain each matching degree; The computing task with the highest matching degree is allocated to the new device through the device abstraction layer, and the hardware resource unit that meets the computing requirements of the computing task is driven by the corresponding driver module of the new device to execute the computing task.

[0080] Optionally, the new device access module 10 is further used for: Monitor whether there are new devices connected to the computing task allocation system; When a new device is detected to be connected to the computing task allocation system, the device type of the new device is identified, and a driver module corresponding to the new device is loaded according to the device type, where the driver module is used to drive a hardware resource unit on the new device; Identify the hardware resource units on the new device and obtain the resource set of the new device.

[0081] Optionally, the task migration module 30 is further used for: Calculate the overlap of hardware resource units between each idle device and the new device; Filter the idle devices whose overlap is lower than the preset overlap threshold, and calculate the matching degree between each filtered idle device and the computing task according to the task requirements of the computing task and the device characteristics of each filtered idle device; The idle device with the highest matching degree is recorded as the target device.

[0082] Optionally, the task migration module 30 is further used for: Acquire a source memory space of the new device and a target memory space of the target device in a unified virtual address space, wherein the memory space of each device in the computing task allocation system is mapped in the unified virtual address space; Save the execution status of the computing task, transfer the computing task from the source memory space to the target memory space through direct memory access, allocate computing resources to the computing task after the transfer is completed, and restore the execution status of the computing task.

[0083] Optionally, the task migration module 30 is further used for: Determine the hardware acceleration library of the target device according to the device characteristics of the target device, and optimize the acceleration path of each operator in the hardware acceleration library; The high-level operators in the computing task are mapped to the hardware acceleration instruction set of the target device according to the hardware acceleration library.

[0084] The computing task allocation device provided by the present application adopts the computing task allocation method in the above embodiment, which can solve the technical problem of unintelligent computing task allocation. Compared with the prior art, the beneficial effects of the computing task allocation device provided by the present application are the same as the beneficial effects of the computing task allocation method provided by the above embodiment, and the other technical features in the computing task allocation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0085] The present application provides an electronic device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the computing task allocation method in the above-mentioned embodiment one.

[0086] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, notebook computers, PADs (Portable Application Description: tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0087] like Figure 5 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0088] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0089] The electronic device provided by the present application adopts the computing task allocation method in the above embodiment to solve the technical problem of unintelligent computing task allocation. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the computing task allocation method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0090] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0091] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0092] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the computing task allocation method in the above-mentioned embodiment.

[0093] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0094] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0095] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the computing task allocation device is applied to the computing task allocation system. The computing task allocation system includes a device abstraction layer, which can load the driver module corresponding to the new device when detecting that a new device is connected to the computing task allocation system, communicate with the new device through the device abstraction layer, and allocate the current computing task to the new device for execution; when the resources of the new device do not meet the resource requirements of the computing task, select a target device that meets the resource requirements of the computing task from the various devices in the computing task allocation system; migrate the computing task to the target device for execution, and map the high-level operators of the computing task to the optimal instruction set supported by the target device.

[0096] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0099] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned computing task allocation method, and can solve the technical problem of unintelligent computing task allocation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the computing task allocation method provided in the above-mentioned embodiment, and will not be repeated here.

[0100] The present application also provides a computer program product, including a computer program, which implements the steps of the computing task allocation method as described above when executed by a processor.

[0101] The computer program product provided by this application can solve the technical problem of unintelligent computing task allocation. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the computing task allocation method provided by the above embodiment, which will not be repeated here.

[0102] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A computing task allocation method, characterized in that: Applied to a computing task allocation system, the computing task allocation system includes a device abstraction layer and various devices, the device abstraction layer includes standardized interfaces of the various devices, and the computing task allocation method includes: When a new device is detected to be connected to the computing task allocation system, the computing task is allocated to the new device for execution through the device abstraction layer; In the case where the resource load of the new device exceeds a preset resource load upper limit, screening out idle devices whose resource load is lower than a preset resource load lower limit from the devices in the computing task allocation system; The idle device whose hardware resource unit overlaps with each other higher than a preset overlap threshold and whose matching degree with the computing task is the highest is recorded as the target device, the computing task is migrated to the target device for execution, and the computing task is matched with the instruction set of the target device.

2. The computing task allocation method according to claim 1, characterized in that: The step of allocating the computing task to the new device for execution through the device abstraction layer comprises: Acquire device characteristics of the new device through a resource set of the new device, and detect task requirements of each computing task in the computing task allocation system; Matching each detected task requirement with the device feature to obtain each matching degree; The computing task with the highest matching degree is allocated to the new device through the device abstraction layer, and the hardware resource unit that meets the computing requirements of the computing task is driven by the corresponding driver module of the new device to execute the computing task.

3. The computing task allocation method according to claim 2, characterized in that: Before the step of allocating the computing task to the new device for execution through the device abstraction layer, the step further includes: Monitoring whether new devices are connected to the computing task allocation system; When a new device is detected to be connected to the computing task allocation system, the device type of the new device is identified, and a driver module corresponding to the new device is loaded according to the device type, wherein the driver module is used to drive a hardware resource unit on the new device; A hardware resource unit on the new device is identified to obtain a resource set of the new device.

4. The computing task allocation method according to claim 1, characterized in that: The step of recording the idle device having the highest matching degree with the computing task and the overlap degree of the hardware resource units in the idle devices higher than the preset overlap degree threshold as the target device comprises: Calculating the overlap of hardware resource units between the idle devices and the new device; Filtering the idle devices whose overlap is lower than a preset overlap threshold, and calculating the matching degree between each of the filtered idle devices and the computing task according to the task requirements of the computing task and the device characteristics of each of the filtered idle devices; The idle device with the highest matching degree is recorded as the target device.

5. The computing task allocation method according to claim 1, characterized in that: The step of migrating the computing task to the target device for execution comprises: Acquire a source memory space of the new device and a target memory space of the target device in a unified virtual address space, wherein the memory space of each device in the computing task allocation system is mapped in the unified virtual address space; The execution state of the computing task is saved, and the computing task is transferred from the source memory space to the target memory space through direct memory access, and computing resources are allocated to the computing task after the transfer is completed, so as to restore the execution state of the computing task.

6. The computing task allocation method according to claim 1, characterized in that: The step of matching the computing task with the instruction set of the target device comprises: Determine a hardware acceleration library of the target device according to the device characteristics of the target device, and optimize the acceleration path of each operator in the hardware acceleration library; The high-level operators in the computing task are mapped to the hardware acceleration instruction set of the target device according to the hardware acceleration library.

7. A computing task allocation system, characterized in that: The computing task allocation system includes a device abstraction layer, which defines a standardized interface for each device in the task allocation system. The computing task allocation device includes: A new device access module is used to allocate computing tasks to the new device for execution through the device abstraction layer when detecting that a new device is connected to the computing task allocation system; A resource reallocation module, configured to select idle devices having resource loads lower than a preset resource load lower limit from among the devices in the computing task allocation system when the resource load of the new device exceeds a preset resource load upper limit; The task migration module is used to record the idle device whose hardware resource unit overlaps with each idle device higher than a preset overlap threshold and has the highest matching degree with the computing task as the target device, migrate the computing task to the target device for execution, and match the computing task with the instruction set of the target device.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the computing task allocation method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the computing task allocation method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the computing task allocation method according to any one of claims 1 to 6 are implemented.

Images