Control method of heterogeneous AI equipment management system and related equipment

Abstract

According to the control method of the heterogeneous AI device management system and the related device, the heterogeneous AI device management system comprises a plurality of AI devices, driving information in device driving libraries between at least two AI devices is different, and the method comprises the steps that firstly, in response to an execution request of an AI processing task, the driving information in the device driving libraries between at least two AI devices is different; target processing equipment is determined from the multiple AI equipment based on the AI processing task, an equipment driver library of the target processing equipment is obtained through the adaptive hardware interface, and driving information in the equipment driver library comprises adaptive driving information of the target processing equipment; then, a control decision is generated based on a heterogeneous device resource table of the heterogeneous AI device management system and task demand information of the AI processing task, the target processing device is controlled to execute the AI processing task based on the control decision and the adaptive driving information, and the heterogeneous device resource table is obtained based on the adaptive driving information; therefore, unified control and resource scheduling of different types of AI equipment are realized, and the resource utilization rate and the management efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a control method and related equipment for a heterogeneous AI device management system. Background Art

[0002] In recent years, with the rise of large model technology, many equipment suppliers have launched their own artificial intelligence computing devices to meet the growing and diverse needs of the market. At the same time, more and more clusters have begun to try to manage AI devices from different equipment suppliers at the same time, so as to cope with more complex and diverse artificial intelligence computing tasks. In related technologies, it is usually necessary to rely on native drivers and supporting management tools (such as nvidia-smi) provided by each equipment supplier to query device status, utilization, memory and other driver information and perform basic operations. Therefore, it is necessary to set up the corresponding device supplier interface for each heterogeneous AI device in the management system to connect these heterogeneous AI devices, and every time these heterogeneous AI devices need to be controlled, it is necessary to first determine the corresponding interface to obtain and manipulate the corresponding information of the heterogeneous AI device, resulting in low resource utilization and management efficiency of the entire heterogeneous AI device management system. Summary of the Invention

[0003] The embodiments of the present application provide a control method and related devices for a heterogeneous AI device management system, which can improve resource utilization and management efficiency when managing multiple heterogeneous AI devices in the heterogeneous AI device management system.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present application provides a control method for a heterogeneous AI device management system, wherein the heterogeneous AI device management system includes multiple AI devices, and at least two of the AI devices have different driver information in their device driver libraries. The method includes:

[0005] In response to a request to execute the AI processing task, determining a target processing device from the plurality of AI devices based on the AI processing task, and obtaining a device driver library for the target processing device through an adapted hardware interface, wherein driver information in the device driver library includes adapted driver information for the target processing device;

[0006] Based on the heterogeneous device resource table of the heterogeneous AI device management system and the task requirement information of the AI processing task, a control decision is generated, and based on the control decision and the adaptation drive information, the target processing device is controlled to perform the AI processing task, and the heterogeneous device resource table is obtained based on the adaptation drive information.

[0007] In some embodiments, the method further comprises:

[0008] In response to an access request from a new AI device, obtaining, through the adaptation hardware interface, an adaptation driver library name, adaptation driver library parameters, and adaptation driver performance indicators of the new AI device, and generating adaptation driver information for the new AI device based on the adaptation driver library name, the adaptation driver library parameters, and the adaptation driver performance indicators;

[0009] Based on the name of the adapter driver library, obtaining the adapter driver library of the new AI device, and initializing the new AI device based on the adapter driver library of the new AI device;

[0010] After the initialization process of the new AI device is completed, the adaptation driver information of the new AI device is added to the heterogeneous device resource table.

[0011] In some embodiments, the target processing device includes a primary processing device and an auxiliary processing device, and determining the target processing device from the plurality of AI devices based on the AI processing task includes:

[0012] Based on the core processing task of the AI processing task and the corresponding core computing resource requirements, select at least one of the AI devices from the multiple AI devices as a first candidate processing device using the heterogeneous device resource table;

[0013] Based on the edge processing task of the AI processing task and the corresponding edge computing resource requirements, select at least one AI device from the multiple AI devices as a second candidate processing device using the heterogeneous device resource table;

[0014] utilizing data compatibility between the AI devices and generating at least one candidate device group based on the first candidate processing device and the second candidate processing device, each candidate device group including at least one first candidate processing device and at least one second candidate processing device;

[0015] Real-time device group performance data of each candidate device group is obtained through the adaptation hardware interface, a target device group is selected from the multiple candidate device groups based on the device group performance data, and the first alternative processing device in the target device group is used as the main processing device, and the second alternative processing device in the target device group is used as the auxiliary processing device.

[0016] In some embodiments, utilizing the data compatibility between the AI devices to generate at least one candidate device group based on the first candidate processing device and the second candidate processing device includes:

[0017] Using each of the first candidate processing devices as a first compatibility verification device one by one, determining first compatibility verification data output by the first compatibility verification device after processing the core processing task, and matching the first compatibility verification data with input data of each of the second candidate processing devices to obtain a first compatibility verification result between each of the first candidate processing devices and each of the second candidate processing devices;

[0018] Using each of the second candidate processing devices as a second compatibility verification device one by one, determining second compatibility verification data output by the second compatibility verification device after processing the edge processing task, and matching the second compatibility verification data with input data of each of the first candidate processing devices to obtain a second compatibility verification result between each of the first candidate processing devices and each of the second candidate processing devices;

[0019] A device group whose first compatibility test results and second compatibility test results both indicate data compatibility is selected one by one from a plurality of the first candidate processing devices and a plurality of the second candidate processing devices as the candidate device group, wherein the device group includes one first candidate processing device and one second candidate processing device.

[0020] In some embodiments, selecting a target device group from the plurality of candidate device groups based on the device group performance data includes:

[0021] Calculate, based on the device group performance data, the execution time, data transmission time, and heterogeneous device collaborative processing time corresponding to each candidate device group executing the AI processing task;

[0022] Obtaining, by weighted accumulation based on the execution time, the data transmission time, and the heterogeneous device collaborative processing time, a processing verification time corresponding to execution of the AI processing task by each candidate device group;

[0023] The candidate device group with the shortest processing verification time is selected from the plurality of candidate device groups as the target device group.

[0024] In some embodiments, the control decision includes a primary control decision, an auxiliary control decision, and a collaborative control decision. Generating a control decision based on the heterogeneous device resource table of the heterogeneous AI device management system and the task requirement information of the AI processing task, and controlling the target processing device to perform the AI processing task based on the control decision includes:

[0025] updating the heterogeneous device resource table based on the device group performance data, and acquiring the main device performance data of the main processing device and the auxiliary device performance data of the auxiliary processing device from the updated heterogeneous device resource table;

[0026] generating the primary control decision of the primary processing device based on the primary device performance data and the core processing task, and generating the secondary control decision of the secondary processing device based on the secondary device performance data and the edge processing task;

[0027] generating the collaborative control decision between the primary processing device and the auxiliary processing device based on a data transmission relationship between the core processing task and the edge processing task;

[0028] Based on the main control decision, the auxiliary control decision and the collaborative control decision, the main processing device and the auxiliary processing device are controlled through the adapted hardware interface to collaboratively process the core processing task and the edge processing task.

[0029] In some embodiments, generating the collaborative control decision between the primary processing device and the auxiliary processing device based on the data transmission relationship between the core processing task and the edge processing task includes:

[0030] Determining data interaction conditions and interaction data information based on the core processing task and the edge processing task, wherein the data transmission relationship mainly consists of the data interaction conditions and the interaction data information;

[0031] generating synchronous clock information based on a primary device clock of the primary processing device and a secondary device clock of the secondary processing device;

[0032] generating protocol conversion information based on a primary device processing protocol of the primary processing device and an auxiliary device processing protocol of the auxiliary processing device;

[0033] The collaborative control decision is generated based on the data interaction condition, the interaction data information, the synchronization clock information, and the protocol conversion information.

[0034] To achieve the above objectives, a second aspect of an embodiment of the present application provides a heterogeneous AI device management system, wherein the heterogeneous AI device management system is configured to execute the control method for a heterogeneous AI device management system as described in the first aspect. The system includes:

[0035] The AI cluster management module, heterogeneous AI device management module, and heterogeneous AI device unified interface adaptation module are connected in sequence;

[0036] The heterogeneous AI device unified interface adaptation module is used to provide the adaptation hardware interface for each AI device;

[0037] The heterogeneous AI device management module is configured to store the heterogeneous device resource table, wherein the heterogeneous device resource table stores device resources and device command line resources of the plurality of AI devices;

[0038] The AI cluster management module is provided with management software for the multiple AI devices, and the AI cluster management module is configured to call the adaptation hardware interface through the management software to control at least one of the AI devices.

[0039] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the control method of the heterogeneous AI device management system as described in the first aspect.

[0040] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the control method of the heterogeneous AI device management system described in the first aspect above.

[0041] The present invention provides a control method and related devices for a heterogeneous AI device management system, wherein the heterogeneous AI device management system includes multiple AI devices, and at least two AI devices have different driver information in their device driver libraries. The method comprises: first, in response to an execution request for an AI processing task, determining a target processing device from multiple AI devices based on the AI processing task, obtaining the device driver library of the target processing device through an adaptive hardware interface, wherein the driver information in the device driver library includes adaptive driver information of the target processing device; then, generating a control decision based on a heterogeneous device resource table of the heterogeneous AI device management system and task requirement information of the AI processing task, and controlling the target processing device to execute the AI processing task based on the control decision and the adaptive driver information, wherein the heterogeneous device resource table is obtained based on the adaptive driver information. The present invention shields the differences in AI device driver libraries of different manufacturers through an adaptive hardware interface, eliminating the need to set up a specific vendor interface for each device. Instead, the driver library and adaptive driver information of the target device are uniformly obtained through the adaptive hardware interface, and a control decision is generated based on a unified heterogeneous device resource table, thereby achieving unified control and resource scheduling of different types of AI devices, improving resource utilization and management efficiency, simplifying management processes, and better handling complex and diverse artificial intelligence computing tasks.

[0042] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a structural diagram of a heterogeneous AI device management system provided by one embodiment of the present application.

[0044] Figure 2 This is a flowchart of a control method for a heterogeneous AI device management system provided by another embodiment of the present application.

[0045] Figure 3 yes Figure 2 Flowchart of step 201 in FIG.

[0046] Figure 4 yes Figure 3 Flowchart of step 303 in FIG.

[0047] Figure 5 yes Figure 3 Flowchart of step 304 in FIG.

[0048] Figure 6 yes Figure 2 Flowchart of step 202 in FIG.

[0049] Figure 7 yes Figure 6 Flowchart of step 603 in FIG.

[0050] Figure 8 This is a flowchart of a new AI device access provided by another embodiment of the present application.

[0051] Figure 9 This is a schematic flowchart of initialization and resource control of a new AI device provided by another embodiment of the present application.

[0052] Figure 10 This is a schematic diagram of a performance indicator provided by another embodiment of the present application.

[0053] Figure 11 This is a schematic diagram of part of the information in a heterogeneous device resource table provided in another embodiment of the present application.

[0054] Figure 12 This is a schematic diagram of an environmental variable provided in another embodiment of the present application.

[0055] Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0059] In recent years, with the rise of large model technology, many equipment suppliers have launched their own artificial intelligence computing devices to meet the growing and diverse needs of the market. At the same time, more and more clusters have begun to try to manage AI devices from different equipment suppliers at the same time, so as to cope with more complex and diverse artificial intelligence computing tasks. In related technologies, it is usually necessary to rely on native drivers and supporting management tools (such as nvidia-smi) provided by each equipment supplier to query device status, utilization, memory and other driver information and perform basic operations. Therefore, it is necessary to set up the corresponding device supplier interface for each heterogeneous AI device in the management system to connect these heterogeneous AI devices, and every time these heterogeneous AI devices need to be controlled, it is necessary to first determine the corresponding interface to obtain and manipulate the corresponding information of the heterogeneous AI device, resulting in low resource utilization and management efficiency of the entire heterogeneous AI device management system.

[0060] In order to improve resource utilization and management efficiency when managing multiple heterogeneous AI devices in a heterogeneous AI device management system, the embodiment of the present application shields the differences in AI device driver libraries of different manufacturers through the adaptation hardware interface. There is no need to set up a specific vendor interface for each device. Instead, the driver library and adaptation driver information of the target device are uniformly obtained through the adaptation hardware interface, and control decisions are generated based on a unified heterogeneous device resource table, thereby realizing unified control and resource scheduling of different types of AI devices, improving resource utilization and management efficiency, simplifying the management process, and being able to better cope with complex and diverse artificial intelligence computing tasks.

[0061] The following first describes a heterogeneous AI device management system provided by an embodiment of the present application. Figure 1 , is a structural diagram of a heterogeneous AI device management system provided by an embodiment of the present application. Figure 1Figure 1 shows the layered architecture of a heterogeneous AI device management system. From top to bottom, it consists of three layers: the AI cluster management layer, the heterogeneous AI device management layer, and the unified interface adaptation layer for heterogeneous AI devices. The top-level AI cluster management layer includes common cluster management tools such as Kubernetes, Docker, and Prometheus, responsible for overall resource scheduling and monitoring. The middle-level heterogeneous AI device management layer undertakes core management functions, including resource allocation, discovery, and monitoring for heterogeneous devices, as well as providing command-line tools and container plugin support. This layer calls underlying interfaces downward and provides services upward to the cluster management layer. The bottom-level unified interface adaptation layer for heterogeneous AI devices defines a unified interface. Through specific device adapters (such as NVIDIA, Ascend, and Cambricon shown in the figure) and corresponding driver libraries, it shields the hardware differences between AI devices from different manufacturers, enabling abstraction and unified access to the underlying hardware. This heterogeneous AI device management system integrates multiple AI devices with different driver information in their device driver libraries (such as NVIDIA, Ascend, and Cambricon).

[0062] Based on the heterogeneous AI device management system described above, the control method and related devices of the heterogeneous AI device management system provided by the embodiment of the present application will be further described below. First, the control method of the heterogeneous AI device management system is described. Figure 2 , which is an optional flowchart of the control method of the heterogeneous AI device management system provided in an embodiment of the present application, Figure 2 The method may include but is not limited to steps 201 to 202. It is also understood that this embodiment is for Figure 2 The order of steps 201 to 202 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs. The control method of the heterogeneous AI device management system provided in the embodiment of the present application can be applied to the heterogeneous AI device management system.

[0063] Step 201: In response to an execution request of an AI processing task, a target processing device is determined from a plurality of AI devices based on the AI processing task, and a device driver library of the target processing device is obtained through an adapted hardware interface.

[0064] Step 201 is described in detail below.

[0065] In some embodiments, when the heterogeneous AI device management system receives an execution request for an AI processing task, such as an AI processing task that requires image recognition, which requires a large amount of GPU computing power. After receiving the task request, the system first analyzes the characteristics and resource requirements of the image recognition task (such as the required calculation type, accuracy, memory size, etc.). Then, the system queries the heterogeneous device resource table, which stores information about all available AI devices, including device type, manufacturer, computing power, available resources, etc. This information is pre-acquired and stored through the hardware adaptation interface. Based on the task requirements and device resource information, the system determines the target processing device that is most suitable for executing the task, such as selecting an idle NVIDIA GPU with sufficient video memory. Finally, the system obtains the device driver library corresponding to the NVIDIA GPU through the hardware adaptation interface, such as calling the unified GetDeviceDriver function, which contains the GPU's adaptation driver information, such as various functions and parameters for controlling the GPU. In this way, the system completes the selection of the appropriate AI device based on the task requirements and prepares its corresponding driver library, preparing for subsequent control and task execution.

[0066] The following further describes how to select a suitable target processing device from multiple AI devices to improve the efficiency of processing the AI processing task.

[0067] Reference Figure 3 , determining a target processing device from multiple AI devices based on an AI processing task, includes the following steps 301 to 304.

[0068] Step 301: Based on the core processing task of the AI processing task and the corresponding core computing resource requirements, at least one AI device is selected from multiple AI devices using a heterogeneous device resource table as a first candidate processing device.

[0069] Step 302: Based on the edge processing task of the AI processing task and the corresponding edge computing required resources, use the heterogeneous device resource table to select at least one AI device from multiple AI devices as the second candidate processing device.

[0070] Step 303: Utilizing the data compatibility between AI devices, at least one candidate device group is generated based on the first candidate processing device and the second candidate processing device.

[0071] Steps 301 to 303 are described in detail below.

[0072] In some embodiments, to improve the processing efficiency of AI processing tasks, this solution decomposes complex AI processing tasks into core processing tasks and edge processing tasks. Assuming the AI processing task is a complex real-time video stream analysis AI processing task, this task can be decomposed into two main parts: 1) Core processing task: Running a large-scale object detection and recognition model (e.g., a high-precision YOLOv7 or Transformer-based model) on video frames requires strong parallel computing power and a large amount of video memory, which are the core computing resources required for processing the core processing task; 2) Edge processing task: Decoding, frame extraction, and image preprocessing (e.g., scaling and normalization) of the input video stream require certain video processing power and medium computing resources, which are the edge computing resources required for processing the edge processing task. The heterogeneous AI device management system includes a variety of heterogeneous AI devices, such as the NVIDIA A100 GPU (with strong computing power and large video memory), the NVIDIA T4 GPU (with good video decoding capabilities and medium inference computing power), and some dedicated AI accelerator cards (such as the Huawei Ascend 310, which excels at specific inference tasks).

[0073] After responding to the processing requirements of the AI processing task, the heterogeneous AI device management system will query a pre-built heterogeneous device resource table, which records information such as the type, model, current available status, remaining video memory, and core computing power indicators (such as TFLOPS) of each device.

[0074] Then, based on the core computing requirements of high computing power and large video memory for core processing tasks, the system will filter the heterogeneous device resource table. For example, it finds that only the A100 GPU can meet the computing power (>XTFLOPS) and video memory (>YGB) requirements required by the core model. Therefore, the system selects the currently available A100 GPUs (assuming there are two with device IDs GPU-001 and GPU-002) as the first candidate processing device set {GPU-001, GPU-002}.

[0075] Furthermore, based on the edge computing resources required for edge processing tasks (video decoding and preprocessing) (requiring a hardware video decoding unit and moderate computing power), the heterogeneous device resource table shows that the NVIDIA T4 GPU (assuming the device ID is GPU-101) has a dedicated decoding engine and sufficient computing power, and the Ascend 310 (assuming the device ID is NPU-201), despite its moderate computing power, can also handle this edge computing task. Therefore, the system selects the most compatible T4 GPU (GPU-101) and Ascend 310 (NPU-201) as the second candidate processing device set, forming {GPU-101, NPU-201}.

[0076] Afterwards, the system needs to consider the data compatibility between heterogeneous AI devices, and thus select at least one AI device from the first candidate processing device and at least one AI device from the second candidate processing device in turn to obtain multiple candidate device groups, as described below.

[0077] Reference Figure 4 , utilizing the data compatibility between AI devices, based on the first candidate processing device and the second candidate processing device, generating at least one candidate device group, including the following steps 401 to 403.

[0078] Step 401: Take each first alternative processing device as the first compatibility verification device one by one, determine the first compatibility verification data output after the first compatibility verification device processes the core processing task, and match the first compatibility verification data with the input data of each second alternative processing device to obtain the first compatibility verification result between each first alternative processing device and each second alternative processing device.

[0079] Step 402: Take each second alternative processing device as a second compatibility verification device one by one, determine the second compatibility verification data output by the second compatibility verification device after processing the edge processing task, and match the second compatibility verification data with the input data of each first alternative processing device to obtain the second compatibility verification result between each first alternative processing device and each second alternative processing device.

[0080] Step 403: Select one by one from the plurality of first candidate processing devices and the plurality of second candidate processing devices a device group for which both the first compatibility check result and the second compatibility check result indicate data compatibility as a candidate device group.

[0081] Steps 401 to 403 are described in detail below.

[0082] In some embodiments, after determining multiple first candidate processing devices and multiple second candidate processing devices, the system first uses each first candidate processing device as a first compatibility verification device to determine the data output after executing the core processing task as the first compatibility verification data. Then, the system matches this first compatibility verification data with the data format of the input data of each second candidate processing device to determine the compatibility between the first compatibility verification data and each second candidate processing device. Through this process, the system can obtain the first compatibility verification result between each first candidate processing device and each second candidate processing device, ensuring that the output of the core processing task can be effectively received and processed by the subsequently selected edge processing device.

[0083] At the same time, the system uses each second candidate processing device as a second compatibility verification device to determine the data output after executing the edge processing task as the second compatibility verification data. The system then matches this second compatibility verification data with the data format of the input data of each first candidate processing device, obtaining the second compatibility verification results between each first candidate processing device and each second candidate processing device. This process ensures that the output of the edge processing task can be effectively received by the subsequently selected core processing device, further verifying the data compatibility between the devices.

[0084] Finally, the system selects, from the multiple first candidate processing devices and multiple second candidate processing devices, those device groups whose first and second compatibility test results indicate data compatibility. These device groups serve as candidate device groups, each of which includes one first candidate processing device and one second candidate processing device. Through this screening process, the system ensures that the selected device groups can work together efficiently when performing AI processing tasks, thereby improving overall computing performance and task execution efficiency.

[0085] Through the above steps 401 to 403, by respectively checking the compatibility of the output data of the first alternative processing device and the input data of the second alternative processing device, and the output data of the second alternative processing device and the input data of the first alternative processing device, and finally selecting a bidirectionally compatible device combination, it can be effectively guaranteed that the data interaction between the core processing task and the edge processing task is unimpeded, thereby improving the efficiency and reliability of the entire AI processing task and avoiding unnecessary calculation and data conversion overhead.

[0086] Step 304: Obtain real-time device group performance data of each candidate device group through the adapted hardware interface, select a target device group from multiple candidate device groups based on the device group performance data, and use the first alternative processing device in the target device group as the main processing device and the second alternative processing device in the target device group as the auxiliary processing device.

[0087] Step 304 is described in detail below.

[0088] In some embodiments, after obtaining multiple candidate device groups, the system obtains the performance data of each candidate device group in real time through the adapted hardware interface to evaluate its capabilities and efficiency in performing specific AI processing tasks. Based on these performance data, the system can intelligently select the optimal target device group from multiple candidate device groups. Then, the first candidate processing device in the target device group is determined as the main processing device, responsible for performing core computing tasks, while the second candidate processing device in the target device group is designated as the auxiliary processing device, responsible for supporting and optimizing the work of the main processing device. Through this process, the system not only ensures the efficiency and stability of task execution, but also improves the overall resource utilization, thereby better meeting complex AI computing needs.

[0089] The following further describes how to intelligently select the optimal target device group from multiple candidate device groups.

[0090] Reference Figure 5 , selecting a target device group from multiple candidate device groups based on device group performance data, includes the following steps 501 to 503.

[0091] Step 501: Calculate the execution time, data transmission time, and heterogeneous device collaborative processing time corresponding to each candidate device group performing the AI processing task based on the device group performance data.

[0092] Step 502: Based on the execution time, data transmission time, and heterogeneous device collaborative processing time, weighted accumulation is performed to obtain the processing verification time corresponding to each candidate device group executing the AI processing task.

[0093] Step 503: Select the candidate device group with the shortest processing inspection time from the multiple candidate device groups as the target device group.

[0094] Steps 501 to 503 are described in detail below.

[0095] In some embodiments, after obtaining multiple candidate device groups, the system first calculates the time cost (including execution time, data transmission time, and heterogeneous device collaborative processing time) required to execute the AI processing task for each candidate device group based on the device group performance data. That is, based on the demand for computing resources of AI processing tasks (such as the computing amount of core tasks, the computing amount of edge tasks, and the amount of data that needs to be transmitted) and the device group performance data of each alternative device group obtained from the adaptive hardware interface (such as the current computing unit utilization, available video memory / memory, PCIe / interconnect bus bandwidth occupancy), considering the impact of the current load on the execution speed, the main execution time corresponding to the first alternative processing device in each alternative device group performing the core processing task and the auxiliary execution time corresponding to the second alternative processing device performing the edge processing task are estimated and calculated, and the execution time is obtained based on the maximum value between the main execution time and the auxiliary execution time; at the same time, considering the availability of the current bus bandwidth, the data transmission time required to transmit necessary data between the first alternative processing device and the second alternative processing device in each alternative device group is estimated; and based on historical data or the inherent characteristics of the device pair, the heterogeneous device collaborative processing time that may be generated by the collaborative work (such as startup, synchronization, data format conversion, etc.) of the first alternative processing device and the second alternative processing device in each alternative device group is estimated.

[0096] Next, the system calculates a comprehensive processing inspection time for each candidate device group. That is, a weighted accumulation method is used. The system assigns weights according to the preset strategy (or according to the characteristics of the current AI task, such as whether it is compute-intensive or I / O-intensive): W_exec (execution time weight), W_transfer (data transmission time weight), W_coord (co-processing time weight). For example, if the task is very sensitive to delay, the weights of W_exec and W_transfer may be relatively high. Based on this, the processing inspection time for each candidate device group is T = (W_exec*execution time) + (W_transfer*data transmission time) + (W_coord*co-processing time). Similarly, the processing inspection time of candidate group B is calculated.

[0097] Finally, the system compares the calculated processing verification time for each candidate device group and selects the candidate device group with the lowest processing verification time as the final target device group. Through this approach based on refined time prediction and weighted evaluation, the system can more accurately select the device combination expected to complete the specific AI processing task the fastest under the current state, thereby achieving better performance and resource utilization.

[0098] Through the above steps 301 to 304, and steps 501 to 503, based on the core and edge computing requirements of the task and the static resource table, the candidate processing equipment with basic capabilities is preliminarily screened out, and the feasibility of collaborative work between devices is ensured through data compatibility checks. The real-time performance data of the candidate device group is further obtained by using the adaptive hardware interface, and based on this, refined time prediction (including execution, transmission, and collaboration time) and weighted evaluation are performed, so that the target device group with the shortest expected processing time and the highest efficiency under the current cluster load and network conditions can be dynamically and accurately selected. This multi-dimensional and dynamic decision-making process significantly improves the execution performance and overall resource utilization of AI tasks, and enhances the system's adaptability to complex heterogeneous environments and the level of intelligent scheduling.

[0099] Step 202: Generate a control decision based on the heterogeneous device resource table of the heterogeneous AI device management system and the task requirement information of the AI processing task, and control the target processing device to perform the AI processing task based on the control decision and the adaptive drive information.

[0100] Step 202 is described in detail below.

[0101] In some embodiments, after determining the target processing device and obtaining its driver library, the system first combines the heterogeneous device resource table (including information such as the available resources and performance indicators of the target device) and the task requirement information of the AI processing task (such as the type of calculation, required resources, accuracy requirements, etc.) to generate a control decision. The control decision contains specific instructions on how to configure and control the target device, such as how many computing units to allocate, what frequency to set, which computing mode to use, etc. For simple tasks, the control decision may only be a simple resource allocation; for complex, multi-device collaborative tasks, the control decision may include collaborative control strategies for multiple devices. Then, based on the generated control decision and the adapted driver information, the system sends a control instruction to the target device through the adapted hardware interface to start task execution. For example, the system can call a function in the driver library, set the GPU's computing mode, and load the task data into the GPU memory, and finally start the GPU to perform the image recognition task.

[0102] The following further describes how to generate control decisions corresponding to the main processing device and auxiliary processing device performing AI processing tasks.

[0103] Reference Figure 6 , based on the heterogeneous device resource table of the heterogeneous AI device management system and the task requirement information of the AI processing task, a control decision is generated, including the following steps 601 to 604.

[0104] Step 601: updating the heterogeneous device resource table based on the device group performance data, and obtaining the main device performance data of the main processing device and the auxiliary device performance data of the auxiliary processing device from the updated heterogeneous device resource table.

[0105] Step 602: Generate a primary control decision for a primary processing device based on primary device performance data and core processing tasks, and generate an auxiliary control decision for an auxiliary processing device based on auxiliary device performance data and edge processing tasks.

[0106] Step 603: Generate a collaborative control decision between the main processing device and the auxiliary processing device based on the data transmission relationship between the core processing task and the edge processing task.

[0107] Steps 601 to 603 are described in detail below.

[0108] In some embodiments, after determining the main processing device corresponding to the core processing task in the AI processing task and the auxiliary processing device corresponding to the edge processing task in the AI processing task, the system first uses the device group performance data of the target device group just obtained or evaluated during the target processor task (for example, current computing unit utilization, available video memory, current load, available memory 1GB, estimated data transmission delay between the two devices) to update the global heterogeneous device resource table. This ensures that the resource table reflects the latest status of the two devices that are about to be scheduled to perform the AI processing task. Subsequently, the system accurately obtains the main device performance data of the main processing device (the updated complete record may include model, driver version, total resources, current available resources, connectivity, etc.) and the auxiliary device performance data of the auxiliary processing device from the updated heterogeneous device resource table.

[0109] Next, based on the main device performance data of the main processing device obtained from the resource table and the requirements of the core processing task, the system generates the main control decisions for the main device performance data, such as allocating video memory on the main device performance data, loading model files, configuring CUDA cores for execution, etc. At the same time, based on the auxiliary device performance data of the auxiliary device performance data and the requirements of the edge processing task, the system also generates auxiliary control decisions for the auxiliary device performance data, such as allocating cache on the auxiliary device performance data, loading preprocessing scripts, preparing the execution environment, etc.

[0110] Finally, the system analyzes the data transfer relationship between core and edge processing tasks. For example, the task flow dictates that edge tasks preprocess input data and then send the processed tensors to the core task for inference. Based on this relationship, the system generates collaborative control decisions between the primary and auxiliary processing devices, as described below.

[0111] Reference Figure 7 , based on the data transmission relationship between the core processing tasks and the edge processing tasks, a collaborative control decision between the main processing device and the auxiliary processing device is generated, including the following steps 701 to 704.

[0112] Step 701: Based on the core processing task and the edge processing task, determine the data interaction conditions and interaction data information.

[0113] Step 702: Generate synchronous clock information based on the primary device clock of the primary processing device and the secondary device clock of the secondary processing device.

[0114] Step 703: Generate protocol conversion information based on the primary device processing protocol of the primary processing device and the secondary device processing protocol of the secondary processing device.

[0115] Step 704: Generate a collaborative control decision based on the data interaction conditions, interaction data information, synchronization clock information, and protocol conversion information.

[0116] Steps 701 to 704 are described in detail below.

[0117] In some embodiments, the system analyzes the data transmission relationship between the core processing task and the edge processing task, which mainly consists of data interaction conditions and interaction data information. For example, it is determined that the data interaction condition is: the edge processing task successfully completes its image preprocessing operation. At the same time, it is determined that the interaction data information is: an image tensor of a specific format and size (for example, a tensor with a dimension of 224x224x3 and a data type of FP16), which is the output of the edge processing task and the direct input of the core processing task. The system also records the source (output buffer of the auxiliary processing device) and target (input buffer of the main processing device or a specified memory address) of the data.

[0118] In order to ensure that operations between two devices can proceed in an orderly manner and set reasonable timeouts, the system needs to consider time synchronization. It queries (or obtains from the heterogeneous device resource table) the main device clock information of the main processing device and the auxiliary device clock information of the auxiliary processing device through the adapted hardware interface. Based on the current readings of these two clocks or their synchronization status with a global time source (such as an NTP server), the system generates synchronized clock information, which can also be a maximum allowed transmission delay (for example, based on the current network conditions and clock difference estimation, data transmission should be completed within 50 milliseconds) or specify a reference timestamp mechanism for marking the start and end of data transmission.

[0119] The system then checks the primary processing device's required primary device processing protocol for receiving data (e.g., it might expect to receive raw binary data via RDMA direct memory access, or via a specific CUDA API) and the secondary processing device's secondary device processing protocol for sending data (e.g., it might default to sending serialized packets using a TCP socket, or publishing messages via MQTT). If the two protocols do not match, the system generates a protocol conversion message. For example, if the secondary processing device sends using TCP and the primary processing device expects to receive using RDMA, a protocol conversion message is generated that instructs the secondary processing device's adapter layer to format the data and send it via an RDMA-enabled interface. If the protocols are compatible (e.g., both parties agree to use TCP to transmit raw tensor data), the protocol conversion message simply confirms compatibility and specifies the specific port numbers and data format conventions.

[0120] Finally, the system integrates the generated data interaction conditions, interaction data information, synchronization clock information, and protocol conversion information to generate the final collaborative control decision.

[0121] Step 604: Based on the main control decision, the auxiliary control decision, and the collaborative control decision, the main processing device and the auxiliary processing device are controlled by adapting the hardware interface to collaboratively process the core processing tasks and the edge processing tasks.

[0122] Step 604 is described in detail below.

[0123] In some embodiments, the heterogeneous AI device management system parses the main control decisions, auxiliary control decisions, and collaborative control decisions through the adaptive hardware interface, and calls the adaptive hardware interface to control the main processing device and the auxiliary processing device through the adaptive driving information corresponding to the main processing device and the auxiliary processing device to collaboratively process core processing tasks and edge processing tasks.

[0124] Through the above steps 601 to 604, and steps 701 to 704, the optimal independent control strategies are generated for the main and auxiliary processing devices respectively based on the real-time device performance data, and by carefully defining the data interaction conditions, interaction content, clock synchronization mechanism and necessary protocol conversion, accurate collaborative control decisions are generated, thereby effectively solving the inherent communication barriers and timing coordination problems between heterogeneous devices. With the help of adaptive hardware interfaces, these complex control decisions containing independent and collaborative instructions are seamlessly converted into specific operations of the underlying hardware, ensuring that different types of AI devices can efficiently and reliably divide the work and collaborate to jointly complete a split AI processing task, significantly improving the overall performance, resource utilization and execution success rate of distributed AI computing, and greatly enhancing the flexibility and robustness of the system in complex heterogeneous environments.

[0125] Reference Figure 8 The control method of the heterogeneous AI device management system also includes the following steps 801 to 803.

[0126] Step 801: In response to an access request from a new AI device, obtain the name of the new AI device's adapter driver library, the adapter driver library parameters, and the adapter driver performance indicators through the adapter hardware interface, and generate adapter driver information for the new AI device based on the adapter driver library name, the adapter driver library parameters, and the adapter driver performance indicators.

[0127] Step 802: Based on the name of the adapter driver library, obtain the adapter driver library of the new AI device, and initialize the new AI device based on the adapter driver library of the new AI device.

[0128] Step 803: After the initialization process of the new AI device is completed, the adaptation driver information of the new AI device is added to the heterogeneous device resource table.

[0129] Steps 801 to 803 are described in detail below.

[0130] In some embodiments, when the system receives an access request from a newly connected heterogeneous AI device (i.e., a new AI device), it first communicates with the new AI device through an adaptation hardware interface. The adaptation hardware interface uses a predefined standardized communication protocol. For example, based on the device information query tool or standard interface provided by the manufacturer, the adaptation driver library name (such as libcambricon.so), adaptation driver library parameters (such as device ID, memory size, etc.), and adaptation driver performance indicators (such as computing power, power consumption, etc.) of the new AI device can be obtained. Then, based on the information obtained, the system generates the adaptation driver information of the new AI device, which can be understood as a structured data object containing key information such as the driver library name, parameters, performance indicators, etc., for subsequent driver loading and device initialization.

[0131] Given the differences in device driver interfaces between different manufacturers, this solution introduces a key design: a unified interface adapter module for heterogeneous AI devices. The hardware adapter module provides an adaptable interface that effectively shields the diversity of underlying manufacturer device driver interfaces, thereby presenting a standardized, consistent heterogeneous device interface to the upper layer. This way, when faced with the need to connect to new manufacturers' devices, the upper-layer architecture does not require complex adjustments or changes; it only needs to be implemented in accordance with established standard interface specifications for the new devices. The interfaces included in the adapted hardware interface include: GetDeviceCount: query the total number of AI devices; GetDeviceUtil: query the resource utilization of AI devices, and the incoming parameter is the device index number; GetDeviceMemoryInfo: query the used memory size, unused memory size and other information of the AI device, and the incoming parameter is the device index number; GetDeviceModel: query the AI device model, and the incoming parameter is the device index number; GetContainerVolume: query the device files, dynamic link libraries and tools that need to be mounted, and the incoming parameter is the device index number; Init: load the device driver library and initialize the device; Shutdown: shut down the device and release device resources.

[0132] Based on the name of the new AI device's adapter driver library (libcambricon.so), the system searches for and loads the corresponding adapter driver library from the system library or the specified path. After loading the driver library, the system uses the functions provided in the library to initialize the new AI device. Initialization includes allocating device resources, setting device parameters, loading firmware, and running self-tests to ensure the new AI device is operational.

[0133] When the initialization process for the new AI device is successfully completed, the system will add the obtained adaptive driver information of the new AI device to the heterogeneous device resource table. The heterogeneous device resource table stores information about all available AI devices in the system, including device type, manufacturer, performance indicators, driver information, and so on. After adding the new AI device information, the system can uniformly manage and schedule this new device. For example, during the subsequent task allocation process, the system can choose whether to use the new device for calculations based on the task requirements and the performance indicators of the new device. Through the above steps, the system realizes the automatic identification, driver loading and initialization of the new AI device and incorporates it into unified management, thereby enhancing the scalability and flexibility of the system.

[0134] Reference Figure 9 , is a schematic flow chart of initialization and resource control of a new AI device provided by an embodiment of the present application. Figure 9As shown in , when a new AI device is connected and the heterogeneous AI device management module of the heterogeneous AI management system is started, the system first searches for the corresponding device driver library based on the preset manufacturer list. If the driver library is found, the heterogeneous device initialization method is called and the initialization is checked to see if the initialization is successful. If the initialization is successful, the system will further interact with the upper-level cluster management system (Kubernetes, i.e., the heterogeneous device resource table): first, the heterogeneous device is registered with Kubernetes as an available resource, and labels with detailed information such as device model and video memory are added to the corresponding Kubernetes Node. The system then enters a continuous running state, listening for device resource requests from AI tasks (in order to allocate and mount drivers / tools on demand) and performance indicator collection requests (to support monitoring and scheduling decisions), thereby achieving fully automated management of heterogeneous AI devices from access to scheduling.

[0135] Reference Figure 10 , is a schematic diagram of a performance indicator provided in an embodiment of the present application. It shows the key monitoring indicators defined by the unified performance indicator collection interface provided by the heterogeneous AI device management system. The table clearly lists the four core performance indicators: device resource utilization (device_util), device memory usage (device_memory_used), device memory total (device_memory_total) and device temperature (device_temp). For each indicator, the table clearly states its name, Chinese description, label fields used to distinguish different devices and contexts (such as node name node_name, container identifier pod, device index device_index and vendor vendor), data type (all integer) and numerical unit (percentage, MB, degrees Celsius). By defining this set of standardized indicators and label systems, the monitoring module can shield the differences between AI devices of different manufacturers at the bottom layer and provide consistent and comparable real-time operating status data to the upper layer (such as the scheduler or user), which is crucial for realizing intelligent scheduling decisions based on the actual load and status of the device.

[0136] Reference Figure 11 , is a schematic diagram of part of the information in a heterogeneous device resource table provided in an embodiment of the present application. Figure 11As shown in , the key mechanism for integrating the device discovery module with the Kubernetes cluster management system in the heterogeneous AI device management system is demonstrated. Specifically, after identifying the various AI devices existing on the node, the device discovery module extracts its key static attribute information, such as the device manufacturer (device.vendor), device model (device.model), and the total device memory (device.memory), and converts this information into standardized Kubernetes labels. These labels are then attached to the Kubernetes Node object where the device is located, that is, stored in the heterogeneous device resource table. In this way, the underlying, physical AI hardware information is effectively abstracted and exposed to the upper-level business layer or scheduler, enabling them to query and perceive the AI device type and capabilities of each node through the standard Kubernetes API, providing basic data support for the subsequent implementation of refined task scheduling and resource management based on device characteristics.

[0137] Reference Figure 12 , is a schematic diagram of an environment variable provided in an embodiment of the present application. When the heterogeneous device container plug-in module packages the container runtime installed on the host, it enables the AI device to be mounted into the container for use by specifying simple and unified environment variables, without the need to understand the directories and files related to various AI devices. When the container needs to use an AI device, no matter which manufacturer's device it is, you only need to execute the following command to automatically identify the AI device and inject it into the container: docker run-it -e VISIBLE_DEVICES=1. The heterogeneous device command line tool is similar to NVIDIA's nvidia-smi, which uses the command line to query AI device information. It can query device model, usage rate, video memory size and other information, but the tool will automatically identify the device manufacturer, perform corresponding initialization and call the driver library function.

[0138] In addition, the AI cluster management module stores AI cluster management-related software such as Kubernetes, Docker, and Prometheus, and calls the interface for heterogeneous AI device management to complete operations such as resource allocation and performance indicator monitoring for AI devices.

[0139] This solution establishes a standardized bridge connecting AI devices from different manufacturers by defining a "unified interface for heterogeneous devices." This interface utilizes a layered design pattern, providing backward compatibility with AI device driver library functions from various manufacturers, directly calling underlying hardware drivers. Upward, it encapsulates a unified driver interface, providing standardized calling methods for upper-level functional modules. Compared to other technical solutions that often lack a device driver layer, this solution innovatively enables direct integration with manufacturer driver libraries, overcoming the hardware compatibility limitations of traditional solutions.

[0140] In terms of device management, this solution significantly improves the integration capabilities of heterogeneous AI clusters. Through unified interface specifications, core management functions such as AI device resource allocation, cluster device discovery, operational status monitoring, command-line tool interaction, and device container plug-in integration are standardized across vendors. Compared to directly connecting to AI device management software from multiple vendors, this solution reduces the management complexity of heterogeneous AI devices from the vendor level to the interface level, providing an innovative solution for unified management of heterogeneous devices in large-scale AI computing clusters.

[0141] The control method and related devices of a heterogeneous AI device management system proposed in an embodiment of the present application include a plurality of AI devices in the heterogeneous AI device management system, wherein at least two AI devices have different driver information in the device driver library. The method includes: first, in response to an execution request of an AI processing task, based on the core processing task of the AI processing task and the corresponding core computing required resources, selecting at least one AI device from the plurality of AI devices as a first candidate processing device using a heterogeneous device resource table; based on the edge processing task of the AI processing task and the corresponding edge computing required resources, selecting at least one AI device from the plurality of AI devices as a second candidate processing device using a heterogeneous device resource table; using each first candidate processing device as a first compatibility verification device one by one, determining first compatibility verification data output by the first compatibility verification device after processing the core processing task; matching the first compatibility verification data with the input data of each second candidate processing device to obtain a first compatibility verification result between each first candidate processing device and each second candidate processing device; using each second candidate processing device as a second compatibility verification device one by one, determining second compatibility verification data output by the second compatibility verification device after processing the edge processing task; and matching the second compatibility verification data with the input data of each first candidate processing device to obtain a first compatibility verification result between each first candidate processing device and each second candidate processing device; The first candidate processing device and the second candidate processing device are matched to obtain a second compatibility test result between each first candidate processing device and each second candidate processing device, and a device group whose first compatibility test result and the second compatibility test result both indicate data compatibility is selected from the plurality of first candidate processing devices and the plurality of second candidate processing devices one by one as a candidate device group. The device group includes a first candidate processing device and a second candidate processing device, and each candidate device group includes at least one first candidate processing device and at least one second alternative processing device. The real-time device group performance data of each candidate device group is obtained through the adaptation hardware interface, and the AI processing task of each candidate device group is calculated based on the device group performance data. The execution time, data transmission time, and heterogeneous device collaborative processing time corresponding to the AI processing task are calculated, and based on the execution time, data transmission time, and heterogeneous device collaborative processing time, a weighted accumulation is performed to obtain the processing verification time corresponding to each candidate device group executing the AI processing task. The candidate device group with the smallest processing verification time is selected from multiple candidate device groups as the target device group, and the first candidate processing device in the target device group is used as the main processing device, and the second candidate processing device in the target device group is used as the auxiliary processing device. The device driver library of the target processing device is obtained through the adaptation hardware interface, and the driver information in the device driver library includes the adaptation driver information of the target processing device;Then, the heterogeneous device resource table is updated based on the device group performance data, and the main device performance data of the main processing device and the auxiliary device performance data of the auxiliary processing device are obtained from the updated heterogeneous device resource table. The main control decision of the main processing device is generated based on the main device performance data and the core processing task, and the auxiliary control decision of the auxiliary processing device is generated based on the auxiliary device performance data and the edge processing task. Based on the core processing task and the edge processing task, the data interaction conditions and interaction data information are determined. The data transmission relationship mainly consists of the data interaction conditions and interaction data information. Based on the main device clock of the main processing device and the auxiliary device clock of the auxiliary processing device, the synchronization clock information is generated. Based on the main device processing protocol of the main processing device and the auxiliary device processing protocol of the auxiliary processing device, the protocol conversion information is generated. Based on the data interaction conditions, interaction data information, synchronization clock information and protocol conversion information, a synchronization clock is generated. Collaborative control decision-making: Based on the primary control decision, the auxiliary control decision, and the collaborative control decision, the primary and auxiliary processing devices are controlled through the adaptive hardware interface to collaboratively process core processing tasks and edge processing tasks. Furthermore, based on the control decision and the adaptive driver information, the target processing device is controlled to execute the AI processing task. The heterogeneous device resource table is obtained based on the adaptive driver information. In response to an access request from a new AI device, the name of the adaptive driver library, the parameters of the adaptive driver library, and the performance indicators of the adaptive driver are obtained through the adaptive hardware interface. Based on the name, the parameters, and the performance indicators, the adaptive driver information of the new AI device is generated. Based on the name, the adaptive driver library is obtained, and the new AI device is initialized based on the adaptive driver library. After the initialization of the new AI device is completed, the adaptive driver information of the new AI device is added to the heterogeneous device resource table.

[0142] The embodiment of the present application shields the differences in AI device driver libraries of different manufacturers by adapting the hardware interface. There is no need to set a specific vendor interface for each device. Instead, the driver library and adaptive driver information of the target device are uniformly obtained through the adapting hardware interface, and control decisions are generated based on a unified heterogeneous device resource table, thereby achieving unified control and resource scheduling of different types of AI devices, improving resource utilization and management efficiency, simplifying the management process, and being able to better cope with complex and diverse artificial intelligence computing tasks; in addition, by respectively checking the compatibility of the output data of the first alternative processing device with the input data of the second alternative processing device, and the output data of the second alternative processing device with the input data of the first alternative processing device, and finally selecting the two-way compatible one The device combination can effectively ensure unimpeded data interaction between core processing tasks and edge processing tasks, thereby improving the efficiency and reliability of the entire AI processing task and avoiding unnecessary computing and data conversion overhead; and, based on the core and edge computing requirements of the task and the static resource table, preliminarily screen out alternative processing equipment with basic capabilities, and ensure the feasibility of collaborative work between devices through data compatibility checks, and further use the adaptive hardware interface to obtain real-time performance data of the candidate device group, and based on this, perform refined time prediction (including execution, transmission, and collaboration time) and weighted evaluation, so as to dynamically and accurately select the target device group with the shortest expected processing time and highest efficiency under the current cluster load and network conditions. This multi-dimensional and dynamic decision-making process significantly improves the execution performance and overall resource utilization of AI tasks, and enhances the system's adaptability to complex heterogeneous environments and the level of scheduling intelligence. Moreover, based on real-time device performance data, it generates optimal independent control strategies for the main and auxiliary processing devices respectively, and generates accurate collaborative control decisions by carefully defining data interaction conditions, interaction content, clock synchronization mechanism and necessary protocol conversion, thereby effectively solving the inherent communication barriers and timing coordination problems between heterogeneous devices. With the help of adaptive hardware interfaces, these complex control decisions containing independent and collaborative instructions are seamlessly converted into specific operations of the underlying hardware, ensuring that different types of AI devices can efficiently and reliably divide the work and collaborate to jointly complete a split AI processing task, significantly improving the overall performance, resource utilization and execution success rate of distributed AI computing, and greatly enhancing the flexibility and robustness of the system in complex heterogeneous environments.

[0143] An embodiment of the present application further provides an electronic device, including:

[0144] at least one memory;

[0145] at least one processor;

[0146] at least one program;

[0147] The program is stored in the memory, and the processor executes the at least one program to implement the control method of the heterogeneous AI device management system implemented in this application. The electronic device can be any smart terminal including a mobile phone, tablet computer, personal digital assistant (PDA), car computer, etc.

[0148] See also Figure 13 , Figure 13 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0149] The processor 1301 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0150] The memory 1302 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 1302 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1302, and the processor 1301 calls and executes the control method of the heterogeneous AI device management system of the embodiment of this application;

[0151] Input / output interface 1303, used to implement information input and output;

[0152] Communication interface 1304, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0153] Bus 1305 , which transmits information between various components of the device (e.g., processor 1301 , memory 1302 , input / output interface 1303 , and communication interface 1304 );

[0154] The processor 1301 , the memory 1302 , the input / output interface 1303 and the communication interface 1304 are connected to each other in communication within the device via a bus 1305 .

[0155] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, which, when executed by a processor, implements the control method of the above-mentioned heterogeneous AI device management system.

[0156] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0158] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0160] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0161] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0162] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A control method for a heterogeneous AI device management system, characterized in that: The heterogeneous AI device management system includes multiple AI devices, and at least two of the AI devices have different driver information in their device driver libraries. The method includes: In response to a request to execute the AI processing task, determining a target processing device from the plurality of AI devices based on the AI processing task, and obtaining a device driver library for the target processing device through an adapted hardware interface, wherein driver information in the device driver library includes adapted driver information for the target processing device; Based on the heterogeneous device resource table of the heterogeneous AI device management system and the task requirement information of the AI processing task, a control decision is generated, and based on the control decision and the adaptation drive information, the target processing device is controlled to perform the AI processing task, and the heterogeneous device resource table is obtained based on the adaptation drive information.

2. The control method for a heterogeneous AI device management system according to claim 1, characterized in that: The method further comprises: In response to an access request from a new AI device, obtaining, through the adaptation hardware interface, an adaptation driver library name, adaptation driver library parameters, and adaptation driver performance indicators of the new AI device, and generating adaptation driver information for the new AI device based on the adaptation driver library name, the adaptation driver library parameters, and the adaptation driver performance indicators; Based on the name of the adapter driver library, obtaining the adapter driver library of the new AI device, and initializing the new AI device based on the adapter driver library of the new AI device; After the initialization process of the new AI device is completed, the adaptation driver information of the new AI device is added to the heterogeneous device resource table.

3. The control method for a heterogeneous AI device management system according to claim 1, characterized in that: The target processing device includes a main processing device and an auxiliary processing device, and determining the target processing device from the plurality of AI devices based on the AI processing task includes: Based on the core processing task of the AI processing task and the corresponding core computing resource requirements, select at least one of the AI devices from the multiple AI devices as a first candidate processing device using the heterogeneous device resource table; Based on the edge processing task of the AI processing task and the corresponding edge computing resource requirements, select at least one AI device from the multiple AI devices as a second candidate processing device using the heterogeneous device resource table; utilizing data compatibility between the AI devices and generating at least one candidate device group based on the first candidate processing device and the second candidate processing device, each candidate device group including at least one first candidate processing device and at least one second candidate processing device; Real-time device group performance data of each candidate device group is obtained through the adaptation hardware interface, a target device group is selected from the multiple candidate device groups based on the device group performance data, and the first alternative processing device in the target device group is used as the main processing device, and the second alternative processing device in the target device group is used as the auxiliary processing device.

4. The control method of the heterogeneous AI device management system according to claim 3 is characterized in that: The generating at least one candidate device group based on the first candidate processing device and the second candidate processing device by utilizing the data compatibility between the AI devices includes: Using each of the first candidate processing devices as a first compatibility verification device one by one, determining first compatibility verification data output by the first compatibility verification device after processing the core processing task, and matching the first compatibility verification data with input data of each of the second candidate processing devices to obtain a first compatibility verification result between each of the first candidate processing devices and each of the second candidate processing devices; Using each of the second candidate processing devices as a second compatibility verification device one by one, determining second compatibility verification data output by the second compatibility verification device after processing the edge processing task, and matching the second compatibility verification data with input data of each of the first candidate processing devices to obtain a second compatibility verification result between each of the first candidate processing devices and each of the second candidate processing devices; A device group whose first compatibility test results and second compatibility test results both indicate data compatibility is selected one by one from a plurality of the first candidate processing devices and a plurality of the second candidate processing devices as the candidate device group, wherein the device group includes one first candidate processing device and one second candidate processing device.

5. The control method for heterogeneous AI device management system according to claim 3, characterized in that: The selecting a target device group from the plurality of candidate device groups based on the device group performance data includes: Calculate, based on the device group performance data, the execution time, data transmission time, and heterogeneous device collaborative processing time corresponding to each candidate device group executing the AI processing task; Obtaining, by weighted accumulation based on the execution time, the data transmission time, and the heterogeneous device collaborative processing time, a processing verification time corresponding to execution of the AI processing task by each candidate device group; The candidate device group with the shortest processing verification time is selected from the plurality of candidate device groups as the target device group.

6. The control method for heterogeneous AI device management system according to claim 3, characterized in that: The control decision includes a main control decision, an auxiliary control decision, and a collaborative control decision. The control decision is generated based on the heterogeneous device resource table of the heterogeneous AI device management system and the task requirement information of the AI processing task, and the target processing device is controlled to perform the AI processing task based on the control decision, including: updating the heterogeneous device resource table based on the device group performance data, and acquiring the main device performance data of the main processing device and the auxiliary device performance data of the auxiliary processing device from the updated heterogeneous device resource table; generating the primary control decision of the primary processing device based on the primary device performance data and the core processing task, and generating the secondary control decision of the secondary processing device based on the secondary device performance data and the edge processing task; generating the collaborative control decision between the primary processing device and the auxiliary processing device based on a data transmission relationship between the core processing task and the edge processing task; Based on the main control decision, the auxiliary control decision and the collaborative control decision, the main processing device and the auxiliary processing device are controlled through the adapted hardware interface to collaboratively process the core processing task and the edge processing task.

7. The control method for a heterogeneous AI device management system according to claim 6, characterized in that: Generating the collaborative control decision between the main processing device and the auxiliary processing device based on the data transmission relationship between the core processing task and the edge processing task includes: Determining data interaction conditions and interaction data information based on the core processing task and the edge processing task, wherein the data transmission relationship mainly consists of the data interaction conditions and the interaction data information; generating synchronous clock information based on a primary device clock of the primary processing device and a secondary device clock of the secondary processing device; generating protocol conversion information based on a primary device processing protocol of the primary processing device and an auxiliary device processing protocol of the auxiliary processing device; The collaborative control decision is generated based on the data interaction condition, the interaction data information, the synchronization clock information, and the protocol conversion information.

8. A heterogeneous AI device management system, characterized in that: The heterogeneous AI device management system is used to execute the control method of the heterogeneous AI device management system according to claim 1, and the system includes: The AI cluster management module, heterogeneous AI device management module, and heterogeneous AI device unified interface adaptation module are connected in sequence; The heterogeneous AI device unified interface adaptation module is used to provide the adaptation hardware interface for each AI device; The heterogeneous AI device management module is configured to store the heterogeneous device resource table, wherein the heterogeneous device resource table stores device resources and device command line resources of the plurality of AI devices; The AI cluster management module is provided with management software for the multiple AI devices, and the AI cluster management module is configured to call the adaptation hardware interface through the management software to control at least one of the AI devices.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the control and management method for heterogeneous AI devices according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the control and management method for heterogeneous AI devices according to any one of claims 1 to 7 is implemented.

Images