Distributed heterogeneous computing power reasoning task dynamic scheduling method and system

Abstract

The invention discloses a distributed heterogeneous computing power reasoning task dynamic scheduling method and system. The system comprises a computing power architecture adaptation module, a task scheduling module, a model updating module, a capability service module, a dynamic scheduling module, a resource evaluation module and a state sensing module. The computing power architecture adaptation module is used for identifying and matching a local heterogeneous computing power architecture and a remote heterogeneous computing power architecture; the task scheduling module is used for selecting a proper computing power terminal from a distributed heterogeneous computing power environment; the model updating module is used for updating the AI model in real time; the capability service module is used for providing a unified AI model service calling interface; the dynamic scheduling module is used for dynamically allocating computing power resources; the resource evaluation module is used for monitoring and evaluating the state of the computing power resource; the state sensing module is used for monitoring the health state of the computing power node in real time and responding to a fault in time; the scheduling efficiency of computing power resource scheduling is improved, and stable operation of the reasoning task is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic scheduling of reasoning tasks, and in particular to a method and system for dynamic scheduling of distributed heterogeneous computing power reasoning tasks. Background Art

[0002] With the rapid development of new technologies, new formats, and new platforms such as 5G, artificial intelligence, and big data, various emerging businesses such as autonomous driving, face recognition, and intelligent manufacturing have put forward new requirements for flexible, convenient, and on-demand matching of computing power. In addition, while opening a new chapter in the coordinated layout of my country's computing power resources across the country, the "East Data West Computing" project also raised the core challenge of realizing cross-regional scheduling of computing power resources, and computing power scheduling based on intelligent allocation strategies also came into being.

[0003] At present, the Chinese invention patent with application number 202310486015.2 discloses a computing power resource scheduling method, a computing power resource scheduling model training method and a system. Although after obtaining the status data in the current state, the status data is input into the computing power resource scheduling model to obtain the scheduling data in the current state, thereby improving the scheduling efficiency of computing power resource scheduling, the computing power of a single device often cannot meet the reasoning requirements of large-scale models, and equipment failure or computing power fluctuations will also cause interruption or delay of the reasoning task. In addition, there are also problems such as high cost, data transmission delay and privacy protection. Summary of the invention

[0004] The technical problem solved by the present invention is that the computing power of a single device in the prior art often cannot meet the reasoning requirements of large-scale models, and device failure or fluctuation in computing power can also lead to interruption or delay of reasoning tasks. In addition, there are also problems such as high cost, data transmission delay and privacy protection.

[0005] In a first aspect, the present invention provides a distributed heterogeneous computing power reasoning task dynamic scheduling system, a distributed heterogeneous computing power reasoning task dynamic scheduling system includes a computing power architecture adaptation module, a task scheduling module, a model update module, a capability service module, a dynamic scheduling module, a resource evaluation module and a state perception module;

[0006] The computing power architecture adaptation module is used to identify and match the local heterogeneous computing power architecture and the remote heterogeneous computing power architecture;

[0007] The task scheduling module is used to select a computing power terminal from a distributed heterogeneous computing power environment according to the type of reasoning task and the computing power architecture required by the reasoning task;

[0008] The model updating module is used to update the AI ​​model in real time;

[0009] The capability service module is used to provide a unified AI model service calling interface;

[0010] The dynamic scheduling module is used to dynamically allocate computing resources according to reasoning tasks and resource status;

[0011] The resource evaluation module is used to monitor and evaluate the status and performance of computing resources;

[0012] The state perception module is used to monitor the health status of the computing power nodes in real time and respond to failures in a timely manner;

[0013] The reasoning task types include logical reasoning task types, causal reasoning task types and comparative reasoning task types;

[0014] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0015] The task scheduling module includes a generating unit, an acquiring unit and a scheduling unit;

[0016] The generating unit is used to generate a workflow according to the task instruction, and the workflow is used to indicate the execution process of completing the target task;

[0017] The acquisition unit is used to acquire the workflow and send the workflow to the scheduling unit;

[0018] The scheduling unit is used to select computing power nodes and computing power resources from distributed heterogeneous computing power according to the workflow;

[0019] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0020] The resource assessment module includes a monitoring unit and an assessment unit;

[0021] The monitoring unit is used to deploy a lightweight monitoring agent for each computing power node;

[0022] The evaluation unit is used to evaluate the resource status of each computing node using the lightweight monitoring agent of each computing node, and obtain the resource evaluation result of each computing node;

[0023] The resource status includes the CPU utilization, GPU utilization, memory and network status of the computing power node;

[0024] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0025] The state perception module includes a detection unit and a fault unit;

[0026] The detection unit is used to monitor the health status of the computing power node using a heartbeat detection mechanism to determine whether the computing power node has a fault;

[0027] The fault unit is used to perform real-time accurate fault diagnosis and fault handling on the failed computing power node using a fault recovery strategy;

[0028] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0029] The fault unit includes a fault tracing subunit and a fault handling subunit;

[0030] The fault tracing subunit is used to trace the fault area of ​​the computing power node and obtain the fault cause of the computing power node;

[0031] The fault processing subunit is used to perform fault processing on the fault area of ​​the computing power node according to the fault cause to ensure the normal operation of the computing power node;

[0032] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0033] The model updating module includes a data unit and an updating unit;

[0034] The data unit is used to obtain an inference data set, and the inference data in the inference data set is used to input into an existing AI model to perform inference;

[0035] The updating unit is used to compare the inference data set with the original training data set, determine the difference between the data distribution of the inference data set and the data distribution of the original training data set, and use the inference data set to update the AI ​​model to obtain the latest AI model;

[0036] The original training data set is a data set used to train the AI ​​model;

[0037] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0038] The dynamic scheduling module includes a partitioning unit, an estimation unit, a strategy unit and an allocation unit;

[0039] The division unit includes using a dynamic division method to divide the reasoning task into a plurality of subtasks according to the characteristics of the task in the calculation process;

[0040] The estimation unit is used to estimate the computing resources required for each subtask by using a prediction algorithm according to the complexity of each subtask, and obtain the computing resource requirements of each subtask, where the required computing resources include the CPU, memory and storage of the computer;

[0041] The strategy unit is used to obtain the optimal resource scheduling strategy based on the computing resource requirements of each subtask and the resource evaluation results of each computing node;

[0042] The allocation unit is used to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy using the priority queue;

[0043] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0044] The use of the heartbeat detection mechanism to monitor the health status of the computing power node includes:

[0045] Set a timeout mechanism so that the computing node sends a heartbeat packet to the system at a certain time interval;

[0046] After receiving the heartbeat packet, it responds with a response packet to the computing power node;

[0047] If the computing power node receives the response packet, it means that the computing power node is normal and the timeout countdown is reset;

[0048] If no response packet is received within a certain period of time, it means that the computing power node is faulty;

[0049] As a preferred solution of the distributed heterogeneous computing power reasoning task dynamic scheduling system described in the present invention, wherein:

[0050] Use the priority queue to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy:

[0051] Determine the priority of each subtask according to its characteristics;

[0052] Allocate each subtask to the corresponding priority waiting queue according to its priority level;

[0053] Determine the current scheduling queue based on the priority waiting queue of the subtask;

[0054] According to the scheduling queue, the optimal resource scheduling strategy is used to allocate available distributed node resources to subtask requests;

[0055] The priority waiting queue includes n priority waiting queues q from high to low priority levels 1 ,q 2 , ... q n ;

[0056] The use of a prediction algorithm to estimate the computing power resources required for each subtask includes selecting the historical computing power resource consumption of the last n groups of historical tasks as observation values ​​from the historical computing power resource records consumed by historical task operations, and obtaining the resource consumption of each subtask based on the observation values.

[0057] In a second aspect, a method for dynamically scheduling distributed heterogeneous computing power reasoning tasks includes the following steps:

[0058] Step S1: The client submits an inference task;

[0059] Step S2: The task scheduling module selects a suitable computing power terminal from a distributed heterogeneous computing power environment according to the type of reasoning task and the computing power architecture required by the reasoning task;

[0060] Step S3: The resource evaluation module evaluates the status and performance of each computing power node and obtains the evaluation results of each computing power node;

[0061] Step S4: The dynamic scheduling module divides the reasoning task into several subtasks, and uses the prediction algorithm to obtain the optimal resource scheduling strategy based on the computing resource requirements of each subtask and the evaluation results of each computing node;

[0062] Step S5: Use the priority queue to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy;

[0063] Step S6: The status perception module monitors the health status of the computing power node in real time and handles faults in a timely manner.

[0064] Beneficial effects of the present invention: The present invention intelligently allocates the most appropriate computing resources for reasoning tasks through dynamic scheduling, whether they are resources locally, in the cloud or in a P2P network, thereby greatly improving resource utilization efficiency and reducing resource waste.

[0065] The introduced state perception mechanism and fault recovery mechanism can detect and respond to computing resource failures in real time, automatically reschedule tasks, and ensure that the system can maintain stable operation in the face of hardware failures or network fluctuations, thereby enhancing the reliability and robustness of the system.

[0066] The unified capability service interface simplifies the calling process of AI models, lowers the threshold for developers, promotes the popularization and personalized application of AI technology, and realizes seamless collaboration in heterogeneous environments. The computing power architecture adaptation module and model update mechanism ensure the efficient execution of tasks in heterogeneous computing power environments, breaking the platform limitations and realizing seamless collaboration between different hardware platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1A basic flow chart of a distributed heterogeneous computing power reasoning task dynamic scheduling system provided by an embodiment of the present invention;

[0068] Figure 2 A basic flow chart of a method for dynamically scheduling distributed heterogeneous computing power reasoning tasks provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0070] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides a distributed heterogeneous computing power reasoning task dynamic scheduling system, including a computing power architecture adaptation module, a task scheduling module, a model update module, a capability service module, a dynamic scheduling module, a resource evaluation module and a state perception module;

[0071] The computing power architecture adaptation module is used to identify and match the local heterogeneous computing power architecture and the remote heterogeneous computing power architecture;

[0072] The task scheduling module is used to select a computing power terminal from a distributed heterogeneous computing power environment according to the type of reasoning task and the computing power architecture required by the reasoning task;

[0073] The model updating module is used to update the AI ​​model in real time;

[0074] The capability service module is used to provide a unified AI model service calling interface;

[0075] The dynamic scheduling module is used to dynamically allocate computing resources according to reasoning tasks and resource status;

[0076] The resource evaluation module is used to monitor and evaluate the status and performance of computing resources;

[0077] The state perception module is used to monitor the health status of the computing power nodes in real time and respond to failures in a timely manner;

[0078] The reasoning task types include logical reasoning task types, causal reasoning task types and comparative reasoning task types;

[0079] In this embodiment, the most suitable computing resources are intelligently allocated to the reasoning tasks through dynamic scheduling, whether they are local, cloud or P2P network resources, which improves the efficiency of resource utilization and reduces resource waste. The state perception mechanism and fault recovery mechanism can detect and respond to the failure of computing resources in real time, automatically reschedule tasks, and ensure that the system can maintain stable operation in the face of hardware failures or network fluctuations, thereby enhancing the reliability and robustness of the system.

[0080] The task scheduling module includes a generating unit, an acquiring unit and a scheduling unit;

[0081] The generating unit is used to generate a workflow according to the task instruction, and the workflow is used to indicate the execution process of completing the target task;

[0082] The acquisition unit is used to acquire the workflow and send the workflow to the scheduling unit;

[0083] The scheduling unit is used to select computing power nodes and computing power resources from distributed heterogeneous computing power according to the workflow;

[0084] In this embodiment, the computing power architecture requires the selection of computing power terminals from a distributed heterogeneous computing power environment, which improves the adaptability of resource utilization and provides adaptive computing power terminal selection for dynamic allocation of computing power resources for reasoning tasks;

[0085] The resource assessment module includes a monitoring unit and an assessment unit;

[0086] The monitoring unit is used to deploy a lightweight monitoring agent for each computing power node;

[0087] The evaluation unit is used to evaluate the resource status of each computing node using the lightweight monitoring agent of each computing node, and obtain the resource evaluation result of each computing node;

[0088] The resource status includes the CPU utilization, GPU utilization, memory and network status of the computing power node;

[0089] In this embodiment, by deploying a lightweight monitoring agent to monitor the computing nodes and evaluate the resource status of the computing nodes, a reliable data basis is provided for intelligent dynamic scheduling of computing resources;

[0090] The state perception module includes a detection unit and a fault unit;

[0091] The detection unit is used to monitor the health status of the computing power node using a heartbeat detection mechanism to determine whether the computing power node has a fault;

[0092] The fault unit is used to perform real-time accurate fault diagnosis and fault handling on the failed computing power node using a fault recovery strategy;

[0093] The fault recovery strategy includes fault isolation, service migration, fault repair and node restart;

[0094] The fault isolation includes that once a fault is detected, the system promptly removes the faulty node from the computing power architecture to prevent the fault from spreading to other healthy nodes, which usually involves updating routing information, stopping sending traffic to the faulty node, and removing the record of the faulty node in the service discovery component;

[0095] The service migration includes restoring the backup data during the inference task running process to the healthy computing node, and migrating the inference task service on the faulty node to the healthy node;

[0096] The fault repair includes accurately diagnosing the faulty computing node and intelligently repairing the faulty computing node according to the diagnosis result, so that the faulty computing node is restored to a healthy state;

[0097] In this embodiment, the heartbeat detection mechanism is used to detect the health status of the computing nodes and timely handle the faults of the computing nodes, ensuring that the system can still maintain stable operation in the face of hardware failures or network fluctuations, thereby enhancing the reliability and robustness of the system;

[0098] The fault unit includes a fault tracing subunit and a fault handling subunit;

[0099] The fault tracing subunit is used to trace the fault area of ​​the computing power node and obtain the fault cause of the computing power node;

[0100] The fault processing subunit is used to perform fault processing on the fault area of ​​the computing power node according to the fault cause to ensure the normal operation of the computing power node;

[0101] In this embodiment, the cause of the failure of the computing power node is obtained by tracing the failure area of ​​the computing power node and the failure of the computing power node is handled, so as to ensure that the system can still maintain stable operation in the face of hardware failure or network fluctuation, thereby enhancing the reliability of the system;

[0102] The model updating module includes a data unit and an updating unit;

[0103] The data unit is used to obtain an inference data set, and the inference data in the inference data set is used to input into an existing AI model to perform inference;

[0104] The updating unit is used to compare the inference data set with the original training data set, determine the difference between the data distribution of the inference data set and the data distribution of the original training data set, and use the inference data set to update the AI ​​model to obtain the latest AI model;

[0105] The original training data set is a data set used to train the AI ​​model;

[0106] In this embodiment, the AI ​​model is updated in real time according to the inference data set to ensure that all computing nodes use the latest AI model, thereby improving the inference accuracy and real-time performance of the inference task;

[0107] The dynamic scheduling module includes a partitioning unit, an estimation unit, a strategy unit and an allocation unit;

[0108] The division unit includes using a dynamic division method to divide the reasoning task into a plurality of subtasks according to the characteristics of the task in the calculation process;

[0109] The estimation unit is used to estimate the computing resources required for each subtask by using a prediction algorithm according to the complexity of each subtask, and obtain the computing resource requirements of each subtask, where the required computing resources include the CPU, memory and storage of the computer;

[0110] The strategy unit is used to obtain the optimal resource scheduling strategy based on the computing resource requirements of each subtask and the resource evaluation results of each computing node;

[0111] The allocation unit is used to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy using the priority queue;

[0112] In this embodiment, the prediction algorithm and priority queue are used to dynamically allocate computing resources to each subtask according to the computing resource demand and computing resource evaluation results, thereby improving resource utilization and reasoning task processing efficiency;

[0113] The use of the heartbeat detection mechanism to monitor the health status of the computing power node includes:

[0114] Set a timeout mechanism so that the computing node sends a heartbeat packet to the system at a certain time interval;

[0115] After receiving the heartbeat packet, it responds with a response packet to the computing power node;

[0116] If the computing power node receives the response packet, it means that the computing power node is normal and the timeout countdown is reset;

[0117] If no response packet is received within a certain period of time, it means that the computing power node is faulty;

[0118] In this embodiment, the heartbeat mechanism is used to detect the health status of the computing power node, which provides a processing target for fault handling of the faulty node and ensures the stable and healthy operation of the computing power node;

[0119] Use the priority queue to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy:

[0120] Determine the priority of each subtask according to its characteristics;

[0121] Allocate each subtask to the corresponding priority waiting queue according to its priority level;

[0122] Determine the current scheduling queue based on the priority waiting queue of the subtask;

[0123] According to the scheduling queue, the optimal resource scheduling strategy is used to allocate available distributed node resources to subtask requests;

[0124] The priority waiting queue includes n priority waiting queues q from high to low priority levels 1 ,q 2 , ... q n ;

[0125] The estimating the computing power resources required for each subtask by using the prediction algorithm includes selecting the historical computing power resource consumption of the last n groups of historical tasks as the observed value from the historical computing power resource records consumed by the historical tasks, and obtaining the resource consumption of each subtask according to the observed value;

[0126] In this embodiment, the priority queue is used to allocate computing resources to computing nodes according to the optimal resource scheduling strategy, thereby improving resource utilization and reasoning task processing efficiency;

[0127] Example 2, reference Figure 2 , which is another embodiment of the present invention. This embodiment is different from the first embodiment in that it provides a method for dynamically scheduling distributed heterogeneous computing power reasoning tasks, including the following steps:

[0128] Step S1: The client submits an inference task;

[0129] Step S2: The task scheduling module selects a suitable computing power terminal from a distributed heterogeneous computing power environment according to the type of reasoning task and the computing power architecture required by the reasoning task;

[0130] Step S3: The resource evaluation module evaluates the status and performance of each computing power node and obtains the evaluation results of each computing power node;

[0131] Step S4: The dynamic scheduling module divides the reasoning task into several subtasks, and uses the prediction algorithm to obtain the optimal resource scheduling strategy based on the computing resource requirements of each subtask and the evaluation results of each computing node;

[0132] Step S5: Use the priority queue to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy;

[0133] Step S6: The status perception module monitors the health status of the computing power node in real time and handles faults in a timely manner

[0134] The present invention intelligently allocates the most appropriate computing resources for reasoning tasks through dynamic scheduling, whether they are local, cloud or P2P network resources, which greatly improves resource utilization efficiency and reduces resource waste.

[0135] The introduced state perception mechanism and fault recovery mechanism can detect and respond to computing resource failures in real time, automatically reschedule tasks, and ensure that the system can maintain stable operation in the face of hardware failures or network fluctuations, thereby enhancing the reliability and robustness of the system.

[0136] Giving priority to local computing power can reduce data transmission delays, especially under poor network conditions; at the same time, sharing computing resources through P2P networks can reduce costs, avoid over-reliance on expensive cloud services, and reduce inference delays and costs.

[0137] The unified capability service interface simplifies the calling process of AI models, lowers the threshold for developers, promotes the popularization and personalized application of AI technology, and realizes seamless collaboration in heterogeneous environments. The computing power architecture adaptation module and model update mechanism ensure the efficient execution of tasks in heterogeneous computing power environments, breaking the platform limitations and realizing seamless collaboration between different hardware platforms.

[0138] It should be understood by those skilled in the art that the embodiments of the present invention can be provided as methods, systems or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A distributed heterogeneous computing power reasoning task dynamic scheduling system, characterized in that: It includes computing power architecture adaptation module, task scheduling module, model update module, capability service module, dynamic scheduling module, resource evaluation module and state perception module; The computing power architecture adaptation module is used to identify and match the local heterogeneous computing power architecture and the remote heterogeneous computing power architecture; The task scheduling module is used to select a computing power terminal from a distributed heterogeneous computing power environment according to the type of reasoning task and the computing power architecture required by the reasoning task; The model updating module is used to update the AI ​​model in real time; The capability service module is used to provide a unified AI model service calling interface; The dynamic scheduling module is used to dynamically allocate computing resources according to reasoning tasks and resource status; The resource evaluation module is used to monitor and evaluate the status and performance of computing resources; The state perception module is used to monitor the health status of the computing power nodes in real time and respond to failures in a timely manner; The reasoning task types include logical reasoning task types, causal reasoning task types and comparative reasoning task types.

2. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 1, characterized in that: The task scheduling module includes a generating unit, an acquiring unit and a scheduling unit; The generating unit is used to generate a workflow according to the task instruction, and the workflow is used to indicate the execution process of completing the target task; The acquisition unit is used to acquire the workflow and send the workflow to the scheduling unit; The scheduling unit is used to select computing nodes and computing resources from distributed heterogeneous computing power according to the workflow.

3. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 1, characterized in that: The resource assessment module includes a monitoring unit and an assessment unit; The monitoring unit is used to deploy a lightweight monitoring agent for each computing power node; The evaluation unit is used to evaluate the resource status of each computing node using the lightweight monitoring agent of each computing node, and obtain the resource evaluation result of each computing node; The resource status includes the CPU utilization, GPU utilization, memory and network status of the computing power node.

4. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 1, characterized in that: The state perception module includes a detection unit and a fault unit; The detection unit is used to monitor the health status of the computing power node using a heartbeat detection mechanism to determine whether the computing power node has a fault; The fault unit is used to perform real-time and accurate fault diagnosis and fault handling on the failed computing power node using a fault recovery strategy.

5. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 4, characterized in that: The fault unit includes a fault tracing subunit and a fault handling subunit; The fault tracing subunit is used to trace the fault area of ​​the computing power node and obtain the fault cause of the computing power node; The fault processing subunit is used to perform fault processing on the fault area of ​​the computing power node according to the fault cause to ensure the normal operation of the computing power node.

6. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 1, characterized in that: The model updating module includes a data unit and an updating unit; The data unit is used to obtain an inference data set, and the inference data in the inference data set is used to input into an existing AI model to perform inference; The updating unit is used to compare the inference data set with the original training data set, determine the difference between the data distribution of the inference data set and the data distribution of the original training data set, and use the inference data set to update the AI ​​model to obtain the latest AI model; The original training data set is the data set used to train the AI ​​model.

7. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 1, characterized in that: The dynamic scheduling module includes a partitioning unit, an estimation unit, a strategy unit and an allocation unit; The division unit includes using a dynamic division method to divide the reasoning task into a plurality of subtasks according to the characteristics of the task in the calculation process; The estimation unit is used to estimate the computing resources required for each subtask by using a prediction algorithm according to the complexity of each subtask, and obtain the computing resource requirements of each subtask, where the required computing resources include the CPU, memory and storage of the computer; The strategy unit is used to obtain the optimal resource scheduling strategy based on the computing resource requirements of each subtask and the resource evaluation results of each computing node; The allocation unit is used to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy using the priority queue.

8. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 4, characterized in that: The use of the heartbeat detection mechanism to monitor the health status of the computing power node includes: Set a timeout mechanism so that the computing node sends a heartbeat packet to the system at a certain time interval; After receiving the heartbeat packet, it responds with a response packet to the computing power node; If the computing power node receives the response packet, it means that the computing power node is normal and the timeout countdown is reset; If no response packet is received within a certain period of time, it means that the computing power node has a fault.

9. A distributed heterogeneous computing power reasoning task dynamic scheduling system as claimed in claim 7, characterized in that: Use the priority queue to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy: Determine the priority of each subtask according to its characteristics; Allocate each subtask to the corresponding priority waiting queue according to its priority level; Determine the current scheduling queue based on the priority waiting queue of the subtask; According to the scheduling queue, the optimal resource scheduling strategy is used to allocate available distributed node resources to subtask requests; The priority waiting queue includes n priority waiting queues q1, q2, ..., q1 from high to low priority. n ; The use of a prediction algorithm to estimate the computing power resources required for each subtask includes selecting the historical computing power resource consumption of the last n groups of historical tasks as observation values ​​from the historical computing power resource records consumed by historical task operations, and obtaining the resource consumption of each subtask based on the observation values.

10. A distributed heterogeneous computing power reasoning task dynamic scheduling method, comprising a distributed heterogeneous computing power reasoning task dynamic scheduling system as described in any one of claims 1-9, characterized in that: The following steps are involved: Step S1: The client submits an inference task; Step S2: The task scheduling module selects a suitable computing power terminal from a distributed heterogeneous computing power environment according to the type of reasoning task and the computing power architecture required by the reasoning task; Step S3: The resource evaluation module evaluates the status and performance of each computing power node and obtains the evaluation results of each computing power node; Step S4: The dynamic scheduling module divides the reasoning task into several subtasks, and uses the prediction algorithm to obtain the optimal resource scheduling strategy based on the computing resource requirements of each subtask and the evaluation results of each computing node; Step S5: Use the priority queue to dynamically allocate computing resources to each subtask according to the optimal resource scheduling strategy; Step S6: The status perception module monitors the health status of the computing power node in real time and handles faults in a timely manner.

Images