Node resource configuration method for artificial intelligence chip clusters
By acquiring runtime telemetry data and status information of AI chips and combining them with historical operating models, sub-healthy nodes can be identified and isolated in real time, solving the problem of AI chip cluster failures that traditional detection mechanisms cannot identify, and improving the stability and availability of AI chip clusters.
Patent Information
- Application Number
- CN202610219094.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-26
- Estimated Expiration
- 2046-02-24
AI Technical Summary
Traditional health monitoring mechanisms struggle to effectively identify sub-health fault modes in AI chip clusters, such as micro-deadlock of computing units and abnormal decline in effective computing power, leading to frequent scheduling and rebuilding, which affects the execution efficiency of AI computing tasks.
By acquiring runtime telemetry data and status information from the AI chip, and combining it with historical operating models, the health status of nodes can be identified in real time. Nodes that do not meet the predetermined conditions are automatically excluded from the resource pool to avoid affecting the service level agreement.
It enables automatic identification and isolation of AI chip cluster node failures, improving stability and availability under high load scenarios and avoiding substantial impact on SLA.
Smart Images

Figure CN121722576B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computers, and more specifically to a method for configuring node resources for artificial intelligence chip clusters. Background Technology
[0002] With the rapid development of artificial intelligence (AI) applications, AI chips are widely deployed in large-scale training and inference scenarios. Unlike traditional cloud-native services, AI computing tasks are typically characterized by computational intensity, long running time, and strong state dependence. Their execution efficiency largely depends on the highly centralized deployment of tasks on the same computing node or even the same chip, in order to reduce the additional overhead caused by cross-node communication, data copying, and context switching.
[0003] In scenarios involving AI computing tasks, the node health monitoring and scheduling mechanisms in traditional cloud-native systems, which focus on traffic, load, and resource utilization, are not entirely suitable for AI chip clusters. On one hand, the execution of AI computing tasks causes a rapid increase in node load, power consumption, and temperature within the AI chip cluster. Traditional health monitoring mechanisms can easily misclassify such "high-load but normal" nodes as abnormal nodes, frequently triggering scheduling, eviction, or rebuild operations. On the other hand, traditional health monitoring methods typically lack the ability to perceive the internal state of AI chips, making it difficult to promptly detect fault patterns that have a substantial impact on the execution of AI computing tasks but may not necessarily be reflected in node load, such as computing unit deadlock or a decline in effective computing power. Summary of the Invention
[0004] The node resource configuration method for an artificial intelligence chip cluster according to embodiments of this disclosure includes, for any node in the artificial intelligence chip cluster: acquiring runtime telemetry data and runtime status information of the node; generating node health status information based on the node's runtime telemetry data and runtime status information, combined with the node's historical operating model, wherein the node's historical operating model includes the node's baseline number of instructions per clock cycle and the baseline of the instruction queue depth of each computing unit in the node; generating node health status information includes: acquiring the node's actual number of instructions per clock cycle based on the node's runtime telemetry data, and determining whether there is an abnormal decline in the node's effective computing power by comparing the node's actual number of instructions per clock cycle with the baseline number of instructions per clock cycle; and determining the depth of the pending instruction queue of each computing unit in the node based on the node's runtime status information, and by comparing the depth of the pending instruction queue of each computing unit in the node with the baseline number of instructions per clock cycle. The system uses the pending instruction queue depth and instruction queue depth baseline of each computing unit to determine whether micro-deadlock has occurred in any computing unit within a node, and to determine whether the ratio between the number of computing units with micro-deadlock and the total number of computing units in the node is greater than a preset deadlock unit percentage threshold. Based on the node's health status information, the system determines whether the node meets the predetermined conditions for executing artificial intelligence computing tasks. These predetermined conditions include that the node does not have an abnormal decline in effective computing power and that the ratio between the number of computing units with micro-deadlock and the total number of computing units in the node is not greater than the deadlock unit percentage threshold. If the node is determined to meet the predetermined conditions for executing artificial intelligence computing tasks, it is included in or kept within the node resource pool for executing artificial intelligence computing tasks; otherwise, the node is excluded from the node resource pool for executing artificial intelligence computing tasks.
[0005] These and other features of the node resource configuration method disclosed herein, as well as the operational methods, functions, and economics of manufacture of related structural elements and combinations thereof, will become more apparent upon consideration of the following description and appended claims with reference to the accompanying drawings (all of which form a part of this specification, wherein like reference numerals identify corresponding portions in the drawings). However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and not for limiting purposes. Attached Figure Description
[0006] Figure 1 This is a schematic flowchart of a node resource configuration method for an AI chip cluster according to an embodiment of the present disclosure.
[0007] Figure 2 This is a schematic block diagram of an example computing device that can implement the node resource configuration method for an AI chip cluster according to embodiments of the present disclosure. Detailed Implementation
[0008] The features and exemplary embodiments of various aspects of this application will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a comprehensive understanding of this application. However, it will be apparent to those skilled in the art that this application can be implemented without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of this application by illustrating examples. This application is by no means limited to any specific configuration presented below, but covers any modifications, substitutions, and improvements to elements, components, and algorithms without departing from the spirit of this application. Well-known structures and techniques are not shown in the accompanying drawings and the following description in order to avoid unnecessary obfuscation of this application.
[0009] Furthermore, the various operations will be described as multiple discrete operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order. In particular, these operations do not need to be performed in the order presented.
[0010] The phrases “in an embodiment,” “in one embodiment,” and “in some embodiments” are used repeatedly throughout this document. These phrases do not typically refer to the same embodiment; however, they may refer to the same embodiment. Unless the context otherwise specifies, the terms “comprising,” “having,” and “including” are synonyms. The phrases “A or B” and “A / B” mean “(A), (B) or (A and B).”
[0011] The inventors discovered that during long-term high-load operation of AI chips (such as Graphics Processing Units (GPUs), Neural Network Processing Units (NPUs), and Tensor Processing Units (TPUs), there exists a type of "sub-health" fault mode that traditional health check methods cannot effectively detect. Specifically, this includes:
[0012] Computing Unit (CU) Micro-deadlock: Some computing units of an AI chip fall into a micro-deadlock state due to timing disorder or voltage drop. This manifests as the computing progress of the computing unit stagnates, but the overall chip temperature, memory utilization, and operating system response are normal, which cannot be detected by traditional monitoring.
[0013] Abnormal decrease in effective computing power (Throughput): After the AI chip has been working for a long time, due to the Dynamic Voltage and Frequency Scaling (DVFS) strategy or increased leakage, the peak computing power remains unchanged but the effective computing throughput decreases (for example, the number of inference frames processed per second decreases by 30% but no error is reported).
[0014] In view of the above, this disclosure provides a node resource configuration method for AI chip clusters. Based on the runtime telemetry data and runtime status information of the AI chip, combined with the historical operation model of the AI chip, it can identify the unique "sub-healthy" state of the AI chip in real time and automatically and smoothly exclude it from the node resource pool used to perform AI computing tasks, so as to avoid substantial impact on the Service Level Agreement (SLA).
[0015] Figure 1 This is a schematic flowchart of a node resource configuration method for an AI chip cluster according to an embodiment of this disclosure. Figure 1 As shown, the node resource configuration method 100 for an AI chip cluster includes the following steps for any node in the AI chip cluster (hereinafter referred to as node N for simplicity): S102, obtaining runtime telemetry data and runtime status information of node N; S104, generating health status information of node N based on the runtime telemetry data and runtime status information of node N, combined with the historical operation model of node N; S106, determining whether node N meets the predetermined conditions for executing AI computing tasks based on the health status information of node N; and S108, if node N meets the predetermined conditions for executing AI computing tasks, then node N is included in or kept in the node resource pool for executing AI computing tasks; otherwise, node N is excluded from the node resource pool for executing AI computing tasks.
[0016] In some embodiments, the historical operating model of node N includes the baseline of instructions per clock cycle for node N and the baseline of instruction queue depth for each compute unit in node N. Here, the baseline of instructions per clock cycle for node N refers to the number of instructions per clock cycle for node N in a healthy baseline state (e.g., idle state or standard load state), and the baseline of instruction queue depth for each compute unit in node N refers to the pending instruction queue depth for each compute unit in node N in a healthy baseline state (e.g., idle state or standard load state).
[0017] In some embodiments, generating health status information for node N includes: obtaining the actual number of instructions per clock cycle for node N based on runtime telemetry data of node N, and determining whether there is an abnormal decline in effective computing power of node N by comparing the actual number of instructions per clock cycle of node N with the baseline number of instructions per clock cycle; obtaining the pending instruction queue depth of each computing unit in node N based on runtime status information of node N, and determining whether each computing unit in node N has experienced micro-deadlock by comparing the pending instruction queue depth of each computing unit in node N with the baseline instruction queue depth, and determining whether the ratio between the number of computing units in node N experiencing micro-deadlock and the total number of computing units in node N is greater than a preset deadlock unit percentage threshold. In this case, the predetermined conditions for executing the artificial intelligence computing task include that there is no abnormal decline in effective computing power of node N and that the ratio between the number of computing units in node N experiencing micro-deadlock and the total number of computing units in node N is not greater than the deadlock unit percentage threshold. Here, the deadlock unit percentage threshold for each node may be equal or unequal.
[0018] In some embodiments, for any computing unit (hereinafter referred to as computing unit CU) in node N, a micro-deadlock is determined to have occurred when the pending instruction queue depth of computing unit CU is greater than the instruction queue depth baseline, no instruction completion signal (e.g., a retirement signal) associated with computing unit CU has reached a preset duration, and the clock gating state of computing unit CU is non-idle. Here, the instruction queue depth baseline of each computing unit can be the same or different.
[0019] In some embodiments, the historical operating model of node N further includes a temperature-frequency curve slope baseline indicating the rate of change of the core frequency of node N as temperature increases under a healthy baseline state. Generating the health status information of node N also includes: obtaining the actual temperature-frequency curve slope of the node based on runtime telemetry data of node N, and determining whether node N has low heat dissipation efficiency by comparing the actual temperature-frequency curve slope of node N with the temperature-frequency curve slope baseline. For example, when the actual temperature-frequency curve slope of node N is greater than the temperature-frequency curve slope baseline and the difference between them is greater than a preset slope difference threshold, it is determined that node N has low heat dissipation efficiency. In this case, the predetermined condition for performing the AI computing task also includes that node N does not have low heat dissipation efficiency.
[0020] In some embodiments, the following process is used to detect whether node N has experienced an abnormal decline in effective computing power: obtaining the number of FP32 / INT8 operation instructions completed by node N in the past second; reading the core clock frequency of node N in the past second; calculating the number of instructions per clock cycle (IPC) of node N, where IPC = number of completed operation instructions / (core clock frequency × time); comparing the actual IPC of node N with the IPC baseline; if the actual IPC of node N is lower than (for example) 90% of the IPC baseline, and the utilization rate of computing units in node N is higher than (for example) 70% (excluding low IPC caused by idle time), then it is determined that node N has experienced an abnormal decline in effective computing power.
[0021] The node resource configuration method according to the embodiments of this disclosure can identify “hidden faulty nodes” that can no longer guarantee SLA by abnormal fluctuations in microarchitecture-level performance indicators (i.e., the number of instructions per clock cycle of the node and the depth of the pending instruction queue of the computing unit) when traditional health check methods are completely blind. And through an automated eviction mechanism, the source of damage is cut off before a substantial and persistent violation of SLA occurs.
[0022] In some embodiments, if it is determined that node N does not meet the predetermined conditions for executing AI computing tasks, the node resource configuration method 100 for the AI chip cluster further includes: prohibiting the scheduling of new AI computing tasks to node N, and performing migration, reconstruction, or recovery operations on the AI computing tasks running on node N. For example, the Kubernetes scheduling and control mechanism can be used to prohibit the scheduling of new AI computing tasks to node N, and to perform migration, reconstruction, or recovery operations on the AI computing tasks running on node N.
[0023] In some embodiments, when it is determined that node N meets the predetermined conditions for executing AI computing tasks, the node resource configuration method 100 for AI chip clusters further includes: allowing new AI computing tasks to be scheduled to node N.
[0024] In summary, the node resource configuration method for AI chip clusters according to the embodiments of this disclosure can realize the automatic identification, isolation, and self-healing of node faults in AI chip clusters, thereby improving the stability and high availability of AI chip clusters under high-load AI computing scenarios.
[0025] Figure 2 This is a schematic block diagram of an example computing device capable of implementing a chip firmware upgrade method for use in a server cluster according to embodiments of the present disclosure. Figure 2As shown, computing device 200 may include bus 202 or other communication mechanisms for transmitting information and one or more hardware processors 204 coupled to bus 202 for processing information. The one or more hardware processors 204 may include, for example, one or more general-purpose microprocessors.
[0026] like Figure 2 As shown, in some embodiments, computing device 200 may further include main memory 206 coupled to bus 202 for storing information and instructions executed by one or more processors 204, such as random access memory (RAM), cache, and / or other dynamic storage devices. Main memory 206 may also be used to store temporary variables or other intermediate information during the execution of instructions executed by one or more processors 204. These instructions, when stored in storage media accessible to one or more processors 204, can cause computing device 200 to become a dedicated machine customized to perform the operations specified in the instructions. Storage device 208 may include non-volatile and / or volatile storage media. Non-volatile storage media may include, for example, optical discs or magnetic disks. Volatile storage media may include dynamic memory. Common forms of storage media may include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, DRAM, PROM, and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge, or their networking versions.
[0027] like Figure 2 As shown, in some embodiments, computing device 200 may further include a communication interface 210 coupled to bus 202. Communication interface 210 may provide bidirectional data communication coupling to one or more network links connected to one or more networks. As another example, communication interface 210 may be a local area network (LAN) card to provide data communication connectivity to a LAN-compatible (or WAN component communicating with a WAN) network. Wireless links may also be implemented.
[0028] The execution of certain operations can be distributed across processors rather than residing within a single machine, but rather deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.
[0029] Each of the processes, methods, and algorithms described in the preceding sections may be embodied in code modules executed by one or more computer systems or computer processors including computer hardware, and may be fully or partially automated by these code modules. The processes and algorithms may be implemented, partially or fully, in dedicated circuit systems.
[0030] When the functions disclosed herein are implemented as software functional units and sold or used as stand-alone products, they may be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, may be embodied in the form of a software product. The software product may be stored in a storage medium and includes several instructions that cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or some steps of the methods of the embodiments of this application. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, other media operable to store program code, or any combination thereof.
[0031] Specific embodiments further provide a system including a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to steps in any method of the embodiments disclosed above. Specific embodiments further provide a non-transitory computer-readable storage medium storing instructions executable by one or more processors to cause the one or more processors to perform operations corresponding to steps in any method of the embodiments disclosed above.
[0032] The embodiments disclosed herein can be implemented via a cloud platform, server, or server cluster (collectively referred to below as the "Service System") that interacts with a client. The client can be a terminal device or a client registered by a user at the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can have the platform application installed.
[0033] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Additionally, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and their associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in an order other than that specifically disclosed, or multiple blocks or states may be combined into a single block or state. Example blocks or states may be executed sequentially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein may be configured differently than described. For example, components may be added to, removed from, or rearranged compared to the disclosed example embodiments.
[0034] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. The algorithm may be included in program code or instructions stored in memory (e.g., the aforementioned non-transitory computer-readable storage medium). This algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not explicitly refer to the computer as performing the function but may learn from training data to generate a predictive model of the function.
[0035] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, these processors can constitute an engine of processor implementation that operates to perform one or more of the operations or functions described herein.
[0036] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are instances of hardware. For example, at least some operations of the methods may be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors may also be operable to support the execution of relevant operations in a “cloud computing” environment or as the execution of relevant operations in a “Software as a Service” (SaaS) context. For example, at least some operations may be performed by a group of computers (as an example of a machine containing processors), wherein these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)).
[0037] The execution of certain operations can be distributed across processors rather than residing within a single machine, and can be deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.
[0038] Throughout this specification, multiple instances may be implemented as components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of these individual operations may be performed simultaneously, and not necessarily in the order illustrated. Structures and functions presented as separate components in the example configuration may be implemented as composite structures or components. Similarly, structures and functions presented as single components may be implemented as single components. These and other variations, modifications, additions, and improvements fall within the scope of this document.
[0039] As used herein, "or" is inclusive rather than exclusive unless explicitly indicated by the context. Therefore, in this document, "A, B, or C" means "A, B, A and B, A and C, B and C, or A, B, and C" unless explicitly indicated by the context. Furthermore, "and" is combined and separate unless explicitly indicated by the context. Therefore, in this document, "A and B" means "A and B, combined or separate" unless explicitly indicated by the context. Additionally, multiple instances of resources, operations, or structures described herein may be provided as a single instance. Furthermore, the boundaries between various resources, operations, engines, and data storage devices are somewhat arbitrary and specific operations are illustrated within the context of a particular illustrative configuration. Other functional assignments are foreseeable and fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as individual resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as single resources. These and other changes, modifications, additions, and improvements fall within the scope of the embodiments of this disclosure as expressed in the appended claims. Therefore, this specification and drawings should be considered illustrative rather than restrictive.
[0040] The terms “comprising” or “including” are used to indicate the presence of a subsequently claimed feature, but do not preclude the addition of other features. Unless otherwise specifically stated or otherwise understood in the context in which they are used, conditional language such as “may,” “can,” “may,” and “can” is generally intended to convey that certain embodiments include certain features, components, and / or steps that are not included in other embodiments. Therefore, this conditional language is generally not intended to imply that one or more embodiments require features, components, and / or steps in any way, or that one or more embodiments must include logic for determining whether such features, components, and / or steps are included in or performed in any particular embodiment, with or without user input or prompts.
[0041] Although the general outline of the subject matter has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broad scope of embodiments of this disclosure. Where more than one embodiment is disclosed, these embodiments of the subject matter may be referred to individually or collectively herein as the term "invention," this is for convenience only and is not intended to automatically limit the scope of this application to any single disclosure or concept.
[0042] The embodiments illustrated herein have been described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Therefore, the term "implementation" is not intended to be limiting, and the scope of the various embodiments is defined only by the appended claims and their full scope.
Claims
1. A method for configuring node resources in an artificial intelligence chip cluster, characterized in that, This includes any node in the aforementioned AI chip cluster: Obtain the runtime telemetry data and runtime status information of the node; Based on the node's runtime telemetry data and runtime status information, and combined with the node's historical operating model, the node's health status information is generated. The node's historical operating model includes the node's baseline instruction count per clock cycle and the baseline instruction queue depth of each computing unit within the node. The generated health status information includes: Based on the node's runtime telemetry data, the actual number of instructions per clock cycle (NCR) of the node is obtained. By comparing the actual NCR with the baseline NCR, it is determined whether the node experiences an abnormal decline in effective computing power. Based on the runtime state information of the node, the pending instruction queue depth of each computing unit in the node is obtained. By comparing the pending instruction queue depth of each computing unit in the node with the instruction queue depth baseline, it is determined whether each computing unit in the node has a micro deadlock. It is also determined whether the ratio between the number of computing units in the node that have micro deadlocks and the total number of computing units in the node is greater than a preset deadlock unit proportion threshold. Based on the health status information of the node, it is determined whether the node meets the predetermined conditions for executing artificial intelligence computing tasks. These predetermined conditions include: the node does not experience an abnormal decline in effective computing power; and the ratio between the number of computing units experiencing micro-deadlocks in the node and the total number of computing units in the node is not greater than a deadlock unit percentage threshold. If the node is determined to meet the predetermined conditions, the node will be included in or kept in the node resource pool for performing artificial intelligence computing tasks; otherwise, the node will be excluded from the node resource pool.
2. The node resource allocation method according to claim 1, characterized in that, When the actual number of instructions per clock cycle of the node is lower than a preset ratio of the baseline number of instructions per clock cycle and the utilization rate of the node's computing unit is greater than a preset computing unit utilization rate threshold, it is determined that the node has an abnormal decline in effective computing power. The computing unit utilization rate of the node is also obtained based on the real-time telemetry data of the node.
3. The node resource allocation method according to claim 1, characterized in that, For any computing unit in the node, a micro deadlock is determined when the computing unit meets all of the following conditions: the depth of the pending instruction queue of the computing unit is greater than the baseline of the instruction queue depth, no instruction completion signal associated with the computing unit has reached a preset duration, and the clock gating state of the computing unit is non-idle.
4. The node resource allocation method according to claim 1, characterized in that, The historical operating model of the node also includes a temperature-frequency curve slope baseline indicating the rate of change of the node's core frequency as temperature increases under a healthy baseline state, and the generation of the node's health status information also includes: The actual temperature-frequency curve slope of the node is obtained based on the node's runtime telemetry data, and the node's heat dissipation efficiency is determined by comparing the actual temperature-frequency curve slope with the baseline temperature-frequency curve slope. The predetermined conditions also include that the node does not have low heat dissipation efficiency.
5. The node resource allocation method according to claim 4, characterized in that, When the slope of the actual temperature-frequency curve of the node is greater than the baseline of the temperature-frequency curve slope and the difference between them is greater than a preset slope difference threshold, the node is determined to have low heat dissipation efficiency.
6. The node resource allocation method according to claim 1, characterized in that, Also includes: While excluding the node from the node resource pool, scheduling new AI computing tasks to the node is prohibited, and migration, reconstruction, or recovery operations are performed on the AI computing tasks running on the node.
7. The node resource allocation method according to claim 1, characterized in that, Also includes: While incorporating the node into the node resource pool, it is possible to schedule new artificial intelligence computing tasks to the node.
8. The node resource allocation method according to claim 6, characterized in that, By utilizing Kubernetes scheduling and control mechanisms, new AI computing tasks are prohibited from being scheduled to the node, and migration, reconstruction, or recovery operations are performed on the AI computing tasks running on the node.
9. A computer program product, comprising instructions, characterized in that, When executed by one or more processors, the instructions cause the one or more processors to perform the node resource configuration method according to any one of claims 1 to 8.
10. A non-transitory computer-readable storage medium, comprising instructions, characterized in that, When executed by one or more processors, the instructions cause the one or more processors to perform the node resource configuration method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Distributed deep neural network performance modeling method based on an instruction queue
CN109657794A
Main control chip task scheduling and dynamic performance optimization method based on neural network
CN120872529A