AI chip adaptive deployment method and system based on dynamic heterogeneous resource awareness

Abstract

The invention provides an AI chip adaptive deployment method and system based on dynamic heterogeneous resource awareness, and belongs to the technical field of artificial intelligence chip deploy.The method comprises the steps that a resource monitoring process is deployed in AI chip nodes in a cluster, heterogeneous index data of the AI chip nodes are collected in real time, and the heterogeneous index data are uploaded to a scheduling controller according to sampling intervals; the scheduling controller responds to the received task request, analyzes task request parameters, and selects a scheduling algorithm according to the real-time mark: predicting computing power resources required by the task request by using an LSTM model, performing reservation on a target AI chip node, and mapping the reserved computing power resources into Kubernetes schedulable resources; and the scheduling controller dynamically adjusts and optimizes the task according to the task execution condition and the heterogeneous index data of the target AI chip node. According to the method, the AI chip resources in the heterogeneous computing environment are scheduled and managed, the resource utilization rate is increased, and the task execution delay is reduced.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence chip deployment technology, and specifically relates to an AI chip adaptive deployment method and system based on dynamic heterogeneous resource perception. Background Art

[0002] With the advancement of artificial intelligence (AI) technology, scheduling AI chips in heterogeneous computing environments is crucial for improving resource utilization and ensuring efficient task execution. Current AI chip deployment suffers from the following drawbacks: First, traditional static scheduling strategies (such as fixed resource quota allocation) fail to perceive the AI chip's load status in real time, easily leading to resource contention or resource throttling. Second, the existing Kubernetes default scheduler allocates resources based solely on static CPU and memory requests, ignoring the AI chip's unique computing power metrics. This results in low GPU memory utilization and high memory fragmentation. Third, while threshold-based dynamic scheduling methods can trigger task migration, threshold setting relies on empirical experience and is difficult to adapt to drastic load fluctuations. Especially when load fluctuations are extreme, scheduling failure rates are excessively high, failing to meet demand. Furthermore, the sudden computing demands of AI tasks often require frequent manual adjustments to deployment parameters, which is inefficient and prone to errors.

[0003] Therefore, there is an urgent need for an efficient task scheduling solution that can dynamically perceive the status of heterogeneous resources, intelligently predict load changes and adaptively adjust, so as to achieve global optimization of resources and real-time and stable execution of tasks to meet the growing demand for high-performance computing. Summary of the Invention

[0004] In a first aspect, an embodiment of the present application provides an AI chip adaptive deployment method based on dynamic heterogeneous resource perception, comprising the following steps: S1. Deploy a resource monitoring process on the AI chip nodes within the cluster to collect heterogeneous indicator data from the AI chip nodes in real time and upload it to the scheduling controller using the gRPC protocol at sampling intervals. S2. The scheduling controller responds to the received task request, parses the task request parameters, and selects a scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; S3. Use the LSTM model to predict the computing resources required for the task request, reserve the required computing resources on the target AI chip node, and map the reserved computing resources to Kubernetes schedulable resources. S4. The scheduling controller dynamically adjusts and optimizes tasks based on the task execution status and the heterogeneous indicator data of the target AI chip node.

[0005] Furthermore, the heterogeneous indicators in step S1 include memory fragmentation rate, computing unit utilization, and cross-node communication delay; Video memory fragmentation rate = (total video memory - maximum continuous video memory block) / total video memory × 100%; Computing unit utilization = actual number of computing units used / total number of computing units × 100%; Communication delay across nodes = round-trip delay in data transmission between nodes.

[0006] Furthermore, the task request parameters in step S2 include video memory requirements, required computing unit type, and real-time level; the real-time level includes real-time tasks and non-real-time tasks; The greedy algorithm is started to select the target AI chip node in the cluster for the task and generate a scheduling plan as follows: Filter all AI chip nodes that meet the task request parameter requirements from the current cluster and calculate the comprehensive resource idleness of each AI chip node: Comprehensive resource idleness = a*(1-memory fragmentation rate) + b*(1-computing unit utilization) + c*(1-cross-node communication delay / maximum communication delay) Where a, b, and c are the weights of the video memory fragmentation rate, computing unit utilization, and cross-node communication delay, respectively, and a+b+c=1; From all AI chip nodes that meet the task request parameter requirements, select the node with the highest comprehensive resource idleness as the target AI chip node.

[0007] Furthermore, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster. The specific steps are as follows: The optimization goals are to minimize task completion time, maximize resource utilization and load balancing, and construct a fitness function: F = α*(1 / task estimated completion time) + β*resource balancing factor + γ*cluster resource utilization Among them, α is the task completion time weight, β is the resource balance weight, and γ is the utilization weight; AI chip nodes in the cluster are layered according to node type; Use the LSTM model to predict the resource requirements and execution time of the task, use the prediction results as the initial population of the genetic algorithm, and initialize the genetic algorithm parameters; the genetic algorithm parameters include the crossover rate and mutation rate; Starting from the initial population, the genetic algorithm is executed according to the genetic algorithm parameters to find the scheduling scheme that minimizes the fitness function. During the execution of the genetic algorithm, the node type is first coarsely screened and then finely screened according to the node, and the genetic algorithm parameters and fitness function weights are adjusted according to the load.

[0008] Furthermore, during the execution of the genetic algorithm, a coarse screening is first performed according to the node type and then a fine screening is performed according to the node as follows: First, perform a coarse screening based on the node type layer to select the optimal node type. Then, perform a fine screening based on the selected node type to determine the target AI chip node. Adjusting genetic algorithm parameters and fitness function weights according to load includes: When the overall load of the cluster is greater than the first set ratio of the maximum load, the crossover rate of the genetic algorithm is increased to the upper limit of the crossover rate, and the mutation rate of the genetic algorithm is reduced to the lower limit of the mutation rate; When there is a second set ratio where the local load is greater than the maximum load, the resource balancing weight in the fitness function is increased according to the set amplitude.

[0009] Furthermore, the specific steps for using the LSTM model to predict the computing resources required for the task request in step S3 are as follows: Collect historical task execution data, extract task request parameters and allocated computing resources during task execution, and train the LSTM model using task request parameters as input and allocated computing resources as output; Input the task request parameters in the task request into the trained LSTM model to predict the required computing resources; The specific steps for mapping reserved computing resources to Kubernetes schedulable resources are as follows: Create the corresponding Pod through the containerized interface Kubernetes API and inject hardware optimization parameters into the Pod annotation to map the required computing power resources of the target AI chip to Kubernetes schedulable resources.

[0010] Furthermore, the specific steps of step S4 are as follows: S41. The scheduling controller obtains the memory fragmentation rate through the resource monitoring process of the AI chip during task execution, and when the memory fragmentation rate exceeds the threshold, initiates defragmentation, and dynamically migrates or merges memory blocks; S42. When the scheduling controller recognizes an increase in tasks, it determines whether a single AI chip can meet the computing resource requirements. If it cannot, it initiates distributed sharing of graphics memory or computing units to collaborate across nodes. S43. When the scheduling controller detects a failure in the target AI chip node during task execution, it starts the pre-assigned task copy and switches the task to the backup AI chip node.

[0011] In a second aspect, an embodiment of the present application further provides an AI chip adaptive deployment system based on dynamic heterogeneous resource perception, including: The heterogeneous indicator collection module is used to execute the AI chip node deployment resource monitoring process in heterogeneous computing within the cluster, collect heterogeneous indicator data of AI chip nodes in real time, and upload it to the scheduling controller according to the sampling interval using the gRPC protocol; The task scheduling module is used to receive task requests through the scheduling controller response, parse the task request parameters, and select the scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; The task allocation and containerized deployment module uses the LSTM model to predict the computing resources required for task requests, reserves the required computing resources on the target AI chip node, and maps the reserved computing resources to Kubernetes schedulable resources. The task adjustment module is used to dynamically adjust and optimize tasks based on the task execution status and target AI chip resource usage through the scheduling controller.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method for adaptive deployment of AI chips based on dynamic heterogeneous resource perception as described in the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for adaptive deployment of AI chips based on dynamic heterogeneous resource perception as described in the first aspect are implemented.

[0014] It can be seen from the above technical solutions that this application has the following advantages: The AI chip adaptive deployment method, system, device and medium based on dynamic heterogeneous resource perception provided in this application realizes the scheduling and management of AI chip resources in heterogeneous computing environments by dynamically monitoring the heterogeneous indicator data of AI chip nodes, combining greedy algorithms with improved genetic algorithms, as well as LSTM model prediction and Kubernetes containerized deployment. It can improve resource utilization, reduce task execution latency, enhance the stability and reliability of the cluster, meet the scheduling requirements of different types of tasks, and adapt to complex and changeable load conditions. It is especially suitable for high-concurrency and high-dynamic load scenarios such as large model reasoning and edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 This is a flow chart of the adaptive deployment method of AI chips based on dynamic heterogeneous resource perception of the present invention.

[0017] Figure 2 Schematic diagram of the AI chip adaptive deployment system based on dynamic heterogeneous resource perception of the present invention. DETAILED DESCRIPTION

[0018] The specific steps of the adaptive deployment method of AI chips based on dynamic heterogeneous resource perception will be described in detail below, and various embodiments of the present disclosure will be described more comprehensively. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather that the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0019] For example, with the advancement of artificial intelligence technology, scheduling AI chips in heterogeneous computing environments is crucial for improving resource utilization and ensuring efficient task execution. Current AI chip deployment suffers from the following drawbacks: First, traditional static scheduling strategies (such as fixed resource quota allocation) fail to perceive the AI chip's load status in real time, easily leading to resource contention or resource throttling. Second, the existing Kubernetes default scheduler allocates resources based solely on static CPU and memory request values, ignoring the AI chip's unique computing power metrics. This results in low GPU memory utilization and high memory fragmentation. Third, while threshold-based dynamic scheduling methods can trigger task migration, threshold setting relies on empirical experience and is difficult to adapt to drastic load fluctuations. Especially when load fluctuations are extreme, scheduling failure rates are excessively high, failing to meet demand. Furthermore, the sudden computing demands of AI tasks often require frequent manual adjustments to deployment parameters, which is inefficient and prone to errors.

[0020] Therefore, there is an urgent need for an efficient task scheduling solution that can dynamically perceive the status of heterogeneous resources, intelligently predict load changes and adaptively adjust, so as to achieve global optimization of resources and real-time and stable execution of tasks to meet the growing demand for high-performance computing.

[0021] To address the above issues, this embodiment provides an AI chip adaptive deployment method based on dynamic heterogeneous resource perception. The process framework of the AI chip adaptive deployment method based on dynamic heterogeneous resource perception implements the key links from resource monitoring, task request processing, computing power resource prediction and reservation to dynamic task adjustment, thereby realizing an efficient and intelligent AI chip scheduling method.

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] See also Figure 1 FIG. 1 is a flowchart of a method for adaptively deploying an AI chip based on dynamic heterogeneous resource perception in a specific embodiment. The method includes the following steps: S1. Deploy a resource monitoring process on the AI chip nodes within the cluster to collect heterogeneous indicator data from the AI chip nodes in real time and upload it to the scheduling controller using the gRPC protocol at sampling intervals. It should be noted that by deploying a resource monitoring process on the AI chip nodes within the cluster, heterogeneous indicator data is collected in real time and uploaded to the scheduling controller. This enables the scheduling controller to promptly and accurately grasp the resource status information of each AI chip node, providing a data basis for subsequent task scheduling and avoiding improper resource allocation caused by delayed or inaccurate information. S2. The scheduling controller responds to the received task request, parses the task request parameters, and selects a scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; It should be noted that the scheduling controller selects a scheduling algorithm based on the real-time flag in the task request parameters. For real-time tasks, a greedy algorithm is launched to quickly select the target node and generate a scheduling plan. This can respond to tasks with high real-time requirements in a timely manner and reduce task waiting time. For non-real-time tasks, an improved genetic algorithm is launched to search for the optimal solution globally. This can better balance resource utilization and task completion efficiency while meeting task execution requirements, thereby improving overall resource utilization. S3. Use the LSTM model to predict the computing resources required for the task request, reserve the required computing resources on the target AI chip node, and map the reserved computing resources to Kubernetes schedulable resources. It should be noted that using the LSTM model to predict and reserve the computing resources required for task requests, and mapping the reserved computing resources to Kubernetes-schedulable resources, enables early prediction and reasonable reservation of task resource requirements, avoiding frequent scheduling or waiting due to insufficient resources during task execution. At the same time, containerization enables resources to be managed and scheduled by Kubernetes, improving the flexibility and efficiency of resource management. S4. The scheduling controller dynamically adjusts and optimizes tasks based on task execution status and the heterogeneous indicator data of the target AI chip; It should be noted that tasks are dynamically adjusted and optimized based on the task execution status and the target AI chip resource usage, so that the scheduling controller can adapt to various changes in the task execution process, such as resource fluctuations and changes in task priority, and timely adjust task allocation and resource utilization strategies to ensure the smooth execution of tasks and the effective use of resources, thereby enhancing the system's adaptability and stability.

[0024] This embodiment can improve the resource utilization of AI chips in heterogeneous computing environments, reduce task execution delays, meet the scheduling requirements of different types of tasks, and adapt to complex and changeable load conditions. It is especially suitable for high-concurrency and high-dynamic load scenarios such as large model reasoning and edge computing.

[0025] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another AI chip adaptive deployment method based on dynamic heterogeneous resource perception is provided, which includes the following steps: S1. Deploy a resource monitoring process on the AI chip nodes within the cluster to collect heterogeneous indicator data from the AI chip nodes in real time and upload it to the scheduling controller using the gRPC protocol at sampling intervals. The heterogeneous indicators in step S1 include the memory fragmentation rate, computing unit utilization, and cross-node communication delay; Video memory fragmentation rate = (total video memory - maximum continuous video memory block) / total video memory × 100%; Computing unit utilization = actual number of computing units used / total number of computing units × 100%; Cross-node communication delay = round-trip delay of data transmission between nodes; It should be noted that the calculation of the video memory fragmentation rate can be used to understand the utilization and fragmentation level of video memory, thereby allocating and managing video memory resources. The computing unit utilization rate reflects the use of computing resources, thereby rationally allocating computing tasks. Cross-node communication delay affects the execution efficiency of multi-node collaborative tasks. Considering this indicator can optimize inter-node communication and task collaboration, quantify and evaluate the resource status of AI chip nodes, provide a basis for subsequent scheduling decisions, and improve the rationality and effectiveness of resource scheduling. S2. The scheduling controller responds to the received task request, parses the task request parameters, and selects a scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; The task request parameters in step S2 include video memory requirements, required computing unit type, and real-time level; the real-time level includes real-time tasks and non-real-time tasks; For example, non-real-time tasks are offline tasks; The greedy algorithm is started to select the target AI chip node in the cluster for the task and generate a scheduling plan as follows: Filter all AI chip nodes that meet the task request parameter requirements from the current cluster and calculate the comprehensive resource idleness of each AI chip node: Comprehensive resource idleness = a*(1-memory fragmentation rate) + b*(1-computing unit utilization) + c*(1-cross-node communication delay / maximum communication delay) Where a, b, and c are the weights of the video memory fragmentation rate, computing unit utilization, and cross-node communication delay, respectively, and a+b+c=1; From all AI chip nodes that meet the task request parameter requirements, select the node with the highest comprehensive resource idleness as the target AI chip node; It should be noted that clarifying the graphics memory requirements and computing unit types ensures that tasks are assigned to AI chip nodes that meet their hardware requirements, thereby improving the success rate of task execution. Differentiating between real-time and non-real-time tasks and adopting different scheduling strategies ensures the timeliness of real-time tasks and the overall efficiency of non-real-time tasks. The specific steps for starting the improved genetic algorithm to generate a scheduling plan for the target AI chip node in the task selection cluster are as follows: The optimization goals are to minimize task completion time, maximize resource utilization and load balancing, and construct a fitness function: F = α*(1 / task estimated completion time) + β*resource balancing factor + γ*cluster resource utilization Among them, α is the task completion time weight, β is the resource balance weight, and γ is the utilization weight; AI chip nodes in the cluster are layered according to node type; Exemplarily, the nodes are divided into computation-intensive nodes and memory-intensive nodes; Use the LSTM model to predict the resource requirements and execution time of the task, use the prediction results as the initial population of the genetic algorithm, and initialize the genetic algorithm parameters; the genetic algorithm parameters include the crossover rate and mutation rate; Starting from the initial population, the genetic algorithm is executed according to the genetic algorithm parameters to find the scheduling scheme that minimizes the fitness function. During the execution of the genetic algorithm, the nodes are first coarsely screened according to the node type and then finely screened according to the node. The genetic algorithm parameters and fitness function weights are adjusted according to the load. It should be noted that by constructing a reasonable fitness function with the optimization goals of minimizing task completion time, maximizing resource utilization, and balancing load, the genetic algorithm can be guided to search for a better scheduling solution, achieving efficient resource utilization and balanced load distribution while meeting task execution requirements. By stratifying AI chip nodes within the cluster by type, targeted scheduling optimization can be performed based on the characteristics of different types of nodes. Using the LSTM model prediction results as the initial population and initializing the algorithm parameters can accelerate the convergence of the genetic algorithm, improve search efficiency, and enable the algorithm to find a better solution more quickly. During the execution of the genetic algorithm, the first step is to coarsely screen according to the node type and then finely screen according to the node as follows: First, perform a coarse screening based on the node type layer to select the optimal node type. Then, perform a fine screening based on the selected node type to determine the target AI chip node. Adjusting genetic algorithm parameters and fitness function weights according to load includes: When the overall load of the cluster is greater than the first set ratio of the maximum load, the crossover rate of the genetic algorithm is increased to the upper limit of the crossover rate, and the mutation rate of the genetic algorithm is reduced to the lower limit of the mutation rate; For example, when the overall complexity of the cluster is greater than 70% of the maximum load, the crossover rate is increased to 0.85 and the mutation rate is reduced to 0.05; When there is a second set ratio where the local load is greater than the maximum load, the resource balancing weight β in the fitness function is increased according to the set amplitude; For example, when the local complexity is greater than 90% of the maximum complexity, the resource balance weight in the fitness function is adjusted from 0.3 to 0.5; It should be noted that the method of first screening by node type and then screening by node improves search efficiency, can quickly locate a relatively suitable node range in a large-scale cluster, and then further accurately select the target node, improving the quality of the scheduling plan; dynamically adjusting the crossover rate and mutation rate of the genetic algorithm and the resource balancing weight parameters in the fitness function according to the overall and local load conditions of the cluster enables the algorithm to adapt to different load conditions, enhances the adaptability of the scheduling plan to complex and changing environments, and better balances the various requirements of task completion time and load balancing; S3. Use the LSTM model to predict the computing resources required for the task request, reserve the required computing resources on the target AI chip node, and map the reserved computing resources to Kubernetes schedulable resources. The specific steps for using the LSTM model to predict the computing resources required for the task request in step S3 are as follows: Collect historical task execution data, extract task request parameters and allocated computing resources during task execution, and train the LSTM model using task request parameters as input and allocated computing resources as output; Input the task request parameters in the task request into the trained LSTM model to predict the required computing resources; The specific steps for mapping reserved computing resources to Kubernetes schedulable resources are as follows: Create a corresponding Pod through the containerized interface Kubernetes API and inject hardware optimization parameters into the Pod annotation to map the required computing power resources of the target AI chip to Kubernetes schedulable resources; Exemplarily, the hardware optimization parameters include video memory management parameters (e.g., page lock memory flag, video memory pre-allocation size), computing acceleration parameters (e.g., FPGA bitstream version, GPU CUDA Stream configuration), and communication optimization parameters (e.g., NCCL topology binding, RDMA enable flag). It should be noted that by collecting historical task data to train the LSTM model, it can accurately predict the computing resource requirements of new tasks, improving the accuracy of resource reservation, reducing resource waste and task waiting time. At the same time, the specific content of hardware optimization parameters is clarified, and the method of creating Pods through the Kubernetes API and injecting hardware optimization parameters to achieve computing resource mapping is defined. This strengthens the integration with existing container orchestration technology, improves the automation and flexibility of resource management, and facilitates the effective implementation of this scheduling solution in actual cloud computing or cluster environments. S4. The scheduling controller dynamically adjusts and optimizes tasks based on task execution status and heterogeneous indicator data of the target AI chip node. The specific steps of step S4 are as follows: S41. The scheduling controller obtains the memory fragmentation rate through the resource monitoring process of the AI chip during task execution, and when the memory fragmentation rate exceeds the threshold, initiates defragmentation, and dynamically migrates or merges memory blocks; S42. When the scheduling controller recognizes an increase in tasks, it determines whether a single AI chip can meet the computing resource requirements. If it cannot, it initiates distributed sharing of graphics memory or computing units to collaborate across nodes. S43. When the scheduling controller detects a failure in the target AI chip node during task execution, it starts the pre-assigned task copy and switches the task to the backup AI chip node; It should be noted that specific measures for scheduling optimization can effectively address various problems that may arise during task execution, such as excessive video memory fragmentation affecting performance, insufficient resources on a single node due to increased task load, and task interruption caused by node failure. By timely defragmentation, resource sharing, and switching task copies, the continuous and stable execution of tasks is guaranteed, the reliability and availability of the system are improved, and it is ensured that in a complex operating environment, the system can still efficiently and stably complete the scheduling and management of various AI tasks.

[0026] Take the deployment of an intelligent security video analysis task as an example: First, lightweight monitoring agents are installed on each AI chip node in the intelligent security system (such as the NVIDIA RTX 3090 GPU installed in the monitoring center). These agents collect video memory fragmentation and computing unit utilization metrics through Prometheus Exporter and upload the data to the scheduling controller every 30 seconds using the gRPC protocol. The scheduling controller then incorporates an improved multi-objective genetic algorithm with the optimization goals of minimizing task completion time, maximizing resource utilization, and balancing load. A greedy strategy can be activated for real-time tasks. An LSTM model is also introduced to predict task resource requirements and execution times. The predictions are used as the initial population for the genetic algorithm, and the algorithm parameters are dynamically adjusted based on the cluster load. Next, the scheduling policy is encapsulated as a Kubernetes custom resource (CRD). The operator listens for smart security video analysis task requests and triggers deployment. When the task is issued, hardware optimization parameters, such as GPU memory page lock, are automatically injected to reduce runtime overhead. Then, when real-time intelligent security video analysis is needed for a critical area, the system issues a task request. Upon receiving the request, the scheduling controller parses the task's required parameters, such as the required video memory ≥10GB, specific video decoding and image recognition operator types, and determines that the task is a high-priority real-time task. Because the task is a high-priority task with strict real-time requirements, the scheduling controller initiates a greedy algorithm to select the least busy GPU node in the current cluster. Assume that after this selection, the task is assigned to the NVIDIA RTX 3090 GPU numbered "GPU Node A." Before the task is officially scheduled, the corresponding video memory and computing units are reserved on "GPU Node A" in advance based on the resources required for the intelligent security video analysis task predicted by the LSTM model, reducing the waiting time during task scheduling; Create the corresponding Pod through the Kubernetes API and inject the hardware optimization parameter "enable_memory_lock=true" into the Pod annotation. Use the Device Plugin mechanism to map the GPU computing power units on "GPU Node A" to Kubernetes schedulable resources to complete the containerized deployment of the task. As video analysis tasks in other areas of the intelligent security system increase or decrease, the cluster's load status constantly changes. The scheduling controller dynamically adjusts the weights in the fitness function based on the real-time load status to balance task completion time and load balancing. For example, if the load on "GPU node A" is gradually increasing while the load on other GPU nodes is relatively low, the weight of task completion time is appropriately reduced and the weight of load balancing is increased, so that subsequent tasks can be more evenly distributed to different nodes. While GPU Node A was executing the intelligent security video analysis task, the monitoring agent detected a gradual increase in the video memory fragmentation rate in real time. When the fragmentation rate reached a certain threshold, the scheduling controller triggered the video memory defragmentation mechanism, which dynamically migrated or merged video memory blocks to reduce fragmentation, improve video memory utilization, and ensure the task could continue to run stably. If the scale of intelligent security video analysis tasks continues to expand, the resources of a single "GPU node A" will be unable to meet the demand. The scheduling controller introduces a distributed shared memory mechanism, allowing tasks to share video memory or computing units between "GPU node A" and other idle GPU nodes or FPGA nodes, realizing cross-node resource collaboration and improving the overall processing capability of the system. If "GPU node A" fails during task execution, the scheduling controller immediately switches the task to the spare GPU node based on the pre-assigned task copy, ensuring the uninterrupted operation of the intelligent security video analysis task and improving the reliability and stability of the system.

[0027] Through the above specific examples, it can be seen that the AI chip adaptive deployment system and method based on dynamic heterogeneous resource perception can effectively improve the resource utilization and task real-time performance of intelligent security video analysis tasks, ensuring the stable and efficient operation of the system in complex and changing scenarios.

[0028] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0029] like Figure 2 As shown, the following is an embodiment of the AI chip adaptive deployment system based on dynamic heterogeneous resource perception provided by the embodiment of the present disclosure. This system and the AI chip adaptive deployment method based on dynamic heterogeneous resource perception in the above embodiments belong to the same inventive concept. For details not fully described in the embodiment of the AI chip adaptive deployment system based on dynamic heterogeneous resource perception, please refer to the embodiment of the AI chip adaptive deployment method based on dynamic heterogeneous resource perception.

[0030] The system includes: The heterogeneous indicator collection module is used to deploy resource monitoring processes on AI chip nodes within the cluster, collect heterogeneous indicator data of AI chip nodes in real time, and upload it to the scheduling controller using the gRPC protocol at sampling intervals; The task scheduling module is used to receive task requests through the scheduling controller response, parse the task request parameters, and select the scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; The task allocation and containerized deployment module uses the LSTM model to predict the computing resources required for task requests, reserves the required computing resources on the target AI chip node, and maps the reserved computing resources to Kubernetes schedulable resources. The task adjustment module is used to dynamically adjust and optimize tasks based on the task execution status and heterogeneous indicator data of the target AI chip node through the scheduling controller.

[0031] This embodiment uses the interactive collaboration of the heterogeneous indicator collection module, task allocation and containerized deployment module, task scheduling module, and task adjustment module to effectively perform various tasks from resource monitoring, task scheduling decision-making, task deployment to dynamic adjustment and optimization, providing a basis for achieving efficient adaptive deployment of AI chips, improving the overall performance and management efficiency of the system, facilitating deployment and implementation in actual applications, and meeting the AI chip scheduling needs under different scales and scenarios.

[0032] The adaptive deployment method of AI chips based on dynamic heterogeneous resource perception provided in the embodiments of the present application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange components differently. In an embodiment of the present invention, electronic devices include but are not limited to laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0033] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a button, a camera, a display, and a SIM card interface, etc.

[0034] It is understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown, or combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0035] A processor may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0036] The processor can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0037] The processor may also include a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or is reusing. If the processor needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0038] The above-mentioned electronic device implements the resource monitoring process of deploying AI chip nodes in the cluster of the AI chip adaptive deployment method based on dynamic heterogeneous resource perception of the present application, collects heterogeneous indicator data of AI chip nodes in real time, and uploads it to the scheduling controller according to the sampling interval using the gRPC protocol; the scheduling controller responds to the received task request, parses the task request parameters, and selects a scheduling algorithm based on the real-time tag in the task request parameters: if it is a real-time task, the greedy algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; if it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; the LSTM model is used to predict the computing power resources required for the task request, the required computing power resources are reserved on the target AI chip node, and the reserved computing power resources are mapped to Kubernetes schedulable resources; the scheduling controller dynamically adjusts and optimizes the task according to the task execution status and the heterogeneous indicator data of the target AI chip node, so as to improve the resource utilization of AI chips in heterogeneous computing environments, reduce task execution delays, meet the scheduling requirements of different types of tasks, and adapt to complex and changeable load conditions.

[0039] The storage medium provided in this application stores a program product that can implement an AI chip adaptive deployment method based on dynamic heterogeneous resource perception.

[0040] The adaptive deployment method of AI chips based on dynamic heterogeneous resource perception includes: deploying a resource monitoring process on the AI chip nodes in the cluster, collecting heterogeneous indicator data of the AI chip nodes in real time, and uploading it to the scheduling controller according to the sampling interval using the gRPC protocol; the scheduling controller responds to the received task request, parses the task request parameters, and selects the scheduling algorithm according to the real-time tag in the task request parameters: if it is a real-time task, starts the greedy algorithm to generate a scheduling plan for the target AI chip node in the task selection cluster; if it is a non-real-time task, starts the improved genetic algorithm to generate a scheduling plan for the target AI chip node in the task selection cluster; uses the LSTM model to predict the computing power resources required for the task request, reserves the required computing power resources on the target AI chip node, and maps the reserved computing power resources to Kubernetes schedulable resources; the scheduling controller dynamically adjusts and optimizes the task according to the task execution status and the heterogeneous indicator data of the target AI chip node.

[0041] In some possible implementations, the AI chip adaptive deployment method based on dynamic heterogeneous resource perception disclosed herein can be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above "Exemplary Method" section of this specification according to various exemplary implementations of the present disclosure.

[0042] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0043] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for adaptive deployment of AI chips based on dynamic heterogeneous resource perception, characterized in that: The steps include: S1. Deploy a resource monitoring process on the AI chip nodes within the cluster to collect heterogeneous indicator data from the AI chip nodes in real time and upload it to the scheduling controller using the gRPC protocol at sampling intervals. S2. The scheduling controller responds to the received task request, parses the task request parameters, and selects a scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; S3. Use the LSTM model to predict the computing resources required for the task request, reserve the required computing resources on the target AI chip node, and map the reserved computing resources to Kubernetes schedulable resources. S4. The scheduling controller dynamically adjusts and optimizes tasks based on the task execution status and the heterogeneous indicator data of the target AI chip node.

2. The method for adaptively deploying AI chips based on dynamic heterogeneous resource perception according to claim 1, characterized in that: The heterogeneous indicators in step S1 include the memory fragmentation rate, computing unit utilization, and cross-node communication delay; Video memory fragmentation rate = (total video memory - maximum continuous video memory block) / total video memory × 100%; Computing unit utilization = actual number of computing units used / total number of computing units × 100%; Communication delay across nodes = round-trip delay in data transmission between nodes.

3. The method for adaptively deploying AI chips based on dynamic heterogeneous resource perception according to claim 1, characterized in that: The task request parameters in step S2 include video memory requirements, required computing unit type, and real-time level; the real-time level includes real-time tasks and non-real-time tasks; The greedy algorithm is started to select the target AI chip node in the cluster for the task and generate a scheduling plan as follows: Filter all AI chip nodes that meet the task request parameter requirements from the current cluster and calculate the comprehensive resource idleness of each AI chip node: Comprehensive resource idleness = a*(1-memory fragmentation rate) + b*(1-computing unit utilization) + c*(1-cross-node communication delay / maximum communication delay) Where a, b, and c are the weights of the video memory fragmentation rate, computing unit utilization, and cross-node communication delay, respectively, and a+b+c=1; From all AI chip nodes that meet the task request parameter requirements, select the node with the highest comprehensive resource idleness as the target AI chip node.

4. The method for adaptively deploying AI chips based on dynamic heterogeneous resource perception according to claim 3 is characterized in that: The specific steps for starting the improved genetic algorithm to generate a scheduling plan for the target AI chip node in the task selection cluster are as follows: The optimization goals are to minimize task completion time, maximize resource utilization and load balancing, and construct a fitness function: F = α*(1 / task estimated completion time) + β*resource balancing factor + γ*cluster resource utilization Among them, α is the task completion time weight, β is the resource balance weight, and γ is the utilization weight; AI chip nodes in the cluster are layered according to node type; Use the LSTM model to predict the resource requirements and execution time of the task, use the prediction results as the initial population of the genetic algorithm, and initialize the genetic algorithm parameters; the genetic algorithm parameters include the crossover rate and mutation rate; Starting from the initial population, the genetic algorithm is executed according to the genetic algorithm parameters to find the scheduling scheme that minimizes the fitness function. During the execution of the genetic algorithm, the node type is first coarsely screened and then finely screened according to the node, and the genetic algorithm parameters and fitness function weights are adjusted according to the load.

5. The method for adaptively deploying AI chips based on dynamic heterogeneous resource perception according to claim 4 is characterized in that: During the execution of the genetic algorithm, the first step is to coarsely screen according to the node type and then finely screen according to the node as follows: First, perform a coarse screening based on the node type layer to select the optimal node type. Then, perform a fine screening based on the selected node type to determine the target AI chip node. Adjusting genetic algorithm parameters and fitness function weights according to load includes: When the overall load of the cluster is greater than the first set ratio of the maximum load, the crossover rate of the genetic algorithm is increased to the upper limit of the crossover rate, and the mutation rate of the genetic algorithm is reduced to the lower limit of the mutation rate; When there is a second set ratio where the local load is greater than the maximum load, the resource balancing weight in the fitness function is increased according to the set amplitude.

6. The method for adaptively deploying AI chips based on dynamic heterogeneous resource perception according to claim 1, characterized in that: The specific steps for using the LSTM model to predict the computing resources required for the task request in step S3 are as follows: Collect historical task execution data, extract task request parameters and allocated computing resources during task execution, and train the LSTM model using task request parameters as input and allocated computing resources as output; Input the task request parameters in the task request into the trained LSTM model to predict the required computing resources; The specific steps for mapping reserved computing resources to Kubernetes schedulable resources are as follows: Create the corresponding Pod through the containerized interface Kubernetes API and inject hardware optimization parameters into the Pod annotation to map the required computing power resources of the target AI chip to Kubernetes schedulable resources.

7. The method for adaptively deploying AI chips based on dynamic heterogeneous resource perception according to claim 5, characterized in that: The specific steps of step S4 are as follows: S41. The scheduling controller obtains the memory fragmentation rate through the resource monitoring process of the AI chip during task execution, and when the memory fragmentation rate exceeds the threshold, initiates defragmentation, and dynamically migrates or merges memory blocks; S42. When the scheduling controller recognizes an increase in tasks, it determines whether a single AI chip can meet the computing resource requirements. If it cannot, it initiates distributed sharing of graphics memory or computing units to collaborate across nodes. S43. When the scheduling controller detects a failure in the target AI chip node during task execution, it starts the pre-assigned task copy and switches the task to the backup AI chip node.

8. An AI chip adaptive deployment system based on dynamic heterogeneous resource perception, characterized in that: include: The heterogeneous indicator collection module is used to deploy resource monitoring processes on AI chip nodes within the cluster, collect heterogeneous indicator data of AI chip nodes in real time, and upload it to the scheduling controller using the gRPC protocol at sampling intervals; The task scheduling module is used to receive task requests through the scheduling controller response, parse the task request parameters, and select the scheduling algorithm based on the real-time flag in the task request parameters: If it is a real-time task, a greedy algorithm is started to select the target AI chip node in the cluster to generate a scheduling plan for the task; If it is a non-real-time task, the improved genetic algorithm is started to generate a scheduling plan for the target AI chip node in the task selection cluster; The task allocation and containerized deployment module uses the LSTM model to predict the computing resources required for task requests, reserves the required computing resources on the target AI chip node, and maps the reserved computing resources to Kubernetes schedulable resources. The task adjustment module is used to dynamically adjust and optimize tasks based on the task execution status and heterogeneous indicator data of the target AI chip node through the scheduling controller.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for adaptive deployment of AI chips based on dynamic heterogeneous resource perception as described in any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the AI chip adaptive deployment method based on dynamic heterogeneous resource perception as described in any one of claims 1 to 7 are implemented.

Images