Artificial intelligence development platform resource scheduling method based on cloud native architecture

Abstract

The invention relates to the crossing field of artificial intelligence and cloud computing, and particularly discloses an artificial intelligence application development platform resource scheduling method based on a cloud native architecture, which comprises the following main steps: evaluating cloud native maturity and component reusability; carrying out multi-scale cloud native resource perception; performing artificial intelligence task demand measurement; and performing artificial intelligence task resource matching. According to the method, a multi-dimensional evaluation index system including the containerization rate, the pooling rate and the nano-management rate is constructed, and the single-dimensional limitation of traditional cloud native maturity evaluation is broken through; a joint optimization model of feature similarity matching and conflict detection is provided, and the resource fluctuation prediction precision is improved in combination with a sliding window algorithm. According to the method, through accurate resource pre-matching, the average completion time of a large-scale model training task is greatly shortened, and the operation and maintenance cost is reduced.

Description

Technical Field

[0001] The intersection of artificial intelligence and cloud computing specifically involves a resource scheduling method for an artificial intelligence application development platform that supports multiple scenarios, multiple tasks, and is scalable. Background Art

[0002] Complete AI development involves the entire development lifecycle, from algorithm frameworks to hardware adaptation. However, the fragmented and heterogeneous nature of the AI ​​technology ecosystem has led to significant discontinuities and fragmentation in its development frameworks and toolchains. AI development platforms need to integrate functional modules such as data management, model training, and deployment toolchains to reduce cross-system integration costs for developers. Some AI development platforms have already integrated algorithm libraries, distributed training, and inference engines, supporting the rapid implementation of various AI scenarios. Currently, most development platforms are built proprietary and closed, making them unable to effectively integrate various open-source AI tools. Furthermore, AI involves a long toolchain, each with varying frequency of use. Failure to effectively coordinate these tools will lead to wasted resources and high operating costs.

[0003] China's public invention CN112598940A proposes a full lifecycle management method covering the entire process of model training, optimization, and deployment. The model training optimization module improves efficiency through algorithm selection and parameter tuning, and the deployment module supports simulated production environment testing. By integrating the development and debugging tool chain, the threshold for developers is lowered. China's public invention CN202410762790.0 achieves functional decoupling through modules such as data quality management, multimodal support, and privacy and security protection, supports cross-modal data integration, and improves model generalization capabilities. However, the current patent only focuses on the combination of functional modules, relies heavily on private deployment, has limited cross-framework support capabilities, and lacks deep integration with the cloud-native tool chain. It also does not fully consider the elastic scaling requirements required for different tasks and algorithm training, making it difficult to cope with sudden computing power demands. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to propose a resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture, which can reduce the waiting time for training queues, improve user experience and resource utilization of the artificial intelligence development platform.

[0005] To solve the above technical problems, the present invention proposes a resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture, which includes the following main steps: Step 1: Assess cloud native maturity and component reusability. This involves establishing an evaluation metric system for heterogeneous physical computing and storage resources, including containerization rate, pooling rate, and management rate. This system verifies the resource's compatibility with the AI ​​framework and the reusability of AI development components. Step 2: Multi-scale cloud-native resource awareness, namely, real-time collection of basic metrics such as CPU / GPU utilization, storage IOPS, and network bandwidth. This includes monitoring Pod resource quotas, container lifecycle events, and inter-service call latency. This includes assessing resource volatility based on a sliding window and predicting resource consumption. Step 3: Measure AI task requirements. This involves setting tracking points to measure, model, and track AI task resource requirements. Step 4: AI task resource matching, that is, matching the feature similarity between tasks and cloud native resource nodes, then calculating the pre-allocated resource conflicts, and finally determining the multi-objective optimization function for task resource matching.

[0006] Furthermore, the step 1 includes the following sub-steps: Step 1-1: Count the total number of CPUs, GPUs / NPUs, and total physical storage capacity of the AI ​​development platform's physical servers; Step 1-2: Use the cloud native container management agent to collect the number of containers and virtualizations of the AI ​​development platform and the distributed pooled capacity of storage resources; Step 1-3: Evaluate the standardization ratio of heterogeneous resource container images; Steps 1-4: Evaluate the cloud-native framework's mapping and support coverage capabilities for the AI ​​framework and NCCL communication set. Steps 1-5: Build a capability assessment matrix for the cloud-native AI development platform ; Where η is the standardized ratio of heterogeneous resource container images, and F is the cloud native framework coverage capability index.

[0007] Furthermore, step 2 includes the following sub-steps: Step 2-1: Establish monitoring models for virtual machines, containers, services, and functions in order of virtualization granularity, and define them as quadruple objects: Step 2-2: Calculate the load fluctuation rate of the measurement perception under the current scale conditions, and adjust the measurement perception granularity according to the load fluctuation rate; Step 2-3: To achieve cloud-native resource awareness in advance, use the ARIMA-LSTM hybrid regression model to predict resource consumption trends; Step 2-4: Integrate the characteristics of resource perception results at each scale and establish cross-scale resource dependency relationships.

[0008] Furthermore, step 3 includes the following sub-steps: Step 3-1: Record the AI ​​task type, data type, and data size; Step 3-2: Decompose the AI ​​task pipeline calculation graph and set the time and space points of the task behavior. Step 3-3: When the task execution reaches the spatiotemporal embedding point, task demand measurement is triggered. Metrics including CPU / GPU utilization, video memory usage, I / O throughput, response latency, and error rate are collected through the cloud-native architecture performance monitoring API. The weighted calculation formula is dynamically adjusted based on the model accuracy. Step 3-4: Record the resource consumption patterns of historical tasks and establish a resource demand mapping relationship between the AI ​​tasks and response latency and error rate in step 3-1 to provide a reference for resource estimation of new tasks; Steps 3-5: Based on the embedded data, use the random forest regression algorithm to predict resources. Use the input data dimension and concurrent request volume as feature variables to predict GPU memory usage and computing time.

[0009] Furthermore, step 4 includes the following sub-steps: Step 4-1: Establish cosine similarity between multi-scale cloud native resource characteristics and AI task resource requirement characteristics; Step 4-2: Establish a resource competition matrix to quantify the probability of resource conflicts between nodes in the cloud native resource pool, and select node combinations with low resource conflict probability for task resource matching. Step 4-3: Construct a multi-objective optimization model for dynamic task resource matching, and minimize the comprehensive task resource cost as the final task resource matching goal.

[0010] This paper constructs a multi-dimensional evaluation indicator system including containerization rate, pooling rate and management rate, breaking through the single-dimensional limitation of traditional cloud native maturity assessment. It combines the sliding window algorithm to improve the accuracy of resource fluctuation prediction and proposes a joint optimization model of feature similarity matching and conflict detection.

[0011] Through precise resource pre-matching, this invention significantly shortens the average completion time of large-scale model training tasks and reduces operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0013] Figure 1 It is a flowchart of the overall process of the present invention. DETAILED DESCRIPTION

[0014] The present invention proposes a resource scheduling method for a general artificial intelligence application development platform based on a cloud native architecture. The general artificial intelligence application development platform involved is composed of an infrastructure layer, a resource management layer, a task scheduling layer, and an application service layer.

[0015] Infrastructure layer: Build an elastic resource pool based on containerization technology, integrate CPU / GPU heterogeneous computing power, distributed storage, and network acceleration components, and realize unified resource orchestration through service grid.

[0016] Resource management layer: Adopts a multi-scale perception mechanism to monitor cluster node resources (CPU / memory / GPU memory), storage IOPS, network bandwidth and other indicators in real time, and builds a visual monitoring system.

[0017] Task scheduling layer: Integrates AI development framework and distributed training tools, supporting automatic scaling and priority queue scheduling.

[0018] Application service layer: Provides functional modules such as visual modeling, API service orchestration, and model version management to support end-to-end AI development and training processes.

[0019] like Figure 1 As shown, the specific steps and processes of the present invention are composed of four key steps: Step 1: Assess cloud native maturity and component reusability. This step establishes an evaluation index system for heterogeneous physical computing and storage resources, including containerization rate, pooling rate, and management rate, to verify the resource's adaptability to the AI ​​framework and the reusability of AI development components. This step specifically includes the following sub-steps: Step 1-1: Count the total number of CPUs, GPUs / NPUs, and total physical storage capacity of the AI ​​development platform's physical servers; Step 1-2: Use the cloud native container management agent to collect the number of containers and virtualizations of the AI ​​development platform and the distributed pooled capacity of storage resources; Step 1-3: Evaluate the normalized ratio of heterogeneous resource container images according to Formula 1; (1) in is the number of CPU cores in all physical server resource pools, The number of CUDA cores in the GPUs of all physical server resource pools. and are the number of CPU cores and GPU CUDA cores of the virtual server resources allocated to the i-th container respectively.

[0020] Steps 1-4: Evaluate the cloud native framework's mapping and support coverage capabilities for AI frameworks such as Pytorch and TensorFlow and NCCL communication sets based on Equation 2. (2) Steps 1-5: Based on steps 1-3 and 1-4 above, build a capability assessment matrix for the cloud-native AI development platform. , Where η is the standardized ratio of heterogeneous resource container images, F is the cloud native framework coverage capability index, , N is the total number of containers,

[0021] Mapping support coverage for PyTorch, Mapping support coverage for TensorFlow, is the coverage of the NCCL communication collection library, is the framework weight coefficient, and satisfies .

[0022] Step 2: Multi-scale cloud native resource awareness. This step collects basic metrics such as CPU / GPU utilization, storage IOPS, and network bandwidth in real time, monitors Pod resource quotas, container lifecycle events, and inter-service call latency, evaluates resource volatility based on a sliding window, and predicts resource consumption. This step includes the following sub-steps: Step 2-1: Establish monitoring models for virtual machines, containers, services, and functions in order of virtualization granularity, and define them as four-tuple object features:

[0023] The VM is the virtual machine as the monitoring object, and the monitoring dimensions include: , is the virtual CPU utilization (%), is the memory allocation / usage (GB), Disk IOPS peak virtualization, Network interface packet loss rate (%), Container monitors containers, including request resource quotas. , image hash , life cycle event status , which can be expressed as

[0024] Service takes service as the monitoring object, and the monitoring dimensions include request latency , response error rate , maximum service capacity per second , which can be expressed as

[0025] Function takes the function as the monitoring object, and the monitoring dimensions include the average cold start time , Number of timeouts , average peak utilization , which can be expressed as

[0026] Step 2-2: Calculate the measured perceived load fluctuation rate under the current scale conditions according to the following formula

[0027] in, is the load measurement value at time t, is the current time window size, is the current scale granularity (unit: measurement unit / time), is the granularity adjustment step, and n is the number of sampling points in the window. The load average formula is as follows: Based on the load fluctuation rate, the measurement perception granularity is adjusted according to the following strategy:

[0028] in , , the particle size level is , the granularity is divided into exponential ladder levels .

[0029] If the load changes drastically, , according to the above formula, the measurement perception scale is adjusted to fine granularity. On the contrary, if the change is stable, that is, , adjust the measurement perception scale to coarse granularity according to the above formula.

[0030] Step 2-3: To achieve cloud-native resource awareness in advance, use the ARIMA-LSTM hybrid regression model (an industry standard method) to predict resource consumption trends.

[0031] Step 2-4: Fusion the characteristics of resource perception results at each scale according to the following formula, and establish cross-scale resource dependency relationships.

[0032]

[0033] in To fusion features, is the resource perception feature collected at the sth scale, that is, the four-tuple object feature measured in step 2-1, is the weight of the s-th scale feature.

[0034] Step 3: Measure and measure AI task requirements. This step measures, models, and tracks AI task resource requirements by setting tracking points. This step includes the following sub-steps: Step 3-1: Record the AI ​​task type, data type, and data size; Step 3-2: Decompose the AI ​​task pipeline calculation graph and set the time and space points of the task behavior. Step 3-3: When the task execution reaches the spatiotemporal embedding point, task demand measurement is triggered. Metrics such as CPU / GPU utilization, video memory usage, and I / O throughput are collected through the cloud-native architecture performance monitoring API. The weighted calculation formula is dynamically adjusted based on the model accuracy (FP16 / INT8). Step 3-4: Record the resource consumption patterns of historical tasks and establish a resource demand mapping relationship between the AI ​​task in step 3-1 and the response delay and error rate in step 3-3 to provide a reference for resource estimation of new tasks; Steps 3-5: Based on the embedded data, use the random forest regression algorithm (an industry standard method) to predict resources. Using the input data dimension and concurrent request volume as feature variables, we predict GPU memory usage and computation time. Step 4: AI task resource matching. This step matches the feature similarity between tasks and cloud-native resource nodes, calculates pre-allocated resource conflicts, and ultimately determines the multi-objective optimization function for task resource matching. This step includes the following sub-steps: Step 4-1: Establish the cosine similarity between the multi-scale cloud native resource features in step 2-4 and the artificial intelligence task resource demand features obtained in step 3-5.

[0035] in is the cosine similarity, is the feature vector of the i-th cloud native resource, is the resource requirement feature vector of the j-th artificial intelligence task.

[0036] Step 4-2: Establish a resource competition matrix to quantify the probability of resource conflicts between nodes in the cloud-native resource pool, and select node combinations with low resource conflict probabilities for task resource matching.

[0037]

[0038] in and is the resource occupancy characteristic of two competing tasks k and l, and is the variance of resource occupancy characteristics of two competing tasks k and l, is the covariance coefficient of the two; Step 4-3: Construct a multi-objective optimization model for dynamic task resource matching, and minimize the comprehensive task resource cost as the final task resource matching goal.

[0039]

[0040] in is the resource requirement of the i-th task, is the total available resources, is the waiting delay of the i-th task.

[0041] The above-mentioned resource scheduling method for the artificial intelligence development platform based on cloud-native architecture realizes elastic scaling of heterogeneous resources and improves dynamic resource utilization through containerization rate evaluation in step 1 and multi-dimensional resource perception in step 2. Based on task feature modeling in step 3 and multi-target matching in step 4, intelligent scheduling is realized to reduce the waiting time of AI training tasks.

[0042] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture, characterized in that: The main steps include: Step 1: Assess cloud native maturity and component reusability. This involves establishing an evaluation metric system for heterogeneous physical computing and storage resources, including containerization rate, pooling rate, and management rate. This system verifies the resource's compatibility with the AI ​​framework and the reusability of AI development components. Step 2: Multi-scale cloud-native resource awareness, namely, real-time collection of basic metrics such as CPU / GPU utilization, storage IOPS, and network bandwidth, monitoring of Pod resource quotas, container lifecycle events, and inter-service call latency, assessing resource volatility based on a sliding window, and predicting resource consumption. Step 3: Measure AI task requirements. This involves setting tracking points to measure, model, and track AI task resource requirements. Step 4: AI task resource matching, that is, matching the feature similarity between tasks and cloud native resource nodes, then calculating the pre-allocated resource conflicts, and finally determining the multi-objective optimization function for task resource matching.

2. The resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture according to claim 1, characterized in that: The step 1 includes the following sub-steps: Step 1-1: Count the total number of CPUs, GPUs / NPUs, and total physical storage capacity of the AI ​​development platform's physical servers; Step 1-2: Use the cloud native container management agent to collect the number of containers and virtualizations of the AI ​​development platform and the distributed pooled capacity of storage resources; Step 1-3: Evaluate the standardization ratio of heterogeneous resource container images; Steps 1-4: Evaluate the cloud-native framework's mapping and support coverage capabilities for the AI ​​framework and NCCL communication set. Steps 1-5: Build a capability assessment matrix for the cloud-native AI development platform ; Where η is the standardized ratio of heterogeneous resource container images, and F is the cloud native framework coverage capability index.

3. The resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture according to claim 1, characterized in that: The step 2 includes the following sub-steps: Step 2-1: Establish monitoring models for virtual machines, containers, services, and functions in order of virtualization granularity, and define them as quadruple objects: Step 2-2: Calculate the load fluctuation rate of the measurement perception under the current scale conditions, and adjust the measurement perception granularity according to the load fluctuation rate; Step 2-3: To achieve cloud-native resource awareness in advance, use the ARIMA-LSTM hybrid regression model to predict resource consumption trends; Step 2-4: Integrate the characteristics of resource perception results at each scale and establish cross-scale resource dependency relationships.

4. The resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture according to claim 1, characterized in that: Step 3 includes the following sub-steps: Step 3-1: Record the AI ​​task type, data type, and data size; Step 3-2: Decompose the AI ​​task pipeline calculation graph and set the time and space points of the task behavior. Step 3-3: When the task execution reaches the spatiotemporal embedding point, task demand measurement is triggered. Metrics including CPU / GPU utilization, video memory usage, I / O throughput, response latency, and error rate are collected through the cloud-native architecture performance monitoring API. The weighted calculation formula is dynamically adjusted based on the model accuracy. Step 3-4: Record the resource consumption patterns of historical tasks and establish a resource demand mapping relationship between the AI ​​tasks and response latency and error rate in step 3-1 to provide a reference for resource estimation of new tasks; Steps 3-5: Based on the embedded data, use the random forest regression algorithm to predict resources. Use the input data dimension and concurrent request volume as feature variables to predict GPU memory usage and computing time.

5. The resource scheduling method for an artificial intelligence application development platform based on a cloud native architecture according to claim 1, characterized in that: The step 4 includes the following sub-steps: Step 4-1: Establish cosine similarity between multi-scale cloud native resource characteristics and AI task resource requirement characteristics; Step 4-2: Establish a resource competition matrix to quantify the probability of resource conflicts between nodes in the cloud native resource pool, and select node combinations with low resource conflict probability for task resource matching. Step 4-3: Construct a multi-objective optimization model for dynamic task resource matching, and minimize the comprehensive task resource cost as the final task resource matching goal.

Images