A virtualized resource scheduling method, device, and storage medium for high-concurrency AI tasks

Abstract

The present invention proposes a virtualized resource scheduling method, device, and storage medium for high-concurrency AI tasks. The method scans and analyzes the source code of the application program called by the server to generate an operation dependency table, and constructs a minimum containerized sandbox for the operation of the application program based on the dependency table. The method receives the AI ​​task input by the user in the cloud, inputs the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server that executes the task. On the server determined to execute the AI ​​task, the method calls the minimized containerized sandbox of the application program corresponding to the AI ​​task according to the priority of the AI ​​task, allocates server resources based on the AI ​​task, and loads other components of the application program. High-concurrency AI tasks refer to the simultaneous execution of more than 500 AI tasks on the server. The present invention realizes the ultra-dense scheduling capability of the server, greatly reduces the memory usage of the task environment, and improves system performance.

Description

Technical Field

[0001] The present invention relates to the field of the technology of integrating artificial intelligence and distributed computing, and specifically to a virtualized resource scheduling method, device and storage medium for high-concurrency AI tasks. Background Art

[0002] As enterprises accelerate their intelligentization process, AI-driven automation tasks are experiencing explosive growth. Existing technical solutions face significant challenges in handling high-concurrency, cross-platform tasks:

[0003] 1. Limitations of traditional virtualization technology: Containerization solutions such as Docker and Kubernetes require a complete runtime environment to be loaded into a single container, resulting in excessive memory usage (average ≥500MB / container). This makes it impossible to support large-scale concurrent AI tasks (typically ≤20 concurrent tasks per node). The resource isolation mechanism of virtual machine technology incurs additional overhead, resulting in task startup delays of seconds (measured average startup time in an ESXi environment is 3.2 seconds), making it difficult to meet the requirements of scenarios with high real-time requirements.

[0004] 2. Heterogeneous tool collaboration challenges: Cross-platform tasks fail to collaborate due to differences in data interfaces. Industry research shows that the failure rate of cross-system tasks is as high as 32%. Existing scheduling systems lack the ability to dynamically perceive tool dependencies, and fixed priority policies lead to resource contention.

[0005] Shortcomings in operation and maintenance monitoring: Traditional log systems have difficulty accurately associating faulty nodes in distributed task chains, and the average fault location time exceeds 45 minutes. The retry mechanism is simple and crude (full process rollback), resulting in up to 73% redundant calculations (MIT research data). Summary of the Invention

[0006] The present invention aims to solve one or more technical deficiencies in the above-mentioned prior art and proposes the following technical solutions.

[0007] A virtualized resource scheduling method for high-concurrency AI tasks, the method comprising:

[0008] A construction step of scanning and analyzing the source code of the application program to be called by the server to generate an operation dependency table, and constructing a minimum containerized sandbox for the operation of the application program based on the dependency table;

[0009] The scheduling step includes receiving the AI ​​task input by the user in the cloud, inputting the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server that executes the task;

[0010] The allocation step involves calling, on the server determined to execute the AI ​​task, a minimized containerized sandbox of the application corresponding to the AI ​​task according to the priority of the AI ​​task, and allocating server resources based on the AI ​​task and loading other components of the application, wherein a high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server.

[0011] Furthermore, the construction step is performed as follows: statically analyzing the source code of the application to identify runtime dependencies of the application, wherein the dependencies include dynamic link libraries, configuration files, and third-party plug-ins; executing the application in a simulated sandbox environment and dynamically tracing the application to obtain the resource call path during the execution of the application, wherein the resource call path includes system calls, network requests, and file reading and writing; generating a dependency table for the application based on the dependencies and resource call paths, wherein the core dependencies and optional dependencies of the application are marked in the dependency table; stripping non-essential runtime components from the base image of the application based on the dependency table; separating the core running components of the application from non-essential components based on a layered construction method, optimizing the memory footprint of the core running components based on memory compression technology to build a minimum containerized sandbox for the application, and when the AI ​​task calls the application, only the core running components of the application are loaded in the minimum containerized sandbox by default.

[0012] Furthermore, the training method of the intelligent scheduling model is: constructing a dependency graph of the dependency relationship between AI tasks, performing statistics on the historical execution logs of the application execution, determining the efficiency of the combined execution between applications, and constructing a graph neural network graph based on the dependency graph, with each AI task as a node in the graph neural network graph, and the set of execution parameters of all applications used to execute an AI task as the feature vector of the node corresponding to the AI ​​task. If there is a dependency relationship between two AI tasks, there is an edge between the nodes corresponding to the two AI tasks; based on the collected historical data of the execution of AI tasks, the graph neural network is trained after being cleaned and processed as a training sample, and the training is stopped after reaching a certain round or the loss function is less than a threshold, and the trained graph neural network is used as the intelligent scheduling model, wherein the feature vector includes: the size of the minimum containerized sandbox of each application, the CPU occupancy during application execution, and the execution time.

[0013] Furthermore, the eigenvalue of each node in the graph of the graph neural network is calculated based on the eigenvector of the node, and the calculation method is:

[0014] + + ;

[0015] in, represents the eigenvalue of the i-th node, represents the number of applications available to execute the AI ​​task corresponding to the i-th node, j represents the j-th application that executes the AI ​​task, 、 、 They represent the minimum sandbox size, CPU usage and execution time of the j-th application respectively. 、 、 Respectively The average minimized sandbox size of each application executing AI tasks, the average CPU occupancy rate during application execution, and the average execution time, where i≥2, ≥j≥1.

[0016] Furthermore, the weight of the edge in the graph of the graph neural network is defined as: if there is a dependency relationship between the AI ​​tasks corresponding to two nodes, the weight of the edge between the two nodes is 1, otherwise it is 0.

[0017] Furthermore, the operation of allocating server resources based on the AI ​​task is: installing a lightweight monitoring program on each server node to monitor the resource usage data of the server in real time, the resource usage data including the CPU, memory, and network data used by each AI task, obtaining the priority of the AI ​​task, and allocating server resources to the AI ​​task based on the priority.

[0018] Furthermore, the execution of the AI ​​task is monitored in real time, and recovery is performed when an error occurs during execution.

[0019] Furthermore, the specific operations include: generating a globally unique AI task ID for each AI, encrypting the AI ​​task ID, application identifier and timestamp to generate a fixed-length operation identifier; embedding the generator AI task ID and operation identifier in the metadata of the data unit; tracing the operation when an error occurs during execution: constructing a task DAG by recording the input and output relationship called by the distributed link tracking record tool; classifying the errors: performing instantaneous error detection through abnormal pattern matching and time window statistics, and marking the detected errors as retryable errors. If the error involves data format or permission issues, it is a persistent error and is marked as an error requiring manual intervention. For retryable error parsing task DAG, the direct upstream and downstream nodes of the faulty node are identified, and only the affected subgraph is rescheduled in the task DAG, and the output data of the upstream node is restored through checkpoints for incremental retries.

[0020] The present invention also proposes a virtualized resource scheduling device for high-concurrency AI tasks, which includes:

[0021] A construction unit scans and analyzes the source code of an application program to be called by the server to generate an operation dependency table, and constructs a minimum containerized sandbox for the application program to run based on the dependency table;

[0022] A scheduling unit receives AI tasks input by users in the cloud, inputs the parameters of the AI ​​tasks into a trained task intelligent scheduling model, and determines the server that executes the tasks;

[0023] An allocation unit, on a server determined to execute the AI ​​task, calls a minimized containerized sandbox of an application corresponding to the AI ​​task according to the priority of the AI ​​task, and allocates server resources based on the AI ​​task and loads other components of the application, wherein a high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server.

[0024] Furthermore, the operation of the construction unit is: statically analyzing the source code of the application to identify the runtime dependencies of the application, wherein the dependencies include dynamic link libraries, configuration files and third-party plug-ins; executing the application in a simulated sandbox environment and dynamically tracing the application to obtain the resource call path when the application is executed, wherein the resource call path includes system calls, network requests and file reading and writing; generating a dependency table of the application based on the dependencies and resource call paths, wherein the core dependencies and optional dependencies of the application are marked in the dependency table; stripping non-essential components of the runtime from the base image of the application based on the dependency table; separating the core running components of the application from non-essential components based on a layered construction method, optimizing the memory usage of the core running components based on memory compression technology to build a minimum containerized sandbox for the application, and when the AI ​​task calls the application, only the core running components of the application are loaded in the minimum containerized sandbox by default.

[0025] Furthermore, the training method of the intelligent scheduling model is: constructing a dependency graph of the dependency relationship between AI tasks, performing statistics on the historical execution logs of the application execution, determining the efficiency of the combined execution between applications, and constructing a graph neural network graph based on the dependency graph, with each AI task as a node in the graph neural network graph, and the set of execution parameters of all applications used to execute an AI task as the feature vector of the node corresponding to the AI ​​task. If there is a dependency relationship between two AI tasks, there is an edge between the nodes corresponding to the two AI tasks; based on the collected historical data of the execution of AI tasks, the graph neural network is trained after being cleaned and processed as a training sample, and the training is stopped after reaching a certain round or the loss function is less than a threshold, and the trained graph neural network is used as the intelligent scheduling model, wherein the feature vector includes: the size of the minimum containerized sandbox of each application, the CPU occupancy during application execution, and the execution time.

[0026] Furthermore, the eigenvalue of each node in the graph of the graph neural network is calculated based on the eigenvector of the node, and the calculation method is:

[0027] + + ;

[0028] in, represents the eigenvalue of the i-th node, represents the number of applications available to execute the AI ​​task corresponding to the i-th node, j represents the j-th application that executes the AI ​​task, 、 、 They represent the minimum sandbox size, CPU usage and execution time of the j-th application respectively. 、 、 Respectively The average minimized sandbox size of each application executing AI tasks, the average CPU occupancy rate during application execution, and the average execution time, where i≥2, ≥j≥1.

[0029] Furthermore, the weight of the edge in the graph of the graph neural network is defined as: if there is a dependency relationship between the AI ​​tasks corresponding to two nodes, the weight of the edge between the two nodes is 1, otherwise it is 0.

[0030] Furthermore, the operation of allocating server resources based on the AI ​​task is: installing a lightweight monitoring program on each server node to monitor the resource usage data of the server in real time, the resource usage data including the CPU, memory, and network data used by each AI task, obtaining the priority of the AI ​​task, and allocating server resources to the AI ​​task based on the priority.

[0031] Furthermore, the execution of the AI ​​task is monitored in real time, and recovery is performed when an error occurs during execution.

[0032] Furthermore, the specific operations include: generating a globally unique AI task ID for each AI, encrypting the AI ​​task ID, application identifier and timestamp to generate a fixed-length operation identifier; embedding the generator AI task ID and operation identifier in the metadata of the data unit; tracing the operation when an error occurs during execution: constructing a task DAG by recording the input and output relationship called by the distributed link tracking record tool; classifying the errors: performing instantaneous error detection through abnormal pattern matching and time window statistics, and marking the detected errors as retryable errors. If the error involves data format or permission issues, it is a persistent error and is marked as an error requiring manual intervention. For retryable error parsing task DAG, the direct upstream and downstream nodes of the faulty node are identified, and only the affected subgraph is rescheduled in the task DAG, and the output data of the upstream node is restored through checkpoints for incremental retries.

[0033] The present invention further provides a computer-readable storage medium, wherein the storage medium stores computer program code, and when the computer program code is executed by a computer, any one of the above methods is executed.

[0034] The technical effect of the present invention is: a virtualized resource scheduling method, device and storage medium for high-concurrency AI tasks of the present invention, construction step S101, scanning and analyzing the source code of the application for server-side call to generate an operation dependency table, and building a minimum containerized sandbox for the operation of the application based on the dependency table; scheduling step S102, receiving the AI ​​task input by the user in the cloud, inputting the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server that executes the task; allocation step S103, on the server determined to execute the AI ​​task, calling the minimized containerized sandbox of the application corresponding to the AI ​​task according to the priority of the AI ​​task, and allocating server resources based on the AI ​​task and loading other components of the application, wherein a high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server. In the present invention, a minimum containerized sandbox is first constructed based on the analysis of the application's source code, and then the server that executes the task is determined based on the parameters of the AI ​​task. Then, the corresponding sandbox is called on the server, and resources are allocated to the AI ​​task based on priority. Other components of the application can be loaded according to the needs of the AI ​​task, thereby achieving ultra-dense scheduling capabilities of a single physical node (server node), greatly reducing the memory usage of the task environment, and improving system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0036] Figure 1 This is a flowchart of a virtualized resource scheduling method for high-concurrency AI tasks according to an embodiment of the present invention.

[0037] Figure 2 This is a structural diagram of a virtualized resource scheduling device for high-concurrency AI tasks according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0039] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] Figure 1A virtualized resource scheduling method for high-concurrency AI tasks according to the present invention is shown, and the method includes:

[0041] Construction step S101, scanning and analyzing the source code of the application to be called by the server to generate an operation dependency table, and constructing a minimum containerized sandbox for the operation of the application based on the dependency table;

[0042] Scheduling step S102: receiving AI tasks input by users in the cloud, inputting parameters of the AI ​​tasks into the trained task intelligent scheduling model to determine the server that executes the tasks;

[0043] In step S103, on the server determined to execute the AI ​​task, the minimized containerized sandbox of the application corresponding to the AI ​​task is called according to the priority of the AI ​​task, and server resources are allocated based on the AI ​​task and other components of the application are loaded. A high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server.

[0044] In the present invention, in order to solve the limitations of traditional virtualization technology and the problem of collaboration with heterogeneous tools, the source code of the application called by the server is first scanned and analyzed to generate a running dependency table, and a minimum containerized sandbox for the application to run is constructed based on the dependency table, wherein the application called by the server is an application of the local system, also known as a heterogeneous tool. A server is connected to multiple local terminals, and one or more local programs are running on the local terminal. The image of the local program is encapsulated in a sandbox for the server to call, and then the cloud receives the AI ​​task input by the user, and inputs the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server to execute the task. The parameters of the AI ​​task include execution time requirements, completed functions, etc. Finally, on the server determined to execute the AI ​​task, the minimized containerized sandbox of the application corresponding to the AI ​​task is called according to the priority of the AI ​​task, and server resources are allocated based on the AI ​​task and other components of the application are loaded. That is, in the present invention, a minimum containerized sandbox is first constructed based on the analysis of the application's source code, and then the server that executes the task is determined based on the parameters of the AI ​​task. Then, the corresponding sandbox is called on the server, and resources are allocated to the AI ​​task based on priority. Other components of the application can be loaded according to the needs of the AI ​​task, thereby achieving an ultra-density scheduling capability of a single physical node (server node) to carry 500+ concurrent AI tasks, and reducing the memory usage of the task environment to less than 10% of the traditional solution. This is an important inventive concept of the present invention.

[0045] In one embodiment, the operation of the construction step S101 is: statically analyzing the source code of the application to identify the runtime dependencies of the application, wherein the dependencies include dynamic link libraries, configuration files, and third-party plug-ins; executing the application in a simulated sandbox environment and dynamically tracing the application to obtain the resource call path when the application is executed, wherein the resource call path includes system calls, network requests, and file reading and writing; generating a dependency table of the application based on the dependencies and resource call paths, wherein the core dependencies (must be loaded) and optional dependencies (loaded on demand) of the application are marked in the dependency table; stripping non-essential components of the runtime from the base image of the application based on the dependency table; separating the core running components of the application from non-essential components based on a layered construction method, optimizing the memory usage of the core running components based on memory compression technology to build a minimum containerized sandbox for the application, and when the AI ​​task calls the application, only the core running components of the application are loaded in the minimum containerized sandbox by default.

[0046] In the present invention, the application's runtime dependencies are identified by statically analyzing the application's source code, and the application is executed in a simulated sandbox environment and dynamically tracked to obtain the resource call path during the application's execution. Then, a dependency table for the application is generated based on the dependencies and resource call path, in which the application's core dependencies, which must be loaded, and optional dependencies, which are loaded on demand, are marked. Based on the dependency table, non-essential runtime components, such as GUI libraries and debugging tools, are stripped from the application's base image. The application's core runtime components are separated from non-essential components based on a layered construction method. Memory compression technology is used to optimize the memory footprint of the core runtime components to construct a minimal containerized sandbox for the application, achieving a single sandbox environment memory size of ≤5-100MB. This solves the problem in the prior art that a single container needs to load a complete runtime environment, resulting in excessive memory footprint and an inability to support large-scale AI task concurrency (single-node concurrency is typically ≤20). This enables the concurrent execution of more than 500 AI tasks on the same hardware, improving the internal performance of computer servers. This is another important aspect of the present invention.

[0047] In one embodiment, another important inventive concept of the present invention is that after the application minimum container is sandboxed, it is necessary to determine the optimal execution server according to the requirements of the AI ​​task. The present invention is determined based on an intelligent scheduling model. The intelligent scheduling model needs to be trained before use. The training method of the intelligent scheduling model is: constructing a dependency graph of the dependency relationship between AI tasks, statistics the historical execution logs of the application execution, and determine the efficiency of the combined execution between applications. A graph neural network graph is constructed based on the dependency graph, and each AI task is used as a node in the graph of the graph neural network. The set of execution parameters of all applications used to execute an AI task is used as a feature vector of the node corresponding to the AI ​​task. If there is a dependency relationship between two AI tasks, there is an edge between the nodes corresponding to the two AI tasks; based on the collected historical data of the execution of AI tasks, it is cleaned and processed as a training sample, and the graph neural network is trained. The training is stopped after reaching a certain round or the loss function is less than a threshold, and the trained graph neural network is used as the intelligent scheduling model, wherein the feature vector includes: the size of the minimum containerized sandbox of each application, the CPU occupancy during application execution, and the execution time.

[0048] The present invention trains the intelligent scheduling model based on a graph neural network, and constructs a graph of the graph neural network for AI tasks. During training, the historical data of AI task execution is cleaned and used as training samples for training. The training samples include the efficiency of combined execution between applications, real-time performance data of the server, applications that can be called by the server, the size of the application's minimized container sandbox, the average time it takes for an application to execute an AI task, and the CPU occupancy rate during application execution. The present invention creatively adds the efficiency of combined execution between applications to the training samples, thereby ensuring that the selected server can maximize the execution efficiency of AI tasks. This is another inventive concept of the present invention.

[0049] In one embodiment, the eigenvalue of each node in the graph of the graph neural network is calculated based on the eigenvector of the node, and the calculation method is:

[0050] + + ;

[0051] in, represents the eigenvalue of the i-th node, represents the number of applications available to execute the AI ​​task corresponding to the i-th node, j represents the j-th application that executes the AI ​​task, 、 、 They represent the minimum sandbox size, CPU usage and execution time of the j-th application respectively. 、 、 Respectively The average minimized sandbox size of each application executing AI tasks, the average CPU occupancy rate during application execution, and the average execution time, where i≥2, ≥j≥1.

[0052] In one embodiment, the weight of an edge in a graph of a graph neural network is defined as follows: if there is a dependency relationship between the AI ​​tasks corresponding to two nodes, the weight of the edge between the two nodes is 1, otherwise it is 0.

[0053] In order to train the graph neural network, the present invention proposes a specific method for setting the node feature values ​​and edge weights in the graph. The feature value of the node mainly considers the number of applications that execute the AI ​​task corresponding to the node, and determines the feature value based on the application's minimized sandbox size, the CPU occupancy during application execution, and the degree of deviation of the execution time from the corresponding average value, so that the trained model can select a server suitable for executing the current AI task. This is another important inventive point of the present invention.

[0054] In addition, the model is continuously adjusted during actual operation. For example, if a certain type of task frequently fails, the relevant resource allocation will be automatically avoided.

[0055] In one embodiment, the operation of allocating server resources based on the AI ​​task is as follows: installing a lightweight monitoring program on each server node to monitor the resource usage data of the server in real time, wherein the resource usage data includes the CPU, memory, and network data used by each AI task, obtaining the priority of the AI ​​task, and allocating server resources to the AI ​​task based on the priority. Allocating resources by priority can be: high-priority tasks (such as payment interfaces): guaranteeing a minimum resource quota, similar to a VIP channel; low-priority tasks (such as data analysis): allowing dynamic resource reduction or even suspension. A pause and resume mechanism is also provided, whereby low-priority tasks that occupy resources for a long time will be temporarily "frozen" (saving the current state to the hard disk) to release resources for urgent tasks; when recovery is required, the task will continue to run from the frozen point without having to start from the beginning. This solves the problem that the existing scheduling system in the background technology lacks the ability to dynamically perceive tool dependencies and adopts a fixed priority strategy that causes resource contention. This is another important inventive concept of the present invention.

[0056] In one embodiment, the execution of the AI ​​task is monitored in real time, and recovery is performed when an error occurs during execution. The specific operations include: generating a globally unique AI task ID for each AI, encrypting the AI ​​task ID, application identifier, and timestamp to generate a fixed-length operation identifier; embedding the generator AI task ID and operation identifier in the metadata of the data unit; tracing the operation source when an error occurs during execution: constructing a task DAG by recording the input and output relationship called by the distributed link tracking record tool; classifying the errors: performing instantaneous error detection through abnormal pattern matching and time window statistics, and marking the detected errors as retryable errors. If the error involves data format or permission issues, it is a persistent error and is marked as an error requiring manual intervention. For retryable error parsing task DAGs, the direct upstream and downstream nodes of the faulty node are identified, and only the affected subgraphs are rescheduled in the task DAG. The output data of the upstream node is restored through checkpoints for incremental retries to avoid full process rollback. This method is called DNA-style traceability tracking.

[0057] In the present invention, a task DAG is constructed by using the input-output relationship called by a distributed link tracking recording tool, and the monitored errors are divided into two categories: retryable errors and errors requiring manual intervention. For retryable errors, the task DAG is parsed to identify the direct upstream and downstream nodes of the faulty node. In the task DAG, only the affected subgraph is rescheduled, and the output data of the upstream node is restored through checkpoints for incremental retries to avoid full-process rollback. This solves the defects of traditional log systems in the background technology that it is difficult to accurately associate faulty nodes in a distributed task chain and that full-process rollback is time-consuming, thereby improving error detection efficiency and recovery speed, and further improving user experience. This is another important inventive concept of the present invention.

[0058] Figure 2 The present invention shows a virtualized resource scheduling device for high-concurrency AI tasks, which includes:

[0059] The construction unit 201 scans and analyzes the source code of the application program to be called by the server to generate an operation dependency table, and constructs a minimum containerized sandbox for the application program to run based on the dependency table;

[0060] The scheduling unit 202 receives the AI ​​task input by the user in the cloud, inputs the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server that executes the task;

[0061] The allocation unit 203 calls the minimized containerized sandbox of the application corresponding to the AI ​​task on the server determined to execute the AI ​​task according to the priority of the AI ​​task, allocates server resources based on the AI ​​task and loads other components of the application, wherein a high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server.

[0062] In the present invention, in order to solve the limitations of traditional virtualization technology and the problem of collaboration with heterogeneous tools, the source code of the application called by the server is first scanned and analyzed to generate a running dependency table, and a minimum containerized sandbox for the application to run is constructed based on the dependency table, wherein the application called by the server is an application of the local system, also known as a heterogeneous tool. A server is connected to multiple local terminals, and one or more local programs are running on the local terminal. The image of the local program is encapsulated in a sandbox for the server to call, and then the cloud receives the AI ​​task input by the user, and inputs the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server to execute the task. The parameters of the AI ​​task include execution time requirements, completed functions, etc. Finally, on the server determined to execute the AI ​​task, the minimized containerized sandbox of the application corresponding to the AI ​​task is called according to the priority of the AI ​​task, and server resources are allocated based on the AI ​​task and other components of the application are loaded. That is, in the present invention, a minimum containerized sandbox is first constructed based on the analysis of the application's source code, and then the server that executes the task is determined based on the parameters of the AI ​​task. Then, the corresponding sandbox is called on the server, and resources are allocated to the AI ​​task based on priority. Other components of the application can be loaded according to the needs of the AI ​​task, thereby achieving an ultra-density scheduling capability of a single physical node (server node) to carry 500+ concurrent AI tasks, and reducing the memory usage of the task environment to less than 10% of the traditional solution. This is an important inventive concept of the present invention.

[0063] In one embodiment, the operation of the construction unit 201 is: statically analyzing the source code of the application to identify the runtime dependencies of the application, wherein the dependencies include dynamic link libraries, configuration files, and third-party plug-ins; executing the application in a simulated sandbox environment and dynamically tracing the application to obtain the resource call path when the application is executed, wherein the resource call path includes system calls, network requests, and file reading and writing; generating a dependency table of the application based on the dependencies and resource call paths, wherein the core dependencies (must be loaded) and optional dependencies (loaded on demand) of the application are marked in the dependency table; stripping non-essential components of the runtime from the base image of the application based on the dependency table; separating the core running components of the application from non-essential components based on a layered construction method, optimizing the memory usage of the core running components based on memory compression technology to build a minimum containerized sandbox for the application, and when the AI ​​task calls the application, only the core running components of the application are loaded in the minimum containerized sandbox by default.

[0064] In the present invention, the application's runtime dependencies are identified by statically analyzing the application's source code, and the application is executed in a simulated sandbox environment and dynamically tracked to obtain the resource call path during the application's execution. Then, a dependency table for the application is generated based on the dependencies and resource call path, in which the application's core dependencies, which must be loaded, and optional dependencies, which are loaded on demand, are marked. Based on the dependency table, non-essential runtime components, such as GUI libraries and debugging tools, are stripped from the application's base image. The application's core runtime components are separated from non-essential components based on a layered construction method. Memory compression technology is used to optimize the memory footprint of the core runtime components to construct a minimal containerized sandbox for the application, achieving a single sandbox environment memory size of ≤5-100MB. This solves the problem in the prior art that a single container needs to load a complete runtime environment, resulting in excessive memory footprint and an inability to support large-scale AI task concurrency (single-node concurrency is typically ≤20). This enables the concurrent execution of more than 500 AI tasks on the same hardware, improving the internal performance of computer servers. This is another important aspect of the present invention.

[0065] In one embodiment, another important inventive concept of the present invention is that after the application minimum container is sandboxed, it is necessary to determine the optimal execution server according to the requirements of the AI ​​task. The present invention is determined based on an intelligent scheduling model. The intelligent scheduling model needs to be trained before use. The training method of the intelligent scheduling model is: constructing a dependency graph of the dependency relationship between AI tasks, statistics the historical execution logs of the application execution, and determine the efficiency of the combined execution between applications. A graph neural network graph is constructed based on the dependency graph, and each AI task is used as a node in the graph of the graph neural network. The set of execution parameters of all applications used to execute an AI task is used as a feature vector of the node corresponding to the AI ​​task. If there is a dependency relationship between two AI tasks, there is an edge between the nodes corresponding to the two AI tasks; based on the collected historical data of the execution of AI tasks, it is cleaned and processed as a training sample, and the graph neural network is trained. The training is stopped after reaching a certain round or the loss function is less than a threshold, and the trained graph neural network is used as the intelligent scheduling model, wherein the feature vector includes: the size of the minimum containerized sandbox of each application, the CPU occupancy during application execution, and the execution time.

[0066] The present invention trains the intelligent scheduling model based on a graph neural network, and constructs a graph of the graph neural network for AI tasks. During training, the historical data of AI task execution is cleaned and used as training samples for training. The training samples include the efficiency of combined execution between applications, real-time performance data of the server, applications that can be called by the server, the size of the application's minimized container sandbox, the average time it takes for an application to execute an AI task, and the CPU occupancy rate during application execution. The present invention creatively adds the efficiency of combined execution between applications to the training samples, thereby ensuring that the selected server can maximize the execution efficiency of AI tasks. This is another inventive concept of the present invention.

[0067] In one embodiment, the eigenvalue of each node in the graph of the graph neural network is calculated based on the eigenvector of the node, and the calculation method is:

[0068] + + ;

[0069] in, represents the eigenvalue of the i-th node, represents the number of applications available to execute the AI ​​task corresponding to the i-th node, j represents the j-th application that executes the AI ​​task, 、 、 They represent the minimum sandbox size, CPU usage and execution time of the j-th application respectively. 、 、 Respectively The average minimized sandbox size of each application executing AI tasks, the average CPU occupancy rate during application execution, and the average execution time, where i≥2, ≥j≥1.

[0070] In one embodiment, the weight of an edge in a graph of a graph neural network is defined as follows: if there is a dependency relationship between the AI ​​tasks corresponding to two nodes, the weight of the edge between the two nodes is 1, otherwise it is 0.

[0071] In order to train the graph neural network, the present invention proposes a specific method for setting the node feature values ​​and edge weights in the graph. The feature value of the node mainly considers the number of applications that execute the AI ​​task corresponding to the node, and determines the feature value based on the application's minimized sandbox size, the CPU occupancy during application execution, and the degree of deviation of the execution time from the corresponding average value, so that the trained model can select a server suitable for executing the current AI task. This is another important inventive point of the present invention.

[0072] In addition, the model is continuously adjusted during actual operation. For example, if a certain type of task frequently fails, the relevant resource allocation will be automatically avoided.

[0073] In one embodiment, the operation of allocating server resources based on the AI ​​task is as follows: installing a lightweight monitoring program on each server node to monitor the resource usage data of the server in real time, wherein the resource usage data includes the CPU, memory, and network data used by each AI task, obtaining the priority of the AI ​​task, and allocating server resources to the AI ​​task based on the priority. Allocating resources by priority can be: high-priority tasks (such as payment interfaces): guaranteeing a minimum resource quota, similar to a VIP channel; low-priority tasks (such as data analysis): allowing dynamic resource reduction or even suspension. A pause and resume mechanism is also provided, whereby low-priority tasks that occupy resources for a long time will be temporarily "frozen" (saving the current state to the hard disk) to release resources for urgent tasks; when recovery is required, the task will continue to run from the frozen point without having to start from the beginning. This solves the problem that the existing scheduling system in the background technology lacks the ability to dynamically perceive tool dependencies and adopts a fixed priority strategy that causes resource contention. This is another important inventive concept of the present invention.

[0074] In one embodiment, the execution of the AI ​​task is monitored in real time, and recovery is performed when an error occurs during execution. The specific operations include: generating a globally unique AI task ID for each AI, encrypting the AI ​​task ID, application identifier, and timestamp to generate a fixed-length operation identifier; embedding the generator AI task ID and operation identifier in the metadata of the data unit; tracing the operation source when an error occurs during execution: constructing a task DAG by recording the input and output relationship called by the distributed link tracking record tool; classifying the errors: performing instantaneous error detection through abnormal pattern matching and time window statistics, and marking the detected errors as retryable errors. If the error involves data format or permission issues, it is a persistent error and is marked as an error requiring manual intervention. For retryable error parsing task DAGs, the direct upstream and downstream nodes of the faulty node are identified, and only the affected subgraphs are rescheduled in the task DAG. The output data of the upstream node is restored through checkpoints for incremental retries to avoid full process rollback. This method is called DNA-style traceability tracking.

[0075] In the present invention, a task DAG is constructed by using the input-output relationship called by a distributed link tracking recording tool, and the monitored errors are divided into two categories: retryable errors and errors requiring manual intervention. For retryable errors, the task DAG is parsed to identify the direct upstream and downstream nodes of the faulty node. In the task DAG, only the affected subgraph is rescheduled, and the output data of the upstream node is restored through checkpoints for incremental retries to avoid full-process rollback. This solves the defects of traditional log systems in the background technology that it is difficult to accurately associate faulty nodes in a distributed task chain and that full-process rollback is time-consuming, thereby improving error detection efficiency and recovery speed, and further improving user experience. This is another important inventive concept of the present invention.

[0076] In one embodiment of the present invention, a computer storage medium is provided, on which a computer program is stored. When the computer program on the computer storage medium is executed by a processor, the above-mentioned method is implemented. The computer storage medium can be a hard disk, DVD, CD, flash memory or other memory.

[0077] For the convenience of description, the above device is described as being divided into various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0078] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the devices described in various embodiments of the present application or certain parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only intended to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A virtualized resource scheduling method for high-concurrency AI tasks, characterized by: The method includes: A construction step of scanning and analyzing the source code of the application program to be called by the server to generate an operation dependency table, and constructing a minimum containerized sandbox for the operation of the application program based on the dependency table; The scheduling step includes receiving the AI ​​task input by the user in the cloud, inputting the parameters of the AI ​​task into the trained task intelligent scheduling model to determine the server that executes the task; The allocation step involves calling, on the server determined to execute the AI ​​task, a minimized containerized sandbox of the application corresponding to the AI ​​task according to the priority of the AI ​​task, allocating server resources based on the AI ​​task and loading other components of the application, wherein a high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server. Among them, the operation of the construction step is: statically analyzing the source code of the application to identify the runtime dependencies of the application, wherein the dependencies include dynamic link libraries, configuration files and third-party plug-ins; executing the application in a simulated sandbox environment and dynamically tracing the application to obtain the resource call path when the application is executed, wherein the resource call path includes system calls, network requests and file reading and writing; generating a dependency table of the application based on the dependencies and resource call paths, wherein the core dependencies and optional dependencies of the application are marked in the dependency table; stripping non-essential components of the runtime from the base image of the application based on the dependency table; separating the core running components of the application from non-essential components based on a layered construction method, optimizing the memory usage of the core running components based on memory compression technology to build a minimum containerized sandbox for the application, and when the AI ​​task calls the application, only the core running components of the application are loaded in the minimum containerized sandbox by default, Among them, the training method of the intelligent scheduling model is: constructing a dependency graph of the dependency relationship between AI tasks, statistics the historical execution logs of the application execution, determine the efficiency of the combined execution between applications, and constructing a graph neural network graph based on the dependency graph. Each AI task serves as a node in the graph neural network graph. The set of execution parameters of all applications used to execute an AI task serves as the feature vector of the node corresponding to the AI ​​task. If there is a dependency relationship between two AI tasks, there is an edge between the nodes corresponding to the two AI tasks; based on the collected historical data of the execution of AI tasks, the data is cleaned and processed as a training sample to train the graph neural network. The training is stopped after reaching a certain round or the loss function is less than a threshold, and the trained graph neural network is used as the intelligent scheduling model, wherein the feature vector includes: the size of the minimum containerized sandbox of each application, the CPU occupancy during application execution, and the execution time.

2. The method according to claim 1, characterized in that The operation of allocating server resources based on the AI ​​task is: installing a lightweight monitoring program on each server node to monitor the resource usage data of the server in real time, the resource usage data including the CPU, memory, and network data used by each AI task, obtaining the priority of the AI ​​task, and allocating server resources to the AI ​​task based on the priority.

3. The method according to claim 2, characterized in that Monitor the execution of the AI ​​task in real time and recover when errors occur. Specifically, it includes: generating a globally unique AI task ID for each AI, encrypting the AI ​​task ID, application identifier and timestamp to generate a fixed-length operation identifier; embedding the generator AI task ID and operation identifier in the metadata of the data unit; tracing the operation source when an error occurs during execution: constructing a task DAG by recording the input and output relationship of the tool call through distributed link tracking; classifying the errors: performing instantaneous error detection through abnormal pattern matching and time window statistics, and marking the detected errors as retryable errors. If the error involves data format or permission issues, it is a persistent error and is marked as an error requiring manual intervention. For retryable error parsing task DAG, the direct upstream and downstream nodes of the faulty node are identified, and only the affected subgraph is rescheduled in the task DAG. The output data of the upstream node is restored through checkpoints for incremental retries.

4. A virtualized resource scheduling device for high-concurrency AI tasks, characterized in that: The device includes: A construction unit scans and analyzes the source code of an application program to be called by the server to generate an operation dependency table, and constructs a minimum containerized sandbox for the application program to run based on the dependency table; A scheduling unit receives AI tasks input by users in the cloud, inputs the parameters of the AI ​​tasks into a trained task intelligent scheduling model, and determines the server that executes the tasks; An allocation unit, on a server determined to execute the AI ​​task, calls a minimized containerized sandbox of an application corresponding to the AI ​​task according to the priority of the AI ​​task, allocates server resources based on the AI ​​task, and loads other components of the application, wherein a high-concurrency AI task refers to executing more than 500 AI tasks simultaneously on the server. Among them, the operation of the construction unit is: statically analyzing the source code of the application to identify the runtime dependencies of the application, wherein the dependencies include dynamic link libraries, configuration files and third-party plug-ins; executing the application in a simulated sandbox environment and dynamically tracing the application to obtain the resource call path when the application is executed, wherein the resource call path includes system calls, network requests and file reading and writing; generating a dependency table of the application based on the dependencies and resource call paths, wherein the core dependencies and optional dependencies of the application are marked in the dependency table; stripping non-essential components of the runtime from the base image of the application based on the dependency table; separating the core running components of the application from non-essential components based on a layered construction method, optimizing the memory usage of the core running components based on memory compression technology to build a minimum containerized sandbox for the application, and when the AI ​​task calls the application, only the core running components of the application are loaded in the minimum containerized sandbox by default, Among them, the training method of the intelligent scheduling model is: constructing a dependency graph of the dependency relationship between AI tasks, statistics the historical execution logs of the application execution, determine the efficiency of the combined execution between applications, and constructing a graph neural network graph based on the dependency graph. Each AI task serves as a node in the graph neural network graph. The set of execution parameters of all applications used to execute an AI task serves as the feature vector of the node corresponding to the AI ​​task. If there is a dependency relationship between two AI tasks, there is an edge between the nodes corresponding to the two AI tasks; based on the collected historical data of the execution of AI tasks, the data is cleaned and processed as a training sample to train the graph neural network. The training is stopped after reaching a certain round or the loss function is less than a threshold, and the trained graph neural network is used as the intelligent scheduling model, wherein the feature vector includes: the size of the minimum containerized sandbox of each application, the CPU occupancy during application execution, and the execution time.

5. The device according to claim 4, characterized in that The execution of the AI ​​task is monitored in real time, and recovery is performed when an error occurs during execution.

6. A computer storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program on the computer storage medium is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images