Resource scheduling method and corresponding device

Abstract

The present application provides a resource scheduling method, comprising: determining at least one target subgraph from an artificial intelligence (AI) model, wherein the at least one target subgraph is a subgraph, which satisfies typicality, within a computation graph of the AI model, an operation resource associated with the at least one target subgraph matches an execution demand of the AI model, and the typicality comprises the frequency of occurrence of the same subgraph being higher than a first threshold; and on the basis of the at least one target subgraph, scheduling the operation resource of the AI model. In this way, a complex AI model can be replaced with target subgraphs having typicality, and then resources are scheduled on the basis of the target subgraphs, such that operation resources of the AI model can be scheduled more accurately and finer, thereby increasing the utilization rate of resources during training or reasoning of the AI model.

Description

A method and corresponding device for resource scheduling

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 20, 2024, with application number 202410327115.5 and application name “A method and corresponding device for resource scheduling”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a resource scheduling method and corresponding device. Background Art

[0003] With the development of AI, AI models are becoming more and more complex and large. Training AI models or using AI models to perform inference processes requires scheduling resources in the cloud system (such as computing resources, storage resources, communication resources, etc.).

[0004] Since the AI ​​model training or inference process usually requires the participation of multiple nodes in the cloud system, access or communication between different nodes usually encounters bottlenecks. If resource scheduling is unreasonable, it will affect the training speed or inference speed of the AI ​​model.

[0005] Therefore, how to reasonably allocate resources during AI model training or inference has become an urgent problem to be solved. Summary of the Invention

[0006] This application provides a resource scheduling method for improving resource utilization during AI model training or inference. This application also provides corresponding devices, computer-readable storage media, and computer program products.

[0007] In a first aspect, the present application provides a method for resource scheduling, comprising: determining at least one target subgraph from an artificial intelligence (AI) model; wherein the at least one target subgraph is a subgraph in a computational graph corresponding to the AI ​​model that satisfies typicality, and the operating resources associated with the at least one target subgraph match the execution requirements of the AI ​​model, and the typicality includes that the frequency of occurrence of the same subgraph is higher than a first threshold; and scheduling the operating resources of the AI ​​model based on the at least one target subgraph.

[0008] The resource scheduling method provided in this application can be applied to cloud systems, independent servers, or terminal devices. For example, when training an AI model, it is necessary to schedule resources in the cloud system for the AI ​​model to complete the iterative training process of the AI ​​model using sample data. When using the AI ​​model for inference, it is also necessary to schedule resources in the cloud system for the AI ​​model to complete the inference of the input data and obtain the inference results.

[0009] In this application, at least one (or more) target subgraphs are subgraphs in the computation graph of the AI ​​model that meet the typicality requirement. In other words, the target subgraphs are subgraphs whose occurrence frequency in the computation graph is higher than a first threshold. The first threshold can be a preset value based on demand or a value dynamically determined based on the complexity of the AI ​​model.

[0010] In the present application, typicality may include not only the case where the frequency of occurrence of a subgraph is higher than the first threshold, but also other typical cases, such as: the first subgraph includes the second subgraph, and the first subgraph may also be understood as a typical subgraph; or, although the frequency of occurrence of the third subgraph does not reach the first threshold, the input tensor or output tensor of the operator in the third subgraph has a greater impact on the execution of other subgraphs, etc. This application does not limit this, and the content of typicality may be set according to needs.

[0011] The resources of this application may include computing resources, storage resources, communication resources, and other resources used to train AI models or use AI models for reasoning.

[0012] In the first aspect mentioned above, by determining at least one target subgraph from the AI ​​model, a complex AI model can be replaced by a typical target subgraph, and then resources can be scheduled based on the target subgraph. The operating resources of the AI ​​model can be scheduled more accurately and fine-grained, thereby improving resource utilization during AI model training or inference.

[0013] In one possible implementation, the above steps: scheduling the running resources of the AI ​​model based on at least one target subgraph, include: determining a target subgraph pattern corresponding to at least one target subgraph, the target subgraph pattern being used to indicate the model structure of the AI ​​model represented by at least one target subgraph and the running strategy of the AI ​​model; determining the target resource information corresponding to the target subgraph pattern according to the correspondence between the subgraph pattern and the resource information, the target resource information being used to indicate the running resources of the AI ​​model; and scheduling the running resources indicated by the target resource information for the AI ​​model.

[0014] In this application, once the AI ​​model is determined, the model structure of the AI ​​model is determined. The structure of the AI ​​model may include the number of layers of the AI ​​model, the type of each layer, computation-intensive or memory-intensive, etc.

[0015] In this application, the AI ​​model's operational strategy may include the concurrency / parallelism strategy and communication strategy of the operators in the AI ​​model's computation graph. The AI ​​model's operational strategy can be determined by the operator type in the target subgraph. In addition, the target subgraph mode can be determined by combining the structure of the AI ​​model.

[0016] In this application, the subgraph pattern may be in the form of an index or an identifier, and different subgraph patterns may correspond to different combinations of model structures and operation strategies.

[0017] In this application, the correspondence between subgraph patterns and resource information can be a relationship library established by multiple different AI models, or a correspondence established for a single AI model when training or using a single AI model for reasoning. A relationship library usually includes multiple correspondences.

[0018] In this possible implementation, the target subgraph pattern can be used as an index to find the target resource information corresponding to the target subgraph pattern from the corresponding relationship between the subgraph pattern and the resource information. Then, the running resources for the AI ​​model can be scheduled based on the target resource information. This not only allows for accurate scheduling of the AI ​​model's running resources, but also speeds up the scheduling of running resources.

[0019] In one possible implementation, the above steps of: scheduling the running resources of the AI ​​model based on at least one target subgraph, include: determining the running strategy of the AI ​​model according to the type of operator in at least one target subgraph; scheduling the running resources of the AI ​​model according to the model structure of the AI ​​model and the running strategy of the AI ​​model.

[0020] In this possible implementation, the AI ​​model's execution strategy can be determined based on the operator type in at least one target subgraph, and the AI ​​model's execution resources can be scheduled based on the AI ​​model's structure. This approach of determining the execution strategy based on the operator type of a typical target subgraph can improve the accuracy of the execution strategy, thereby improving the accuracy of execution resource scheduling and increasing resource utilization.

[0021] In one possible implementation, the resource information in the corresponding relationship is used to indicate the proportion or size of different types of resources in the running resources; or, the resource information in the corresponding relationship includes identification information and / or address information of different types of resources in the running resources; wherein, the running resources include computing resources, as well as storage resources and / or communication resources.

[0022] In this possible implementation, if the resource information is used to indicate the proportion or size of different types of resources in the running resources, then when scheduling resources, appropriate resources can be selected for scheduling according to the proportion or size of different types of resources. If the resource information includes identification information of different types of resources in the running resources (such as: identification information of the central processing unit (CPU), graphics processing unit (GPU), neural-network processing unit (NPU)) and / or address information (such as: address information of the storage location or communication address information, etc.), the resources indicated by the corresponding identification information and / or address information are scheduled. In the present application, scheduling resources according to the instructions of the resource information can improve the speed and accuracy of resource scheduling.

[0023] In a possible implementation, the operating resources are virtual resources and / or physical resources.

[0024] In one possible implementation, the operating resources indicated by the target resource information are a set of resources that are estimated to take the shortest time to execute the AI ​​model.

[0025] In this possible implementation, the target resource information can be determined by sampling different resource combinations to estimate the time required to execute the AI ​​model. The information of the group of resources with the shortest time is selected as the target resource information, which can improve the resource utilization in the actual AI model training process or the AI ​​model reasoning process.

[0026] In one possible implementation, the above steps of determining at least one target subgraph from an artificial intelligence AI model include converting the AI ​​model into a computational graph, and determining multiple subgraphs from the computational graph; processing the multiple subgraphs to obtain at least one target subgraph that satisfies typicality.

[0027] In the present application, the AI ​​model can be converted into a computational graph by identifying the operators in the code of the AI ​​model and the association relationship between the operators. Then, the computational graph can be traversed to find multiple subgraphs that meet the constraints. Then, the multiple subgraphs can be pruned (for example, only one copy of a repeated subgraph in multiple subgraphs can be retained, and the frequency of occurrence of the subgraph can be marked, or, for subgraphs with an inclusion relationship, the inclusion relationship can be marked on a larger subgraph) to obtain at least one target subgraph.

[0028] In this possible implementation, converting the computational graph of the AI ​​model helps to quickly determine the target subgraph in the AI ​​model.

[0029] In one possible implementation, each subgraph in the multiple subgraphs satisfies correctness and connectivity. Correctness means that each subgraph needs to conform to the computational logic of the AI ​​model, and connectivity means that any two operators in each subgraph are directly or indirectly connected.

[0030] In this possible implementation, each of the multiple subgraphs meets the requirements of correctness and connectivity, which is beneficial to improving the credibility of the target subgraph.

[0031] In one possible implementation, multiple subgraphs need to satisfy completeness; wherein, completeness means that no subgraphs that satisfy correctness and connectivity in the computation graph are omitted.

[0032] In this possible implementation, multiple subgraphs need to meet integrity requirements, which can improve the credibility of at least one target subgraph in representing the AI ​​model.

[0033] A second aspect of the present application provides a resource scheduling device, comprising:

[0034] a first processing unit, configured to determine at least one target subgraph from an artificial intelligence (AI) model; wherein the at least one target subgraph is a subgraph in a computation graph corresponding to the AI ​​model that satisfies typicality, and the runtime resources associated with the at least one target subgraph match the execution requirements of the AI ​​model, wherein the typicality includes a frequency of occurrence of the same subgraph exceeding a first threshold;

[0035] The second processing unit is used to schedule the running resources of the AI ​​model based on at least one target subgraph.

[0036] In one possible implementation, the second processing unit is specifically used to: determine a target subgraph pattern corresponding to at least one target subgraph, the target subgraph pattern is used to indicate the model structure of the AI ​​model represented by at least one target subgraph and the operation strategy of the AI ​​model; determine the target resource information corresponding to the target subgraph pattern according to the correspondence between the subgraph pattern and the resource information, the target resource information is used to indicate the operation resources of the AI ​​model; and schedule the operation resources indicated by the target resource information for the AI ​​model.

[0037] In one possible implementation, the second processing unit is specifically used to: determine the operation strategy of the AI ​​model based on the type of operator in at least one target subgraph; and schedule the operation resources of the AI ​​model based on the model structure of the AI ​​model and the operation strategy of the AI ​​model.

[0038] In one possible implementation, the resource information in the corresponding relationship is used to indicate the proportion or size of different types of resources in the running resources; or, the resource information in the corresponding relationship includes identification information and / or address information of different types of resources in the running resources; wherein, the running resources include computing resources, as well as storage resources and / or communication resources.

[0039] In a possible implementation, the operating resources are virtual resources and / or physical resources.

[0040] In one possible implementation, the operating resources indicated by the target resource information are a set of resources that are estimated to take the shortest time to execute the AI ​​model.

[0041] In one possible implementation, the first processing unit is specifically used to: convert the AI ​​model into a computational graph, and determine multiple subgraphs from the computational graph; process the multiple subgraphs to obtain at least one target subgraph that meets typicality.

[0042] In one possible implementation, each subgraph in the multiple subgraphs satisfies correctness and connectivity. Correctness means that each subgraph needs to conform to the computational logic of the AI ​​model, and connectivity means that any two operators in each subgraph are directly or indirectly connected.

[0043] In one possible implementation, multiple subgraphs need to satisfy completeness; wherein, completeness means that no subgraphs that satisfy correctness and connectivity in the computation graph are omitted.

[0044] A third aspect of the present application provides a computer device, which includes a processor and a computer-readable storage medium storing a computer program; the processor is coupled to the computer-readable storage medium, and when the computer program is executed by the processor, it implements the method as described in the first aspect or any possible implementation method.

[0045] The fourth aspect of the present application provides a computer-readable storage medium storing one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method as described in the first aspect or any possible implementation of the first aspect.

[0046] The fifth aspect of the present application provides a computer program product that stores one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method of the above-mentioned first aspect or any possible implementation of the first aspect.

[0047] A sixth aspect of the present application provides a chip system, which includes a processor for supporting a computer device to implement the functions involved in the above-mentioned first aspect or any possible implementation of the first aspect. In one possible design, the chip system may also include a memory for storing program instructions and data necessary for training nodes. The chip system may be composed of a chip or may include a chip and other discrete devices.

[0048] Among them, the technical effects brought about by the second aspect or any possible implementation method of the second aspect to the sixth aspect can refer to the technical effects brought about by the first aspect or different possible implementation methods of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1A is a schematic diagram of an AI model training scenario;

[0050] Figure 1B is a schematic diagram of an AI model reasoning scenario;

[0051] FIG2A is a schematic diagram of a structure of a cloud system provided in an embodiment of the present application;

[0052] FIG2B is another schematic diagram of the structure of the cloud system provided in an embodiment of the present application;

[0053] FIG2C is a schematic diagram of a structure of a data center provided in an embodiment of the present application;

[0054] FIG3 is a schematic diagram of a structure of a resource scheduling device provided in an embodiment of the present application;

[0055] FIG4 is a schematic diagram of an embodiment of a resource scheduling method provided in an embodiment of the present application;

[0056] FIG5A is an example diagram of a code fragment of an AI model provided in an embodiment of the present application;

[0057] FIG5B is a computational diagram corresponding to a code snippet of an AI model provided in an embodiment of the present application;

[0058] FIG6A is an example diagram of a subgraph provided in an embodiment of the present application;

[0059] FIG6B is an example diagram of an incorrect sub-image provided by an embodiment of the present application;

[0060] FIG6C is an example diagram of an unconnected subgraph provided in an embodiment of the present application;

[0061] FIG7 is a schematic diagram of establishing a correspondence between a subgraph mode and resource information according to an embodiment of the present application;

[0062] FIG8 is a schematic diagram of another embodiment of a resource scheduling method provided in an embodiment of the present application;

[0063] FIG9 is another schematic diagram of the structure of the resource scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms "first," "second," and the like in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0066] The present application provides a resource scheduling method for improving resource utilization during AI model training or inference. The present application also provides a corresponding apparatus, a computer-readable storage medium, and a computer program product. These are described in detail below.

[0067] To facilitate understanding, the following briefly introduces the technical terms involved in the embodiments of this application:

[0068] 1. Artificial Intelligence (AI): AI is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theory.

[0069] 2. AI model: The AI ​​model can be a neural network model built based on the AI ​​framework to solve problems in different application scenarios.

[0070] 3. AI Framework: The core function of an AI framework is to provide developers with an interface (mathematical operations) for building neural networks. It automatically performs reverse derivation of neural network training, approximating the optimal value, and obtaining a neural network model (approximation function) that can be used to solve classification, regression, and fitting problems, enabling application scenarios such as object classification and speech recognition. Mainstream AI frameworks in the industry include MindSpore, TensorFlow, and PyTorch. AI frameworks generally use computational graphs to represent network structures. This AI framework can be a deep learning framework or another machine learning (ML) framework.

[0071] 4. Deep learning (DL): A branch of machine learning, DL is an algorithm that uses artificial neural networks as a framework to learn and represent data. Deep learning frameworks include deep neural networks, convolutional neural networks, deep belief networks, and recurrent neural networks.

[0072] 5. Computational Graph: A computational graph is a directed graph with operators as nodes, representing a computational function. In the AI ​​framework, this computational function sequentially calls the operators in the directed graph on the input tensor to produce the final output tensor. An operator is essentially a computational function, such as Add and Sigmoid. A computational graph also includes edges, which connect two operators with a production-consumption relationship (input-output relationship).

[0073] 6. Subgraph: A subgraph is a part of a computational graph. A subgraph can be cut out of a computational graph according to certain constraints.

[0074] The resource scheduling method provided in the embodiments of the present application can be applied to the AI ​​model training stage or the AI ​​model reasoning stage. Whether in the AI ​​model training stage or the AI ​​model reasoning stage, the resource scheduling process can be performed on a cloud system, an independent server, or a terminal device. The following is an example of an AI model executed by a cloud system. Figure 1A shows the process of the cloud system training an AI model, and Figure 1B shows the process of the cloud system executing an AI model reasoning.

[0075] As shown in Figure 1A, the cloud system includes a resource scheduling device and a resource pool. The resource pool includes a computing resource pool, a communication resource pool, and a storage resource pool. The computing resource pool includes computing resources 1, ... computing resources M. These computing resources can be single-card computing resources (such as central processing unit (CPU), graphics processing unit (GPU), neural-network processing unit (NPU)), etc., or cluster computing resources. The communication resource pool includes communication resources 1, ... communication resources N. These communication resources can be video memory bandwidth or other resources that can communicate. The storage resource pool includes storage resources 1, ... storage resources P. These storage resources can be video memory, internal memory, or other types of storage, where M, N, and P are all positive integers.

[0076] During the AI ​​model training process, the cloud system can receive the AI ​​model uploaded by the tenant or user and the sample data used to train the AI ​​model through the resource scheduling device. The resource scheduling device can schedule resources for the AI ​​model, such as: the resources scheduled for training the AI ​​model include computing power resources 1, computing power resources 2, communication resources 2, communication resources 3, and storage resources 1, and then schedule the training tasks and sample data of the AI ​​model to computing power resources 1, computing power resources 2, communication resources 2, communication resources 3, and storage resources 1. Then, computing power resources 1, computing power resources 2, communication resources 2, communication resources 3, and storage resources 1 use the sample data to complete the training of the AI ​​model through multiple rounds of iterations, thereby outputting the trained AI model.

[0077] As shown in Figure 1B, the structure of the cloud system can be understood by referring to the introduction of Figure 1A. During the inference process of the AI ​​model, the cloud system can receive the identifier of the AI ​​model and the input data for inference through the resource scheduling device. The resource scheduling device can determine the AI ​​model to be used based on the identifier of the AI ​​model and schedule resources for the AI ​​model. For example, the resources scheduled for training the AI ​​model include computing power resources 1, computing power resources 2, communication resources 2, communication resources 3, and storage resources 1. The AI ​​model and input data are then scheduled to computing power resources 1, computing power resources 2, communication resources 2, communication resources 3, and storage resources 1. Then, computing power resources 1, computing power resources 2, communication resources 2, communication resources 3, and storage resources 1 run the AI ​​model and input the input data to the AI ​​model, completing the inference process and obtaining the inference result.

[0078] The structure of the cloud system can also be understood by referring to FIG. 2A to FIG. 2C .

[0079] Figure 2A is a schematic diagram of the structure of a cloud system provided by an embodiment of the present application. As shown in Figure 2A, the cloud system includes a resource scheduling device and multiple resource nodes, and the resource scheduling device can communicate with the multiple resource nodes. The resource scheduling device can convert the AI ​​model into a computational graph, and then determine at least one target subgraph from the computational graph, and then schedule resources for the AI ​​model based on the at least one target subgraph. For example, one or more resource nodes among multiple resource nodes are scheduled to execute the AI ​​model.

[0080] The resource scheduling device in FIG2A may be an independent device, or may be integrated into a control platform of a cloud system, or integrated into a certain resource node.

[0081] The functions of the resource scheduling device can be implemented through software or hardware.

[0082] As an example of a software functional unit, a resource scheduling device may include code running on a computing instance. The computing instance may include at least one of a physical host (computer device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the resource scheduling device may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0083] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Inter-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0084] As an example of a hardware functional unit, a resource scheduling device may include at least one computer device, such as a server. Alternatively, the resource scheduling device may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0085] The multiple computer devices included in the resource scheduling apparatus can be distributed in the same zone or in different zones. The multiple computer devices included in the resource scheduling apparatus can be distributed in the same AZ or in different AZs. Similarly, the multiple computer devices included in the resource scheduling apparatus can be distributed in the same VPC or in multiple VPCs. The multiple computer devices can be any combination of computer devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0086] The cloud system provided in the embodiments of the present application may be a cloud service system. In the cloud service system, as shown in FIG2B , the cloud service system includes a cloud platform and basic resources. The cloud platform includes a cloud platform manager, and the resource scheduling device in FIG2A may be the cloud platform manager in FIG2B . The basic resources may include multiple servers, each of which may include multiple resource nodes.

[0087] The resource nodes in Figures 2A and 2B may be computing device cards or virtual machines (VMs), wherein the computing device cards may be at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processing unit (NPU).

[0088] The cloud platform manager can convert the AI ​​model into a computation graph, determine at least one target subgraph from the computation graph, and then schedule resources for the AI ​​model based on the at least one target subgraph. For example, it can schedule one or more resource nodes from multiple resource nodes to execute the AI ​​model.

[0089] The cloud system provided in the embodiments of the present application can be a data center. As shown in FIG2C , the data center includes a data center management platform, an internal data center network, and multiple servers. Each server includes a hardware layer and a software layer. The hardware layer includes memory, a network card, a processor, and a disk, which are connected via a bus. The hardware layer provides the hardware resources required for the virtual machines in the software layer to operate. The software layer includes a host operating system and multiple virtual machines. The host operating system can include a data center management platform client that can interact with the data center management platform.

[0090] Virtualization technology, which mainly consists of computing virtualization and input / output (I / O) virtualization, shares a physical server with multiple tenants at the granularity of virtual machines, allowing tenants to use physical resources conveniently and flexibly under the premise of secure isolation, and can greatly improve the utilization of physical resources.

[0091] Compute virtualization is the process of making a server's computing resources, such as processors and memory, available to virtual instances. For example, virtual machines (VMs) are virtual instances. In other scenarios, these virtual instances are containers or bare metal servers.

[0092] In Figure 2C, each server obtains multiple virtual machines through virtualization technology, and each virtual machine can be understood as a resource node. The resource scheduling device in Figure 2A can be the data center management platform in Figure 2C.

[0093] Among them, virtual machines can also be called cloud servers (Elastic Compute Service, ECS) or elastic instances (different cloud service providers have different names).

[0094] The data center management platform can provide an access interface (such as an interface or application programming interface (API)). Tenants can operate the client remote access access interface to register an account and password on the data center management platform and log in to the data center management platform. After the data center management platform successfully authenticates the account and password, the tenant can send the AI ​​model (such as the code of the AI ​​model) to the data center management platform through the client. The data center management platform can then convert the AI ​​model into a computational graph, and then determine at least one target subgraph from the computational graph, and then schedule resources for the AI ​​model based on the at least one target subgraph. For example: scheduling one or more resource nodes among multiple resource nodes to execute the AI ​​model. The task completion response can then be returned to the tenant.

[0095] Figure 3 is a possible logical structure diagram of the resource scheduling device provided in an embodiment of the present application. As shown in Figure 3, the resource scheduling device 30 provided in an embodiment of the present application includes: a processor 301, a communication interface 302, a memory 303 and a bus 304. The processor 301, the communication interface 302 and the memory 303 are interconnected via the bus 304. In an embodiment of the present application, the processor 301 is used to control and manage the actions of the resource scheduling device 30. For example, the processor 301 is used to execute the process of converting the AI ​​model into a calculation graph, and then determining at least one target subgraph from the calculation graph, and then scheduling resources for the AI ​​model based on the at least one target subgraph. The communication interface 302 is used to support the resource scheduling device 30 to communicate. For example, the communication interface 302 can execute the process of obtaining the AI ​​model. The memory 303 is used to store the program code and data of the resource scheduling device 30.

[0096] The processor 301 may be a central processing unit (CPU), a general-purpose processor (GPOR), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device (PLD), a transistor logic device (TLD), a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. A processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The bus 304 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, for example. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG. 3 shows only one thick line, but this does not imply that there is only one bus or only one type of bus.

[0097] Based on the cloud system described above, the resource scheduling method provided by the embodiment of the present application is described below. The content involved in the resource scheduling device execution in the method can be executed by the resource scheduling device, or by a component of the resource scheduling device (such as a processor, chip, or chip system, etc.).

[0098] As shown in FIG4 , the resource scheduling method provided in the embodiment of the present application includes:

[0099] 401. Determine at least one target subgraph from an artificial intelligence (AI) model; wherein, at least one target subgraph is a subgraph in a computational graph corresponding to the AI ​​model that satisfies typicality, and the operating resources associated with at least one target subgraph match the execution requirements of the AI ​​model, and the typicality includes that the frequency of occurrence of the same subgraph is higher than a first threshold.

[0100] In this application, at least one (or more) target subgraphs are subgraphs in the computation graph of the AI ​​model that meet the typicality requirement. In other words, the target subgraphs are subgraphs whose occurrence frequency in the computation graph is higher than a first threshold. The first threshold can be a preset value based on demand or a value dynamically determined based on the complexity of the AI ​​model.

[0101] In the present application, typicality may include not only the case where the frequency of occurrence of a subgraph is higher than the first threshold, but also other typical cases, such as: the first subgraph includes the second subgraph, and the first subgraph may also be understood as a typical subgraph; or, although the frequency of occurrence of the third subgraph does not reach the first threshold, the input tensor or output tensor of the operator in the third subgraph has a greater impact on the execution of other subgraphs, etc. This application does not limit this, and the content of typicality may be set according to needs.

[0102] 402. Schedule the running resources of the AI ​​model based on at least one target subgraph.

[0103] The resources of this application may include computing resources, storage resources, communication resources, and other resources used to train AI models or use AI models for reasoning.

[0104] The solution provided in the embodiment of the present application, by determining at least one target subgraph from the AI ​​model, can replace a complex AI model with a typical target subgraph, and then schedule resources based on the target subgraph, which can schedule the operating resources of the AI ​​model more accurately and fine-grainedly, thereby improving resource utilization during AI model training or reasoning.

[0105] The above-mentioned step 401 may include: converting the AI ​​model into a computational graph, and determining multiple subgraphs from the computational graph; processing the multiple subgraphs to obtain at least one target subgraph that meets typicality.

[0106] The process may include: identifying the code of the AI ​​model, determining the operators and the association relationships between the operators from the code, and then constructing edges between the operators according to the association relationships between the operators, thereby converting the AI ​​model into a computational graph.

[0107] Because AI models are usually complex and include many operators, for the sake of convenience, we will use a code snippet from the AI ​​model as an example to illustrate the conversion process from code to computational graphs.

[0108] As shown in Figure 5A, which includes 10 lines of code, the first line, Node0, represents an operator identified as 0, which can be described as Operator 0. The second line, "OutEdge: 2," indicates that there is an edge between Node0 and Node2, from Node0 to Node2. This means that there is an input-output relationship between Operator 0 and Operator 2, and the output of Operator 0 can be used as the input of Operator 2.

[0109] Similarly, the third line, Node1, represents an operator identified as 1 and can be described as Operator 1. The fourth line, "OutEdge: 2," indicates that there is an edge between Node1 and Node2, from Node1 to Node2. This means that there is an input-output relationship between Operator 1 and Operator 2, and the output of Operator 1 can be used as the input of Operator 2.

[0110] Lines 5 through 8 in Figure 5A show that Node2's input (InEdge) comes from Node0 and Node1, and its output (OutEdge) points to Node3. In other words, Operator 2 uses the output data of Operator 0 and Operator 1 as input data. After completing the operation, Operator 2 passes the output data of Operator 2 to Operator 3 as input data.

[0111] In FIG5A , lines 9 and 10 indicate that the input (InEdge) of Node2 comes from Node2. In other words, operator 3 uses the output data of operator 2 as input data to perform a logical operation.

[0112] By identifying and converting the code in FIG5A , a computation graph as shown in FIG5B can be obtained.

[0113] Of course, Node3 can also have outputs pointing to other operators. This application will not expand on this. This code snippet is used as an example to illustrate the conversion process from the AI ​​model to the computational graph.

[0114] In this application, after the AI ​​model is converted into a computational graph, the computational graph can be traversed according to the constraints, and multiple subgraphs that meet the constraints can be determined in the computational graph.

[0115] Constraints in this application may include correctness, connectivity, and completeness. Correctness means that each traversed subgraph must conform to the computational logic of the AI ​​model. Connectivity means that any two operators in each subgraph are directly or indirectly connected. Completeness means that no subgraphs in the computational graph that meet correctness and connectivity are omitted.

[0116] As shown in Figure 6A, if a subgraph shown in Figure 6A is traversed from the calculation graph after the AI ​​model conversion, the subgraph includes a pooling operator, two convolution (Conv) operators, two batch normalization (BN) operators, two rectified linear unit (ReLU) operators, and an addition (Add) operator. Among them, the output of the pool operator is used as the input of each Conv operator (connected by edges), each Conv operator is connected to a BN operator, and the output of the corresponding Conv operator is used as the input of the connected BN operator. Each BN operator is connected to a ReLU operator, and both ReLU operators are connected to the Add operator, that is, the output data of the two ReLU operators are added in the Add operator.

[0117] The subgraph shown in Figure 6A satisfies both correctness and connectivity. However, if a subgraph such as that shown in Figure 6B is traversed from the computational graph converted from the AI ​​model, the subgraph shown in Figure 6B does not meet correctness requirements. In Figure 6B, the two Conv operators are connected to the same BN operator, which violates the computational logic and modeling principles of the AI ​​model.

[0118] If a subgraph as shown in Figure 6C is traversed from the computational graph converted from the AI ​​model, the subgraph shown in Figure 6C does not meet connectivity requirements. A ReLU operator in Figure 6C is not connected to the Add operator, which means that the output data of the ReLU operator on the right side of the subgraph shown in Figure 6C cannot reach the Add operator.

[0119] Following the aforementioned requirements for correctness and connectivity, after traversing all subgraphs in the computational graph after the AI ​​model conversion, multiple subgraphs that meet correctness and connectivity can be obtained. Multiple traversals can be performed to minimize the omission of subgraphs that meet correctness and connectivity, ensuring that these subgraphs meet completeness. If each subgraph in these multiple subgraphs meets correctness and connectivity, and if these subgraphs meet completeness, this helps improve the credibility of the target subgraphs subsequently determined.

[0120] After obtaining multiple subgraphs from the calculation graph, the multiple subgraphs can be processed, and a target subgraph that meets the typicality can be determined from the multiple subgraphs. The process of selecting a target subgraph that meets the typicality can check each subgraph in the multiple subgraphs, mark the frequency of occurrence of repeated subgraphs, and if there is a cyclic relationship between the layout operators in the calculation graph, the frequency of occurrence of these subgraphs with a cyclic relationship may be very high. If the frequency of occurrence of the same subgraph is higher than the first threshold, the subgraph can be determined as the target subgraph. Of course, the frequency statistics of different subgraphs can also be used to determine the top Q (Q is a positive integer) subgraphs with higher frequencies as target subgraphs. The value of the first threshold or the top Q with the highest frequency can be set according to demand, or it can be adjusted dynamically, and this application does not limit this.

[0121] The multiple subgraphs may also contain a first subgraph and a second subgraph that have a containment relationship. For example, if the first subgraph contains the second subgraph, the containment relationship can be marked in the first subgraph, or the first subgraph with the containment relationship can be identified as the target subgraph. After determining at least one target subgraph from the AI ​​model as described above, the AI ​​model's operating resources can be scheduled based on the target subgraph.

[0122] In the embodiment of the present application, the above step 402 can be implemented in multiple ways, two of which are introduced below.

[0123] 1. Schedule the running resources of the AI ​​model based on the correspondence between subgraph patterns and resource information.

[0124] In the embodiments of the present application, the correspondence between subgraph patterns and resource information can be a relationship library established by multiple different AI models, which generally includes multiple correspondences. Alternatively, it can be a correspondence established for a single AI model when training or using a single AI model for inference.

[0125] The multiple corresponding relationships included in the relationship library can be recorded in a table format, and can be understood by referring to Table 1.

[0126] Table 1:

[0127] In which, the subgraph pattern can be in the form of an index or an identifier, and the resource information can be used to indicate the proportion or size of different types of resources in the running resources; or, the resource information in the corresponding relationship includes identification information and / or address information of different types of resources in the running resources; wherein, the running resources include computing resources, as well as storage resources and / or communication resources.

[0128] The relationship between the subgraph mode and resource information can be further understood by referring to Table 2.

[0129] Table 2:

[0130] The way of expressing the resource information in Table 2 is only an example. The resource information can also be expressed in other forms, such as in the form of a ratio. For example, the ratio of different types of resources in the resource information corresponding to sub-graph mode 1 is: computing resources: storage resources: communication resources is 2:1:10. This ratio is just a way of writing. The 2 means that 2 computing resources are required, such as 2 CPUs, 2 GPUs, 2 NPUs, or a combination of CPU and GPU, a combination of CPU and NPU, a combination of GPU and NPU, etc. The 1 in the ratio means that the storage resource is 1GB, and the 10 in the ratio means that the communication resource is 10MB of bandwidth. It should be noted that when expressed in the form of a ratio, the position of various types of resources in the ratio and the units of various types of resources can be pre-set.

[0131] If the resource information is represented by identification information and / or address information of different types of resources, the above correspondence can be understood by referring to Table 3.

[0132] Table 3:

[0133] The resource information in Table 3 indicates the identifiers and address information of various types of resources, so that the corresponding resources can be scheduled directly according to the identifiers and address information.

[0134] In the embodiment of the present application, different subgraph modes correspond to different combinations of model structures and operation strategies. The combinations of model structures and operation strategies represented by the subgraph modes can be understood by referring to Table 4.

[0135] Table 4:

[0136] Among them, once the AI ​​model is determined, the model structure of the AI ​​model is determined. The structure of the AI ​​model may include the number of layers of the AI ​​model, the type of each layer, computation-intensive or memory-intensive, etc.

[0137] The operation strategy of an AI model may include the concurrency / parallelism strategy and communication strategy of the operators in the computational graph of the AI ​​model. Among them, parallel strategies may include data parallelism strategy, pipeline parallelism strategy, model parallelism (tensor parallelism) strategy, and expert mixture strategy. Among them, the data parallelism strategy means that different subsets of the same batch of data can be run on different computing device cards (such as GPU, NPU or CPU); the pipeline parallelism strategy means that different layers of the model can be run on different computing device cards; the model parallelism strategy means that a single data operation (such as matrix multiplication) can be split and run on different computing device cards; the expert mixture strategy means that a small part of each layer of the model is used to process data.

[0138] It should be noted that the model structures and operation strategies introduced in Table 4 above are examples. In fact, there may be multiple model structures and multiple operation strategies. The combination of different model structures and operation strategies can correspond to a subgraph pattern. Therefore, after determining the operation strategy based on the type of operator in at least one target subgraph, the target subgraph pattern can be determined in combination with the model structure of the AI ​​model (for example, in combination with Table 4). In combination with any one of the tables in Tables 1 to 3 above, the corresponding target resource information can be determined according to the target subgraph pattern, and then the corresponding resources can be scheduled according to the target resource information.

[0139] In this way, using the target subgraph pattern as an index, the target resource information corresponding to the target subgraph pattern is found from the correspondence between the subgraph pattern and the resource information, and then the running resources for the AI ​​model are scheduled based on the target resource information. This not only accurately schedules the running resources of the AI ​​model, but also speeds up the scheduling of running resources.

[0140] The above describes the process of using the correspondence between the subgraph mode and the resource information. The following describes the process of establishing the correspondence between the subgraph mode and the resource information in conjunction with FIG. 7 .

[0141] As shown in Figure 7, the resource scheduling device in the cloud system may include a computation graph conversion module 701, a subgraph mining module 702, a subgraph processing module 703, an operation strategy translation module 704, a performance estimation module 705 and a resource matching module 706.

[0142] Among them, the computation graph conversion module 701 can convert the code of the AI ​​model into a computation graph. This process can be understood by referring to the example process of Figures 5A to 5B above.

[0143] The subgraph mining module 702 can traverse the computation graph according to constraints such as correctness, connectivity, and completeness, and mine multiple subgraphs that meet the above conditions. The process of obtaining multiple subgraphs from the computation graph can be understood by referring to the introduction of Figures 6A to 6C above.

[0144] The subgraph processing module 703 can frequency-mark repeated subgraphs, determine subgraphs with inclusion relationships, set inclusion marks for larger subgraphs among the subgraphs with inclusion relationships, and thereby determine at least one typical target subgraph among the multiple subgraphs.

[0145] The operation strategy translation module 704 can analyze at least one target subgraph, determine the type of operators in the target subgraph, and determine the operation strategy of the AI ​​model according to the type of operators.

[0146] Combined with the introduction in Table 4 above, the structure of the AI ​​model is known. After the operation strategy translation module 704 determines the operation strategy of the AI ​​model, it can determine the subgraph mode of the AI ​​model, such as subgraph mode W.

[0147] The performance estimation module 705 can estimate the time taken to execute the AI ​​model based on the operation strategy determined by the strategy translation module 704, the model structure of the AI ​​model and the parameters of the resources in the resource pool, that is, the performance estimation result.

[0148] Among them, the estimation process can be estimated using a prediction formula, which can be a prediction formula established based on operator computing volume, communication volume and memory constraints for predicting AI model training or inference time.

[0149] The above-mentioned operation strategy and AI model can be used to estimate the general demand for various types of resources (computing resources, storage resources, and communication resources). Then, the parameters of the resources in the resource pool are combined to determine multiple sets of resource parameters. Using these multiple sets of resource parameters, the prediction formula is substituted to predict the time corresponding to each set of resource parameters. This is shown in Table 5:

[0150] Table 5:

[0151] The performance estimation module 705 may output the execution time of the AI ​​model corresponding to each resource parameter group.

[0152] The resource matching module 706 can determine the shortest time among the AI ​​model execution times corresponding to each resource parameter group. For example, if T2 is the shortest time between T1 and TZ, the resource information of resource parameter group 2 corresponding to T2 can be selected as the resource solution / resource information to be output, such as resource information W. For the identification format of this resource information, please refer to the introduction of Tables 2 and 3 above for understanding.

[0153] Combined with the operation strategy determined at the above-mentioned operation strategy translation module 704 and the model structure of the AI ​​model, combined with the above-mentioned Table 4, the subgraph pattern of the AI ​​model is determined, such as: subgraph pattern W, and then, the corresponding relationship between the subgraph pattern W and the resource information W is established (subgraph pattern W→resource information W), and the corresponding relationship between the subgraph pattern W and the resource information W can be stored in the relationship library.

[0154] Subsequently, when the AI ​​model to be trained or the AI ​​model to perform inference determines that the subgraph pattern is subgraph pattern W through the target subgraph, the corresponding resource information W can be found from the relationship library, and then the AI ​​model can be executed according to the resource information W.

[0155] 2. Schedule the AI ​​model's operating resources based on the target subgraph's operating strategy and the AI ​​model's structure.

[0156] This process of scheduling the running resources of the AI ​​model may not be combined with the correspondence between the above-mentioned subgraph pattern and resource information, but dynamically determine the running strategy of the AI ​​model based on the type of operator in at least one target subgraph; and schedule the running resources of the AI ​​model based on the model structure of the AI ​​model and the running strategy of the AI ​​model.

[0157] The process can be understood by referring to Figure 8. As shown in Figure 8:

[0158] 801. Identify the relationship between operators in the AI ​​model code and convert the AI ​​model code into a computational graph.

[0159] This process can be understood by referring to the example process of FIG. 5A to FIG. 5B .

[0160] 802. Traverse the computation graph and find multiple subgraphs that meet the constraints.

[0161] The process of step 802 can be to traverse the computation graph according to constraints such as correctness, connectivity, and completeness, and mine out multiple subgraphs that meet the above conditions. The process of obtaining multiple subgraphs from the computation graph can be understood by referring to the introduction of Figures 6A to 6C above.

[0162] 803. Process the multiple subgraphs and determine at least one target subgraph from the multiple subgraphs.

[0163] Step 803 may be to frequency-mark repeated subgraphs, determine subgraphs with inclusion relationships, set inclusion marks for larger subgraphs among the subgraphs with inclusion relationships, and thereby determine at least one typical target subgraph among the multiple subgraphs.

[0164] 804. Analyze at least one target subgraph, determine the type of operator in the target subgraph, and determine the operation strategy of the AI ​​model based on the type of operator.

[0165] 805. Estimate the time required to execute the AI ​​model based on the operation strategy, the model structure of the AI ​​model, and the parameters of the resources in the resource pool.

[0166] The estimation process and the determination of a set of resource parameters that takes the shortest time to execute the AI ​​model can be understood by referring to the introduction of the performance estimation module 705 and Table 5 above.

[0167] 806. Determine target resource information based on a set of resource parameters that takes the shortest time.

[0168] In step 806, the set of resource parameters with the shortest time usage can be directly used as the target resource information, or the target resource information can be obtained by obtaining the representation of the resource information in Table 2 or Table 3 based on the set of resource parameters with the shortest time usage.

[0169] 807. Schedule the running resources of the AI ​​model based on the target resource information.

[0170] The solution provided by the embodiments of this application can determine the AI ​​model's execution strategy based on the operator type in at least one target subgraph, and then schedule the AI ​​model's execution resources based on the AI ​​model's model structure. This method of determining the execution strategy based on the operator type of a typical target subgraph can improve the accuracy of the execution strategy, thereby improving the accuracy of execution resource scheduling and increasing resource utilization.

[0171] The resource scheduling method is introduced above. The resource scheduling device provided in the embodiment of the present application is described below with reference to the accompanying drawings.

[0172] As shown in FIG9 , a structure of a resource scheduling device 90 provided in an embodiment of the present application includes:

[0173] A first processing unit 901 is configured to determine at least one target subgraph from an artificial intelligence (AI) model; wherein the at least one target subgraph is a subgraph in a computation graph corresponding to the AI ​​model that satisfies typicality, and running resources associated with the at least one target subgraph match execution requirements of the AI ​​model, where typicality includes an occurrence frequency of the same subgraph exceeding a first threshold.

[0174] The second processing unit 902 is used to schedule the running resources of the AI ​​model based on at least one target subgraph.

[0175] The resource scheduling device provided in the embodiment of the present application can replace a complex AI model with a typical target subgraph by determining at least one target subgraph from the AI ​​model, and then schedule resources based on the target subgraph. It can schedule the operating resources of the AI ​​model more accurately and in a finer granularity, thereby improving resource utilization during AI model training or reasoning.

[0176] Optionally, the second processing unit 902 is specifically used to determine a target subgraph pattern corresponding to at least one target subgraph, the target subgraph pattern is used to indicate the model structure of the AI ​​model represented by at least one target subgraph and the operation strategy of the AI ​​model; according to the correspondence between the subgraph pattern and the resource information, determine the target resource information corresponding to the target subgraph pattern, the target resource information is used to indicate the operation resources of the AI ​​model; and schedule the operation resources indicated by the target resource information for the AI ​​model.

[0177] Optionally, the second processing unit 902 is specifically used to determine the operation strategy of the AI ​​model based on the type of the operator in at least one target subgraph; and schedule the operation resources of the AI ​​model based on the model structure of the AI ​​model and the operation strategy of the AI ​​model.

[0178] Optionally, the resource information in the corresponding relationship is used to indicate the proportion or size of different types of resources in the running resources; or, the resource information in the corresponding relationship includes identification information and / or address information of different types of resources in the running resources; wherein, the running resources include computing resources, as well as storage resources and / or communication resources.

[0179] Optionally, the running resources are virtual resources and / or physical resources.

[0180] Optionally, the running resources indicated by the target resource information are a set of resources that are estimated to take the shortest time to execute the AI ​​model.

[0181] Optionally, the first processing unit 901 is specifically used to convert the AI ​​model into a computational graph, and determine multiple subgraphs from the computational graph; and process the multiple subgraphs to obtain at least one target subgraph that meets typicality.

[0182] Optionally, each subgraph in the multiple subgraphs satisfies correctness and connectivity. Correctness means that each subgraph needs to conform to the computational logic of the AI ​​model, and connectivity means that any two operators in each subgraph are directly or indirectly connected.

[0183] Optionally, multiple subgraphs need to satisfy completeness; wherein, completeness means that subgraphs that satisfy correctness and connectivity in the computation graph are not omitted.

[0184] In another embodiment of the present application, a computer-readable storage medium is further provided, in which computer-executable instructions are stored. When the processor of the resource scheduling device executes the computer-executable instructions, the resource scheduling device executes the steps performed by the resource scheduling device in Figures 4 to 8 above.

[0185] In another embodiment of the present application, a computer program product is provided. The computer program product includes computer program code. When the computer program code is executed on a computer, the computer device executes the steps executed by the resource scheduling apparatus in FIG. 4 to FIG. 8 .

[0186] In another embodiment of the present application, a chip system is also provided, which includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the memory of the resource scheduling device and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the resource scheduling device executes the steps performed by the resource scheduling device in Figures 4 to 8 above. In one possible design, the chip system may also include a memory, which is used to store program instructions and data necessary for the control device. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0188] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0189] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in whole or in part through software, hardware, firmware, or any combination thereof.

[0190] When software is used to implement the integrated unit, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

Claims

1. A resource scheduling method, characterized in that: include: Determining at least one target subgraph from an artificial intelligence (AI) model; wherein the at least one target subgraph is a subgraph in a computation graph corresponding to the AI ​​model that satisfies typicality, and running resources associated with the at least one target subgraph match execution requirements of the AI ​​model, wherein the typicality includes an occurrence frequency of the same subgraph being greater than a first threshold; Based on the at least one target subgraph, scheduling the running resources of the AI ​​model.

2. The method according to claim 1, characterized in that Scheduling the running resources of the AI ​​model based on the at least one target subgraph includes: Determining a target subgraph mode corresponding to the at least one target subgraph, where the target subgraph mode is used to indicate a model structure of the AI ​​model represented by the at least one target subgraph and an operation strategy of the AI ​​model; Determining target resource information corresponding to the target subgraph pattern according to a correspondence between the subgraph pattern and the resource information, where the target resource information is used to indicate the operating resources of the AI ​​model; Scheduling the operating resources indicated by the target resource information for the AI ​​model.

3. The method according to claim 1, characterized in that Scheduling the running resources of the AI ​​model based on the at least one target subgraph includes: Determining an operation strategy of the AI ​​model according to a type of an operator in the at least one target subgraph; Scheduling the operating resources of the AI ​​model according to the model structure of the AI ​​model and the operating strategy of the AI ​​model.

4. The method according to claim 2, characterized in that The resource information in the corresponding relationship is used to indicate the proportion or size of different types of resources in the running resources; or, the resource information in the corresponding relationship includes identification information and / or address information of different types of resources in the running resources; wherein, the running resources include computing resources, storage resources and / or communication resources.

5. The method according to any one of claims 1 to 4, characterized in that The operating resources are virtual resources and / or physical resources.

6. The method according to claim 2, characterized in that The operating resources indicated by the target resource information are a set of resources that are estimated to take the shortest time to execute the AI ​​model.

7. The method according to any one of claims 1 to 6, characterized in that Determining at least one target subgraph from the artificial intelligence (AI) model includes: Converting the AI ​​model into a computation graph, and determining a plurality of subgraphs from the computation graph; The multiple subgraphs are processed to obtain the at least one target subgraph that meets the typicality.

8. The method according to claim 7, characterized in that Each subgraph among the multiple subgraphs satisfies correctness and connectivity. The correctness means that each subgraph needs to conform to the computational logic of the AI ​​model, and the connectivity means that any two operators in each subgraph are directly or indirectly connected.

9. The method according to claim 8, characterized in that The multiple subgraphs need to satisfy completeness; wherein, the completeness means that no subgraphs in the computation graph that satisfy the correctness and connectivity are omitted.

10. A resource scheduling device, characterized in that: include: a first processing unit, configured to determine at least one target subgraph from an artificial intelligence (AI) model; wherein the at least one target subgraph is a subgraph in a computation graph corresponding to the AI ​​model that satisfies typicality, and running resources associated with the at least one target subgraph match execution requirements of the AI ​​model, wherein the typicality includes an occurrence frequency of the same subgraph being greater than a first threshold; The second processing unit is used to schedule the running resources of the AI ​​model based on the at least one target subgraph.

11. The device according to claim 10, characterized in that The second processing unit is specifically configured to: Determining a target subgraph mode corresponding to the at least one target subgraph, where the target subgraph mode is used to indicate a model structure of the AI ​​model represented by the at least one target subgraph and an operation strategy of the AI ​​model; Determining target resource information corresponding to the target subgraph pattern according to a correspondence between the subgraph pattern and the resource information, where the target resource information is used to indicate the operating resources of the AI ​​model; Scheduling the operating resources indicated by the target resource information for the AI ​​model.

12. The device according to claim 10, characterized in that The second processing unit is specifically configured to: Determining an operation strategy of the AI ​​model according to a type of an operator in the at least one target subgraph; Scheduling the operating resources of the AI ​​model according to the model structure of the AI ​​model and the operating strategy of the AI ​​model.

13. The device according to claim 11, characterized in that The resource information in the corresponding relationship is used to indicate the proportion or size of different types of resources in the running resources; or, the resource information in the corresponding relationship includes identification information and / or address information of different types of resources in the running resources; wherein, the running resources include computing resources, storage resources and / or communication resources.

14. The device according to any one of claims 10 to 13, characterized in that The operating resources are virtual resources and / or physical resources.

15. The device according to claim 11, characterized in that The operating resources indicated by the target resource information are a set of resources that are estimated to take the shortest time to execute the AI ​​model.

16. The device according to any one of claims 10 to 15, characterized in that The first processing unit is specifically configured to: Converting the AI ​​model into a computation graph, and determining a plurality of subgraphs from the computation graph; The multiple subgraphs are processed to obtain the at least one target subgraph that meets the typicality.

17. The device according to claim 16, characterized in that Each subgraph among the multiple subgraphs satisfies correctness and connectivity. The correctness means that each subgraph needs to conform to the computational logic of the AI ​​model, and the connectivity means that any two operators in each subgraph are directly or indirectly connected.

18. The device according to claim 17, characterized in that The multiple subgraphs need to satisfy completeness; wherein, the completeness means that no subgraphs in the computation graph that satisfy the correctness and connectivity are omitted.

19. A computer device, characterized in that: comprising a processor and a computer-readable storage medium storing a computer program; The processor is coupled to the computer-readable storage medium, and when the computer program is executed by the processor, the method according to any one of claims 1 to 9 is implemented.

20. A computer program product, characterized in that The computer program product comprises a computer program code, which, when run on a computer device, causes the computer device to perform the method according to any one of claims 1 to 9.

Images