Method and device for determining operation configuration of AI model and related equipment

Abstract

The invention discloses a method and device for determining operation configuration of an AI model and related equipment, and relates to the technical field of artificial intelligence. A search space of each sub-model in the AI model is obtained, the search space comprises multiple operation configurations and operation time consumed when the sub-model is operated based on each operation configuration, and the operation configurations comprise the number of computing resources and a parallel strategy; obtaining the maximum number of computing resources; according to the search space and the maximum number of each sub-model, target operation configurations are selected for each sub-model, the target operation configurations of the multiple sub-models form a target configuration combination, and the total amount of computing resources in the target operation configurations of the multiple sub-models is smaller than or equal to the maximum number; the sum of the operation time consumption corresponding to each target operation configuration is minimum. Thus, under the condition that the computing resources are limited, the number of the computing resources allocated to each sub-model and the adopted parallel strategy are automatically searched, the labor cost can be effectively reduced, and it is guaranteed that the overall training efficiency of the AI model reaches a high level.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and related equipment for determining the operating configuration of an AI model. Background Art

[0002] With the development of artificial intelligence (AI) technology, the scale of parameters in AI models continues to increase. At this time, multiple computing nodes can be used to adopt one or more parallel strategies to perform distributed training on AI models. Among them, the parallel strategy refers to a strategy that decomposes the training tasks of the AI ​​model and distributes them to multiple computing nodes for simultaneous execution to improve the overall training efficiency of the AI ​​model. For example, it can be tensor parallelism (TP), context parallelism (CP), etc. In addition, the AI ​​model will have different running times when running based on different numbers of computing nodes or using different parallel strategies, which will lead to differences in the overall training efficiency of the AI ​​model.

[0003] An AI model can include multiple different sub-models, each of which can be a component of an AI model with a specific function. For example, in a multimodal model, a single modality model is a sub-model. For example, the large language and vision assistant (LLaVA) model can include two sub-models, namely a language sub-model for processing text (such as Vicuna-13B) and a visual sub-model for processing images (such as ViT-L / 14). A sub-model can also be an independently executable part of an AI model, such as a forward computing model and a backward computing model.

[0004] In actual application scenarios, the computing resources used for distributed training of AI models are limited, such as the limited number of computing nodes involved in training the AI ​​model. Therefore, it is usually the technicians who determine the amount of computing resources used for training each sub-model of the AI ​​model and the parallel strategy (which can be one or more parallel strategies) based on expert knowledge and theoretical calculations. However, this method of determining resource allocation and parallel strategies for each sub-model in the AI ​​model not only requires high manpower costs, but also makes it difficult to achieve a high level of overall training efficiency for distributed training of AI models. Summary of the Invention

[0005] This application provides a method for determining the operational configuration of an AI model to reduce the human effort required to determine resource allocation and parallelization strategies for each sub-model within the AI ​​model and improve the overall efficiency of distributed training of the AI ​​model. Furthermore, this application also provides a corresponding apparatus, computing device, computer-readable storage medium, and computer program product for determining the operational configuration of the AI ​​model.

[0006] In a first aspect, the present application provides a method for determining an operating configuration of an AI model, which can be performed by a corresponding determination device for determining an operating configuration of an AI model. Specifically, the determination device obtains a search space for each of the multiple sub-models included in the AI ​​model, wherein the search space of each sub-model includes multiple operating configurations of each sub-model and the operating time when the sub-model is run based on each operating configuration. The operating configuration includes the number of computing resources (such as the number of computing nodes allocated to the sub-model) and a parallel strategy. The number of parallel strategies included in each operating configuration can be one or more. In addition, the determination device also obtains the maximum number of computing resources allocated to the AI ​​model. Thus, the determination device can select a target operating configuration for each sub-model in each search space based on the operating time of each sub-model under each operating configuration in the search space of each sub-model and the maximum number of computing resources. The target operating configurations of the multiple sub-models constitute a target configuration combination, and in the target configuration combination, the total amount of computing resources in the target operating configurations of the multiple sub-models is less than or equal to the maximum number of computing resources, and at the same time, the sum of the operating times corresponding to the target operating configurations in the target configuration combination is the smallest.

[0007] In this way, the determination device can automatically search for the number of computing resources allocated to each sub-model and the parallel strategy adopted when the running time of the AI ​​model is low when computing resources are limited, based on the search space of each sub-model in the AI ​​model and the maximum number of computing resources allocated to the AI ​​model. There is no need to manually configure the number of computing resources and the parallel strategy for each sub-model in the AI ​​model. This can not only effectively reduce the manpower cost required in the AI ​​model training process, but also, the AI ​​model is distributed trained based on the target configuration combination automatically searched out, and the running time of the AI ​​model is low, which can also ensure that the overall training efficiency of the AI ​​model can reach a high level.

[0008] In one possible implementation, when the determination device selects a target running configuration for each sub-model in each search space based on the running time of each sub-model in each running configuration and the maximum number of computing resources in the search space of each sub-model, it can specifically determine the optimal running time based on the running time in the search space of each sub-model, so that in the running configuration of each sub-model that is less than the optimal time, the running configuration with the total number of computing resources less than or equal to the maximum number of computing resources can be selected as the target running configuration for each sub-model. In this way, the determination device automatically determines the number of computing resources allocated to each sub-model and the parallel strategy adopted based on the running time in the search space, under the condition that the running time is as small as possible (that is, the optimal time) and the total amount of allocated computing resources does not exceed the maximum number. This can effectively reduce the manpower cost required in the AI ​​model training process, and ensure that the AI ​​model is trained based on the determined target configuration combination, so that the overall training efficiency of the AI ​​model can reach a high level.

[0009] In a possible embodiment, the determining device determines the optimal time consumption according to the running time in the search space of each sub-model, including: traversing the search space of each sub-model to obtain the maximum running time and the minimum running time among the running times of multiple sub-models; determining the optimal time consumption according to the maximum running time and the minimum running time consumption, and the optimal time consumption is between the maximum running time consumption and the minimum running time consumption, for example, the optimal time consumption can be the middle value between the maximum running time consumption and the minimum running time consumption; then, the determining device selects the running configuration whose total number of computing resources is less than or equal to the maximum number of computing resources for each sub-model in the running configuration less than the optimal time consumption. When determining the target running configuration, specifically, the total number of computing resources allocated to each sub-model can be determined based on the optimal time consumption; when the total number is greater than the maximum number of computing resources, the first part of the space where the running time is less than the optimal time consumption is determined from the search space of each sub-model, and the target running configuration of each sub-model is determined based on the first part of the space of each sub-model and the maximum number of computing resources; when the total number is less than the maximum number of computing resources, the second part of the space where the running time is greater than the optimal time consumption is determined from the search space of each sub-model, and the target running configuration of each sub-model is determined based on the second part of the space of each sub-model and the maximum number of computing resources. In this way, the determination device can automatically search for the running configuration that can minimize the running time for each sub-model in the AI ​​model while ensuring that the total amount of allocated computing resources does not exceed the maximum number, thereby ensuring that the AI ​​model is trained based on the determined target configuration combination, and enabling the overall training efficiency of the AI ​​model to reach a high level.

[0010] In one possible implementation, when obtaining the search space for each of the multiple sub-models included in the AI ​​model, the determination device may specifically obtain multiple run records of the AI ​​model. The multiple run records are used to indicate the operating configuration and running time adopted by each sub-model during the AI ​​model's operation. Thus, the determination device can construct a search space for each of the multiple sub-models based on the multiple run records of the AI ​​model. In this way, the determination device constructs the search space for each sub-model based on the data recorded during the actual operation of the AI ​​model, thereby improving the accuracy of the constructed search space, thereby helping to improve the accuracy of the target operating configuration subsequently determined from the search space.

[0011] In one possible implementation, the multiple sub-models in the AI ​​model include a first sub-model, the search space of the first sub-model includes multiple operating configurations of the first sub-model, different operating configurations of the multiple operating configurations of the first sub-model include different numbers of computing resources, each operating configuration of the first sub-model includes at least one parallel strategy adopted to generate the minimum operating time, the multiple operating configurations include a first operating configuration, and the first operating configuration includes a first number of computing resources; then, when the determination device constructs the search space of each sub-model in the multiple sub-models based on multiple operating records of the AI ​​model, taking the construction of the search space of the first sub-model as an example, it can specifically be to determine the multiple parallel strategy combinations that can be adopted when the first sub-model is running based on the first number of computing resources, and according to the multiple operating records, parallelly calculate the operating time generated when the first sub-model is running based on the first number of computing resources using the multiple parallel strategy combinations; thus, in the parallel computing process, the determination device can sort the multiple parallel strategy combinations adopted by the first sub-model based on the first number of computing resources according to the operating time, obtain the minimum operating time and the first operating configuration, and the parallel strategy combination adopted by the first operating configuration to generate the minimum operating time. In this way, during the parallel computing process, the determination device can perform a sorting process based on the running time, without having to wait for the parallel computing to end and obtain the entire running time before sorting. This can improve the efficiency of the parallel strategy combination used to determine the first number of computing resources for the first sub-model to generate the minimum running time.

[0012] Similarly, for the second sub-model in the AI ​​model, the determination device can also refer to the above-mentioned method of determining the search space of the first sub-model to construct a corresponding search space for the second sub-model, which will not be elaborated.

[0013] In one possible implementation, when the determination device constructs the search space of each of the multiple sub-models based on multiple operation records of the AI ​​model, it can specifically construct the initial search space of each of the multiple sub-models based on the multiple operation records of the AI ​​model; and the determination device can also obtain a pruning strategy for the initial search space of each sub-model, and prune the initial search space of each sub-model according to the pruning strategy to obtain the search space of each sub-model. In this way, by pruning the initial search space of each sub-model, the range of the search space of each sub-model can be effectively reduced, thereby effectively improving the efficiency of subsequently determining the target configuration combination.

[0014] In one possible implementation, when the determination device constructs the search space of each sub-model in the multiple sub-models based on the multiple running records of the AI ​​model, it can specifically construct a proxy model for the multiple sub-models based on the maximum number of computing resources. The proxy model is used to fit the mapping relationship between the number of computing resources allocated to each sub-model and the running time generated when each sub-model is running, when the sum of the number of computing resources allocated to each sub-model is less than or equal to the maximum number of computing resources; then, the determination device can use the multiple running records of the AI ​​model to iteratively train the proxy model and determine at least one parallel strategy that generates the minimum running time when each sub-model is running based on each number of computing resources. In this way, the determination device uses the proxy model as the search space for each sub-model, and there is no need to calculate the running time generated when each sub-model is running based on various possible numbers of computing resources allocated and various possible combinations of parallel strategies, thereby reducing the cost of constructing the search space for each sub-model.

[0015] In one possible embodiment, when the determination device uses multiple operation records of the AI ​​model to iteratively train the proxy model, it can specifically use the proxy model to determine a first resource allocation result and a predicted result of the running time generated when each sub-model runs based on the computing resources allocated based on the first resource allocation result, and the first resource allocation result is used to indicate the number of computing resources allocated to each sub-model; then, the determination device can determine the actual result of the minimum running time generated when each sub-model runs based on the computing resources allocated based on the first resource allocation result based on the multiple operation records of the AI ​​model, and update the parameters in the proxy model based on the predicted result and the actual result; thereby, the determination device can use the updated proxy model to obtain a second resource allocation result, and the second resource allocation result is used to indicate the number of computing resources allocated to each model component; in this way, when it is determined that the proxy model meets the iteration termination condition based on the first resource allocation result and the second resource allocation result, the determination device ends the iterative training of the proxy model. In this way, the determination device iteratively updates the parameters in the proxy model, so that the proxy model can accurately fit the mapping relationship between the amount of computing resources allocated to each sub-model and the running time generated when each sub-model runs, thereby improving the accuracy of the target configuration combination determined by using the proxy model.

[0016] In one possible implementation, when the determination device obtains multiple running records of the AI ​​model, it may specifically obtain configuration parameters, where the configuration parameters include the number of computing resources used by multiple sub-models when running the AI ​​model and the parallel strategy, thereby collecting multiple running records of the AI ​​model in the process of running the AI ​​model based on the configuration parameters.

[0017] In a second aspect, the present application provides a device for determining the running configuration of an AI model, the device comprising: an acquisition module for acquiring the search space of each sub-model of the multiple sub-models included in the AI ​​model, the search space of each sub-model including multiple running configurations of each sub-model and the running time when the sub-model is run based on each running configuration, the running configuration including the number of computing resources and the parallel strategy; obtaining the maximum number of computing resources allocated to the AI ​​model; a selection module for selecting a target running configuration for each sub-model in each search space according to the running time of each sub-model in each running configuration in the search space of each sub-model and the maximum number of computing resources, the target running configurations of multiple sub-models constitute a target configuration combination, in which the total amount of computing resources in the target running configurations of the multiple sub-models is less than the maximum number of computing resources, and the sum of the running times corresponding to the target running configurations in the target configuration combination is the smallest.

[0018] In one possible implementation, when selecting a target running configuration for each submodel in each search space based on the running time of each submodel in each running configuration and the maximum number of computing resources in the search space of each submodel, the selection module is specifically used to: determine the optimal running time based on the running time in the search space of each submodel; and select, among the running configurations of each submodel that are less than the optimal time, the running configuration in which the total number of computing resources is less than or equal to the maximum number of computing resources as the target running configuration for each submodel.

[0019] In a possible implementation, when the selection module determines the optimal running time based on the running time in the search space of each sub-model, it is specifically used to: traverse the search space of each sub-model to obtain the maximum running time and the minimum running time among the running times of multiple sub-models; determine the optimal running time based on the maximum running time and the minimum running time, and the optimal running time is between the maximum running time and the minimum running time; when the selection module selects the running configuration whose total number of computing resources is less than or equal to the maximum number of computing resources in the running configuration of each sub-model that is less than the optimal time as the target running configuration of each sub-model, it is specifically used to: based on According to the optimal time consumption, determine the total number of computing resources allocated to each sub-model; when the total number is greater than the maximum number of computing resources, determine the first part of the space in which the running time is less than the optimal time consumption from the search space of each sub-model, and determine the target running configuration of each sub-model based on the first part of the space of each sub-model and the maximum number of computing resources; when the total number is less than the maximum number of computing resources, determine the second part of the space in which the running time is greater than the optimal time consumption from the search space of each sub-model, and determine the target running configuration of each sub-model based on the second part of the space of each sub-model and the maximum number of computing resources.

[0020] In one possible implementation, when the acquisition module acquires the search space of each sub-model in the multiple sub-models included in the AI ​​model, it is specifically used to: acquire multiple operation records of the AI ​​model, where the multiple operation records are used to indicate the operation configuration and operation time adopted by each sub-model when the AI ​​model is running; and construct the search space of each sub-model in the multiple sub-models based on the multiple operation records of the AI ​​model.

[0021] In one possible embodiment, multiple sub-models include a first sub-model, the search space of the first sub-model includes multiple operating configurations of the first sub-model, different operating configurations of the multiple operating configurations of the first sub-model include different numbers of computing resources, each operating configuration of the first sub-model includes at least one parallel strategy adopted to generate minimum operating time, the multiple operating configurations include a first operating configuration, and the first operating configuration includes a first number of computing resources; when the acquisition module constructs the search space of each sub-model in the multiple sub-models according to multiple operating records of the AI ​​model, it is specifically used to: determine the multiple parallel strategy combinations that can be adopted when the first sub-model runs based on the first number of computing resources; based on multiple operating records, parallelly calculate the operating time generated when the first sub-model runs based on the first number of computing resources using multiple parallel strategy combinations; in the parallel computing process, sort the multiple parallel strategy combinations adopted by the first sub-model based on the first number of computing resources according to the operating time to obtain the minimum operating time and the first operating configuration, and the parallel strategy combination adopted by the first operating configuration to generate the minimum operating time.

[0022] In one possible implementation, when the acquisition module constructs the search space of each sub-model in multiple sub-models based on multiple running records of the AI ​​model, it is specifically used to: construct the initial search space of each sub-model in the multiple sub-models based on multiple running records of the AI ​​model; obtain a pruning strategy for the initial search space of each sub-model; and prune the initial search space of each sub-model according to the pruning strategy to obtain the search space of each sub-model.

[0023] In one possible implementation, when the acquisition module constructs the search space of each sub-model in multiple sub-models based on multiple running records of the AI ​​model, it is specifically used to: construct proxy models for multiple sub-models based on the maximum number of computing resources, the proxy model being used to fit the mapping relationship between the number of computing resources allocated to each sub-model and the running time generated when each sub-model is running, when the total number of computing resources allocated to each sub-model is less than or equal to the maximum number of computing resources; iteratively train the proxy model using multiple running records of the AI ​​model, and determine at least one parallel strategy adopted by each sub-model to generate the minimum running time when the sub-model is running based on each number of computing resources.

[0024] In one possible embodiment, when the acquisition module uses multiple running records of the AI ​​model to iteratively train the proxy model, it is specifically used to: use the proxy model to determine the first resource allocation result and the predicted result of the running time generated when each sub-model runs based on the computing resources allocated based on the first resource allocation result, the first resource allocation result is used to indicate the number of computing resources allocated to each sub-model; based on the multiple running records of the AI ​​model, determine the actual result of the minimum running time generated when each sub-model runs based on the computing resources allocated based on the first resource allocation result; update the parameters in the proxy model based on the predicted result and the actual result; use the updated proxy model to obtain the second resource allocation result, the second resource allocation result is used to indicate the number of computing resources allocated to each model component; when it is determined that the proxy model meets the iteration termination condition based on the first resource allocation result and the second resource allocation result, end the iterative training of the proxy model.

[0025] In one possible implementation, when acquiring multiple running records of an AI model, the acquisition module is specifically used to: acquire configuration parameters, where the configuration parameters include the number of computing resources used by multiple sub-models and parallel strategies when running the AI ​​model; and collect multiple running records of the AI ​​model in the process of running the AI ​​model based on the configuration parameters.

[0026] The device for determining the operating configuration of the AI ​​model provided in the second aspect corresponds to the method for determining the operating configuration of the AI ​​model provided in the first aspect. Therefore, the technical effects of any implementation of the device for determining the operating configuration of the AI ​​model provided in the second aspect can be found in the description of the technical effects of the corresponding implementation in the method for determining the operating configuration of the AI ​​model provided in the first aspect, and will not be repeated here.

[0027] In a third aspect, the present application provides a computing device comprising a processor and a memory. The processor and the memory communicate with each other. The processor is used to execute instructions stored in the memory so that the computing device executes the method for determining the operating configuration of the AI ​​model as in the first aspect or any one of the implementations of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The computing device may further include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computing device, the computing device executes the operating steps of the method for determining the operating configuration of the AI ​​model described in the first aspect or any implementation of the first aspect.

[0029] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device, enables the computing device to execute the operational steps of the method for determining the operating configuration of the AI ​​model as described in the first aspect or any one of the implementations of the first aspect.

[0030] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of the structure of an exemplary cluster provided in this application;

[0032] Figure 2 A flowchart of a method for determining the operating configuration of an AI model provided in this application;

[0033] Figure 3 A schematic diagram of the implementation process of determining the target configuration combination using a binary search method provided by this application;

[0034] Figure 4 A schematic diagram of the implementation process of using a proxy model to realize the search space and determine the target configuration combination provided by this application;

[0035] Figure 5 A schematic diagram of another implementation process of using a proxy model to realize the search space and determine the target configuration combination provided by this application;

[0036] Figure 6 A schematic diagram of the structure of a device for determining the operating configuration of an AI model provided in this application;

[0037] Figure 7 A schematic diagram of the hardware structure of a computing device provided in this application. DETAILED DESCRIPTION

[0038] The terms "first," "second," and so on, in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate and are merely used to describe the manner in which objects with the same attributes are distinguished in the embodiments of this application.

[0039] The technical solution in this application will be described below in conjunction with the drawings provided in this application.

[0040] See also Figure 1 , is a schematic diagram of the structure of an exemplary cluster 10. Figure 1As shown, the cluster 10 includes a scheduler 100 and multiple computing nodes. Figure 1 In the example, cluster 10 including computing nodes 201 to 204 is used for illustration. Different computing nodes may be connected via a bus, such as a peripheral component interconnect express (PCIe) bus, a compute Express link (CXL) bus, a unified bus (UB or Ubus), or other types of buses. Alternatively, different computing nodes may be connected via a switch node, such as Figure 1 As shown, computing nodes 201 to 204 may be connected via switching node 210 and switching node 211 .

[0041] Scheduler 100 is used for task scheduling, such as scheduling AI models or other AI tasks to one or more computing nodes in cluster 10 for execution. Scheduler 100 can be implemented in software or hardware. When implemented in software, scheduler 100 can be program code running on hardware, such as a process running on a processor or controller. When implemented in hardware, scheduler 100 can be implemented by a processor or a computing device including a processor.

[0042] Computing node 201 (other computing nodes are similar) refers to a node with data computing capability, which can be implemented by a processor, or can be implemented by a computing device including a processor, such as a server. Exemplarily, a processor can be, for example, a central processing unit (CPU), an accelerator, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system on chip (SoC), a software-defined infrastructure (SDI) chip, an AI processing chip, a data processing unit (DPU), or any other processor or any combination thereof. Among them, an accelerator can be a neural-network processing unit (NPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a deep-learning processing unit (DPU), etc.

[0043] Switching node 210 (similar to switching node 211) refers to a node with data exchange capabilities, such as an electrical switching node or an optical switching node. An electrical switching node refers to a switching node that uses electrical signals for data exchange, and these electrical signals can be transmitted via twisted pair or coaxial cables, such as an electrical switch, a network card, or a router. An optical switching node refers to a switching node that uses optical signals for data exchange, such as an optical cross-connect (OXC) device, and these optical signals can be transmitted via optical fiber. In actual applications, a switching node may also be other types of devices, such as a network card or a high-performance network card, and this is not limited to this.

[0044] In actual application, such as Figure 1As shown, cluster 10 can also provide a client 101 to the outside world, and client 101 can interact with the user, such as receiving an AI model provided by the user, or receiving a configuration file for the provided AI model. For example, client 101 can be an application running on the user's terminal device, or a web browser provided by cluster 10, etc., but this is not limited to this.

[0045] In actual application scenarios, the scheduler 100 can schedule multiple computing nodes to perform distributed training on the AI ​​model, which may include multiple sub-models. For example, the AI ​​model may be a multimodal model (i.e., the input data of the AI ​​model may be data in multiple modalities such as text, image, voice, video, etc.), so that the AI ​​model may include multiple single-modal sub-models (each sub-model may be responsible for processing the input data of one modality). In actual application, each sub-model of the AI ​​model may also be referred to as a model component in the AI ​​model. Generally, the computing resources in the cluster 10 are limited. In the process of distributed training of the AI ​​model, the computing resources allocated to each sub-model in the AI ​​model are limited. At this time, if the user (such as a technician) configures the amount of computing resources (for training the sub-model) that can be allocated to each sub-model in the AI ​​model and the combination of parallel strategies used when training the sub-model, it will not only incur high labor costs, but also the user's expert experience and knowledge may have blind spots, which will make it difficult to achieve a high level of overall training efficiency for distributed training of the AI ​​model. For example, in the process of training an AI model, data interaction may be required between the various sub-models. At this time, if sub-model A has higher computing efficiency based on the allocated computing resources and the adopted parallel strategy combination, and sub-model B has lower computing efficiency based on the allocated computing resources and the adopted parallel strategy combination, then the training efficiency of sub-model A will be restricted by the training efficiency of sub-model B, thereby lowering the overall training efficiency of the AI ​​model.

[0046] Based on this, Figure 1 The cluster 10 shown may also include a determination device 300, which can be used to determine the operating configuration of the AI ​​model. Specifically, it can be used to automatically search for a better allocation result of the number of computing resources and at least one parallel strategy adopted for each sub-model in the AI ​​model, so as to improve the overall training efficiency of the AI ​​model. The determination device 300 can be deployed on the same physical node device as the scheduler 100, or it can be deployed independently.

[0047] In a specific implementation, the determination device 300 obtains the search space of each of the multiple sub-models included in the AI ​​model. The search space of each sub-model includes multiple operating configurations of each sub-model and the operating time when the sub-model is run based on each operating configuration. The operating configuration includes the number of computing resources and a parallel strategy (which can be one or more parallel strategies). In addition, the determination device 300 also obtains the maximum number of computing resources allocated to the AI ​​model, such as the number of computing nodes in the cluster 10 that participate in the distributed training of the AI ​​model. Thus, the determination device 300 selects a target operating configuration for each sub-model in the search space of each sub-model based on the operating time of each sub-model under each operating configuration and the maximum number of computing resources allocated to the AI ​​model. The target operating configurations of the multiple sub-models constitute a target configuration combination, and in the target configuration combination, the total amount of computing resources in the target operating configurations of the multiple sub-models is less than or equal to the maximum number of computing resources allocated to the AI ​​model. At the same time, the sum of the operating times corresponding to the various target operating configurations in the target configuration combination is the smallest.

[0048] In this way, the determination device 300 can automatically search for the number of computing resources allocated to each sub-model and one or more parallel strategies adopted when the running time of the AI ​​model is low when computing resources are limited, based on the search space of each sub-model in the AI ​​model and the maximum number of computing resources allocated to the AI ​​model. There is no need to manually configure the number of computing resources and parallel strategies for each sub-model in the AI ​​model. This can not only effectively reduce the manpower cost required in the AI ​​model training process, but also, by performing distributed training on the AI ​​model based on the target configuration combination automatically searched, the running time of the AI ​​model is low, which can also ensure that the overall training efficiency of the AI ​​model can reach a high level.

[0049] Exemplarily, the determining device 300 may be implemented by software or hardware.

[0050] When the determining device 300 is implemented via software, the determining device 300 may be an application program running on hardware, such as a process or software running on a processor or other hardware. When implemented via hardware, the determining device 300 may be a processor or a computing device including a processor, such as a server; or the determining device 300 may be a device implemented using an ASIC or a PLD.

[0051] It is worth noting that the above Figure 1The cluster 10 shown is only used as an exemplary illustration and is not intended to be limiting. For example, in other possible implementations, the cluster 10 may include a larger number of computing nodes or switching nodes, and the multiple switching nodes in the cluster 10 may be deployed into multiple levels. Taking deployment into two levels as an example, some switching nodes may be deployed as leaf nodes in the first level to connect multiple switching nodes, and another part of the switching nodes may be deployed as spine nodes in the second level to forward communication data between different leaf nodes. For another example, in other possible implementations, the cluster 10 may also include other types of nodes, such as storage nodes for storing data generated by the training AI model during the training process, or management nodes for managing the various nodes in the cluster 10.

[0052] in addition, Figure 1 The example in this article uses multiple computing nodes and switching nodes to form a cluster. In other implementations, Figure 1 Multiple computing nodes and switching nodes in the cluster can also constitute a super pod in the cluster. In this case, the cluster 10 can include multiple super nodes, and different super nodes can communicate with each other through a bus or a network.

[0053] For ease of understanding, the following describes an embodiment of the method for determining the operating configuration of an AI model provided in this application in conjunction with the accompanying drawings.

[0054] See also Figure 2 , Figure 2 A flow chart of a method for determining the operating configuration of an AI model provided in an embodiment of the present application, which can be applied to Figure 1 The cluster 10 described above may be applied to other applicable clusters. Figure 1 The cluster 10 shown is taken as an example for illustrative description.

[0055] in, Figure 2 The method for determining the running configuration of the AI ​​model shown may specifically include the following steps.

[0056] S201: The determination device 300 obtains the search space of each sub-model among the multiple sub-models included in the AI ​​model, wherein the search space of each sub-model includes multiple operating configurations of each sub-model and the operating time when the sub-model is run based on each operating configuration, and the operating configuration includes the number of computing resources and the parallel strategy.

[0057] As some examples, the AI ​​model may be a multimodal model that can support reasoning on input data of any multiple modalities such as text, images, audio, and video. In this case, the AI ​​model may include multiple single-modal sub-models, and each single-modal sub-model can be used to calculate input data of one modality. Alternatively, in a knowledge distillation scenario, the AI ​​model may include a teacher model and a student model. In this case, the teacher model and the student model are two sub-models in the AI ​​model, respectively. Alternatively, the AI ​​model may include a policy network and a value network, wherein the policy network is used to output the next decision-making strategy based on the current state data, and the value network is used to evaluate the value of the current state data. In this case, the policy network and the value network may be different sub-models in the AI ​​model, respectively.

[0058] In this embodiment, the determination device 300 can obtain the search space of each sub-model among the multiple sub-models included in the AI ​​model, so as to subsequently automatically search from the search space of each sub-model for the target operating configuration of each sub-model that can enable the overall training efficiency of the AI ​​model to reach a higher level.

[0059] In a possible implementation, the determining device 300 may construct a corresponding search space for each sub-model in the AI ​​model.

[0060] In a specific implementation, the determination device 300 can obtain multiple operation records of the AI ​​model. The multiple operation records can be used to indicate the resource usage and operation time of the AI ​​model during operation. Specifically, they can be used to indicate the operation configuration and operation time adopted by each sub-model during the operation of the AI ​​model. The operation configuration adopted by each sub-model includes the number of computing resources allocated to the sub-model and the parallel strategy adopted when running the sub-model. Among them, multiple operation records refer to operation records collected by running the AI ​​model multiple times. The number of computing resources used and the parallel strategy adopted when running the AI ​​model for different times may be different.

[0061] Exemplarily, the multiple running records of the AI ​​model obtained may be data collected when the AI ​​model runs in the cluster 10 and can be used for performance analysis, such as profiling data.

[0062] As an implementation example, Figure 2 As shown, the scheduler 100 can schedule the AI ​​model to multiple computing nodes in the cluster 10 for execution, and during the execution of the AI ​​model, the determination device 300 can collect multiple operation records of the AI ​​model.

[0063] Specifically, the client 101 can present an interactive interface to the user, and the user can configure the number of computing resources allocated to each sub-model when the AI ​​model is running and one or more parallel strategies adopted in the interactive interface. Accordingly, the client 101 can generate configuration parameters based on the configuration operation performed by the user, and send the configuration parameters to the determination device 300. Among them, the configuration parameters generated by the client 101 include the number of computing resources allocated to each sub-model when the AI ​​model is running and the combination of parallel strategies adopted, which are specified by the user. The combination of parallel strategies includes one or more parallel strategies. Exemplarily, parallel strategies can be, for example, tensor parallelism (TP), context parallelism (CP), data parallelism (DP), pipeline parallelism (PP), expert parallelism (EP), etc., which are not limited to this. The number of computing resources specified by the user can be, for example, the number of computing nodes, etc., or can be achieved by other means that can directly or indirectly characterize the number of computing resources.

[0064] Then, the determination device 300 can instruct the scheduler 100 to schedule the AI ​​model to the corresponding computing node for execution according to the configuration parameters, and use the scheduled computing node to run the AI ​​model. During the operation of the AI ​​model, the determination device 300 can collect the number of computing resources actually used during the operation of the AI ​​model. In addition, the determination device 300 can also collect the running time generated when the AI ​​model is running. The running time can specifically include the time consumed by each sub-model in the AI ​​model to perform forward calculations, and can further include the time consumed by each sub-model to complete reverse calculations and parameter updates. In this way, the determination device 300 can collect multiple operation records of the AI ​​model.

[0065] In this embodiment, the actual operation of the AI ​​model in the cluster 10 to obtain multiple operation records is used as an example for explanation. In other embodiments, the determination device 300 can also obtain the multiple operation records in other ways, such as by the user sending them to the determination device 300 through the client 101, etc., and this is not limited to this.

[0066] Then, the determination device 300 can construct a search space for each sub-model in the AI ​​model according to the multiple operation records of the obtained AI model.

[0067] In one possible implementation, the determination device 300 constructs a search space for each sub-model based on multiple operation records of the AI ​​model, and may record multiple operation configurations of each sub-model and the operation time consumed when running the sub-model based on each operation configuration. Each operation configuration includes the number of computing resources and one or more parallel strategies. In actual application scenarios, it can also be said that the operation configuration includes the number of computing resources and a parallel strategy combination, and the parallel strategy combination includes one parallel strategy or multiple different parallel strategies.

[0068] For example, the determination device 300 can determine the runtime of each sub-model when running based on a user-specified number of computing resources and a user-specified combination of parallel strategies based on multiple run records of the AI ​​model, and use this to infer the runtime of each sub-model when running based on the user-specified number of computing resources using other combinations of parallel strategies, and infer the runtime of each sub-model when running based on other numbers of computing resources using multiple combinations of parallel strategies. For example, for sub-model 1 in the AI ​​model, assume that the user specifies that 2 computing nodes be used to train sub-model 1, the parallel strategy used is data parallelism, and the actual runtime is X minutes. Then, when 4 computing nodes are used to train sub-model 1, and data parallelism is still used, the determination device 300 can infer that the runtime of sub-model 1 is Y minutes (Y is less than X). When 4 computing nodes are used to train sub-model 1, and the parallel strategy combination used includes data parallelism and tensor parallelism, the determination device 300 can infer that the runtime of sub-model 1 is approximately Z minutes (Z is less than Y).

[0069] Exemplarily, the determination device 300 can perform performance modeling for each sub-model based on multiple operation records of the AI ​​model, and use the mathematical model obtained by the modeling to infer the operation time generated by the operation of each sub-model based on any number of computing resources and any combination of parallel strategies. In this way, for each sub-model, the determination device 300 can use a value of the number of computing resources allocated to the sub-model and a combination of parallel strategies that can be adopted as an operation configuration of the sub-model, so that multiple operation configurations of the sub-model can be obtained based on multiple values ​​of the number of computing resources and multiple parallel strategy combinations, and based on the multiple operation configurations and the operation time generated by running the sub-model with each operation configuration, the search space of the sub-model is constructed. In this way, the determination device 300 can realize the construction of the search space of the sub-model for each sub-model.

[0070] Among them, when performing performance modeling for each sub-model separately, the determination device 300 can determine the shapes of each operator in the operator sequence executed by the computing node when the sub-model is running based on the assigned computing node (i.e., computing resources) using different parallel strategies based on multiple running records of the AI ​​model. For example, the operator shape can be determined by the control variable method, etc., so that the functional relationship between the number of computing resources, parallel strategy, operator shape and the actual running time of the sub-model can be modeled. In this way, for other numbers of computing nodes assigned to the sub-model and other parallel strategies adopted, the determination device 300 can, based on the functional relationship obtained by modeling, infer the shapes of each operator in the operator sequence executed by each computing node that executes the sub-model, and further calculate the overall computational time of the computing node executing the operator sequence (such as the overall computational time of the operator sequence executed in the forward calculation phase) based on the shape of each operator, thereby determining the running time of the sub-model based on the number of computing nodes assigned and the parallel strategy adopted. Of course, the determination device 300 can also use other methods to perform performance modeling, which is not limited to this.

[0071] In practical applications, for each sub-model, since the running time generated when the sub-model is run based on a certain number of computing resources and different parallel strategy combinations is usually different, the parallel strategy combination used by the sub-model to run based on this number of computing resources and produce the minimum running time is the optimal parallel strategy combination (which may be a single parallel strategy combination or a combination of multiple parallel strategies). In practical application scenarios, when there are restrictions on the search space, such as when multiple sub-models all use data parallelism and the same number of parallel operations, the optimal parallel strategy combination may be the parallel strategy combination that produces the minimum running time among the multiple parallel strategy combinations that meet the restrictions. Therefore, for each sub-model, the determination device 300 can separately determine the optimal parallel strategy combination used by the sub-model to run based on each number of computing resources and produce the minimum running time, and thereby construct the search space for the sub-model. That is, the search space includes multiple running configurations, different of which include different numbers of computing resources, and each running configuration includes the number of computing resources and the optimal parallel strategy combination used when the sub-model runs based on this number of computing resources. In this way, the scope of the search space can be effectively reduced, and the subsequent search efficiency can be improved. For example, for sub-model 1 in the AI ​​model, based on the four allocated computing nodes, it can be run using parallel strategy combination A, parallel strategy combination B, and parallel strategy combination C, respectively. The determining device 300 can then determine the runtime a generated when sub-model 1 is run using the four computing nodes and parallel strategy combination A, the runtime b generated when running using the four computing nodes and parallel strategy combination B, and the runtime c generated when running using the four computing nodes and parallel strategy combination C. The determining device 300 can then compare the runtimes a, b, and c and determine that runtime a is the smallest. In this case, the determining device 300 can select parallel strategy combination A as the optimal parallel strategy combination and, based on the four allocated computing nodes, the adopted parallel strategy combination A, and the resulting runtime a, include it in constructing the search space for sub-model 1. Parallel strategy combinations B and C are not involved in constructing the search space. That is, only parallel strategy combination A exists in the running configuration including four computing nodes, thereby reducing the size of the search space.

[0072] Among them, for sub-model 1 in the AI ​​model, when determining the optimal parallel strategy combination adopted for each number of computing resources for sub-model 1, taking the determination of the optimal parallel strategy combination adopted for the first number of computing resources as an example, the determination device 300 can first determine the multiple parallel strategy combinations that can be adopted by the sub-model 1 based on the first number of computing resources, and then, based on multiple running records, parallelly calculate the running time generated when the sub-model 1 adopts multiple parallel strategy combinations based on the first number of computing resources. For example, multiple threads can be used to parallelly calculate the running time corresponding to each parallel strategy combination under the condition of the same number of computing resources based on the mathematical model obtained by modeling. Since the time required for different threads to calculate the running time in actual application scenarios usually varies, the determination device 300 can sort the various parallel strategy combinations adopted by the sub-model 1 based on the first number of computing resources according to the calculated running time during the parallel calculation process, such as sorting based on an asynchronous queue algorithm, so as to obtain the shortest running time and the parallel strategy combination adopted to generate the shortest running time. In this way, during the parallel computing process, the determination device 300 can perform a sorting process based on the running time, without having to wait for the parallel computing to end and obtain the total running time before sorting, thereby improving the efficiency of determining the optimal parallel strategy combination for the first number of computing resources for sub-model 1. Similarly, for other numbers of computing resources, the determination device 300 can refer to the above-mentioned similar method to determine the corresponding optimal parallel strategy combination. In this way, the overall efficiency of constructing the search space can be improved. Similarly, referring to the above-mentioned method, the determination device 300 can construct corresponding search spaces for sub-model 2 and the remaining sub-models in the AI ​​model, which will not be elaborated on.

[0073] Moreover, in the process of constructing the search space of each sub-model, the determination device 300 can also perform a pruning operation to construct a search space of an appropriate range. In specific implementation, the determination device 300 can first construct an initial search space for each sub-model based on multiple running records of the AI ​​model. The initial search space includes the running time generated when the sub-model runs based on different numbers of computing resources and the parallel strategy combination adopted. In addition, the determination device 300 will also obtain a pruning strategy for the initial search space of each sub-model, and prune the initial search space of each sub-model according to the pruning strategy to obtain the search space of each sub-model after pruning.

[0074] The pruning strategy can be any strategy that restricts the initial search space. For example, the pruning strategy can be a computing node divisibility strategy corresponding to TP (Tensor Parallelism) or EP (Expert Parallelism). That is, when the parallel strategy combination includes TP or EP, the parallelism parameter in TP or EP must be divisible by the number of allocated computing nodes. If it is not divisible, pruning is performed. For example, for submodel 1, assuming that when submodel 1 runs on three computing nodes, the available parallel strategy combinations include TP. Then, when the dimension of the input data tensor (i.e., the aforementioned parallel parameter) is not divisible by three, such as when the dimension of the tensor is 256×256, the determining device 300 may determine to prune the parallel strategy combination used when the number of allocated computing nodes is three. Thus, the search space of submodel 1 may not include the runtime consumed by submodel 1 when running on three computing nodes using the parallel strategy combination and the parallel strategy combination used. Alternatively, the search space of submodel 1 may include the runtime consumed by submodel 1 when running on three computing nodes using the parallel strategy combination and the parallel strategy combination used, both of which are "None." For another example, the pruning strategy may be a memory overflow prevention strategy, i.e., when the memory resource demand of submodel 1 when running on the allocated computing resources and using the parallel strategy combination exceeds the memory limit supported by the allocated computing resources, the parallel strategy combination is pruned. When all parallel strategies that can be adopted when the sub-model 1 is run based on three computing nodes are pruned, the determining device 300 can set the number of computing nodes to "None" based on the number of computing nodes.

[0075] Alternatively, after pruning the initial search space of each sub-model, the determination device 300 can determine, for each sub-model, from the pruned initial search space, the optimal parallel strategy combination adopted when the sub-model runs based on various quantities of computing resources and generates the minimum running time, and thereby construct a search space for the sub-model, which includes the running time generated when the sub-model runs based on each quantity of computing resources and the optimal parallel strategy combination adopted.

[0076] It should be noted that the above-mentioned implementation method of constructing the search space is for illustrative purposes only and is not intended to be limiting. For example, in other possible implementations, a technician may also provide the determination device 300 with the search space of each sub-model. S202: The determination device 300 obtains the maximum number of computing resources allocated to the AI ​​model.

[0077] In actual application scenarios, the number of computing resources used for distributed training of AI models is limited. This makes it difficult for some sub-models in the multiple sub-models included in the AI ​​model to be allocated sufficient computing resources for training, thereby reducing the training efficiency of the entire AI model. In this embodiment, the determination device 300 can obtain the maximum number of computing resources allocated to the AI ​​model for training the AI ​​model, so as to automatically search for a configuration result that can maximize the overall training efficiency of the AI ​​model within the maximum number of computing resources allocated to the AI ​​model.

[0078] As an implementation example, the client 101 may present an interactive interface to the user, and the user may configure the maximum number of computing resources allocated to the AI ​​model that can be used for AI model training on the interactive interface. The client 101 may then send the user-specified maximum number of computing resources allocated to the AI ​​model to the determination device 300 based on the configuration parameters executed by the user.

[0079] S203: The determination device 300 selects a target operating configuration for each sub-model in each search space based on the operating time of each sub-model in each operating configuration in the search space of each sub-model and the maximum number of computing resources allocated to the AI ​​model, wherein the target operating configurations of multiple sub-models constitute a target configuration combination, and, in the target configuration combination, the total amount of computing resources in the target operating configurations of the multiple sub-models is less than or equal to the maximum number of computing resources allocated to the AI ​​model, and the sum of the operating times corresponding to the target operating configurations in the target configuration combination is the smallest.

[0080] After pruning the initial search spaces of each sub-model and obtaining the search space of each sub-model, the determination device 300 can determine a target configuration combination for the AI ​​model based on the search space of each sub-model (such as the pruned search space) and the maximum number of computing resources allocated to the AI ​​model.

[0081] In one possible implementation, the determination device 300 may use a binary search method to determine the optimal resource allocation method (used to indicate the number of computing resources allocated to each sub-model) and the adopted parallel strategy combination from the search space of each sub-model, that is, to determine the target configuration combination.

[0082] In specific implementation, the determination device 300 determines the optimal running time based on the running time in the search space of each sub-model, and selects the running configuration whose total number of computing resources is less than or equal to the maximum number of computing resources in the running configuration of each sub-model that is less than the optimal time as the target running configuration of each sub-model, thereby obtaining the target configuration combination corresponding to the AI ​​model.

[0083] For ease of understanding, the following Figure 3 , introduces the implementation process of the determination device 300 using a binary search method to determine the target configuration combination.

[0084] Step 1-1: If Figure 3 As shown, the determination device 300 can traverse the search space of each sub-model to obtain the maximum running time High and the minimum running time Low among the running times generated by the multiple sub-models. The maximum running time High generated by the multiple sub-models is the maximum running time in the search space of all sub-models; the minimum running time Low is the minimum running time in the search space of all sub-models.

[0085] Step 1-2: The determining device 300 determines the optimal time according to the maximum operation time High and the minimum operation time Low, and the optimal time is between the maximum operation time and the minimum operation time. Figure 3 In the illustrated embodiment, the determining device 300 may calculate an intermediate value Mid between the maximum running time High and the minimum running time Low, such as the intermediate value Mid=(High+Low) / 2, etc. The intermediate value Mid may be the optimal time.

[0086] Step 1-3: For each sub-model, the determination device 300 traverses the search space of the sub-model to determine the minimum number of resources Num that can be allocated to the sub-model when the running time is less than the middle value Mid (that is, the minimum number of computing resources that can be allocated to the sub-model), such as Figure 3 In this way, the determining device 300 can calculate the minimum resource quantity Num corresponding to each sub-model.

[0087] Step 1-4: The determining device 300 may calculate the sum Sum of the minimum resource quantities Num corresponding to each sub-model.

[0088] Step 1-5: The determination device 300 determines whether the sum Sum is the same as the maximum number of computing resources allocated to the AI ​​model. If yes, then execute step 1-6; if not, then execute step 1-7

[0089] Step 1-6: When the sum Sum is the same as the maximum number of computing resources allocated to the AI ​​model, the determination device 300 can determine the minimum number of resources currently allocated to each sub-model and the parallel strategy combination adopted (which can be the above-mentioned optimal parallel strategy combination) as the target operation configuration of the sub-model, thereby obtaining the target configuration combination corresponding to the AI ​​model based on the target operation configurations of multiple sub-models, such as Figure 3At this point, based on the computing resources allocated to each sub-model, the sum of the running time corresponding to each target running configuration in the target configuration combination is the smallest among all possible configuration combinations.

[0090] Step 1-7: When the sum Sum is not the same as the maximum number of computing resources allocated to the AI ​​model, the determining device 300 may calculate the difference Dif between the current maximum running time High and the minimum running time Low. Then, the determining device 300 compares the difference Dif with the threshold, and when the difference Dif is greater than the threshold, continues to perform steps 1-8 or 1-9, such as Figure 3 When the difference Dif is less than or equal to the threshold, continue to perform steps 1-10, as shown in Figure 3 As shown. Exemplarily, the threshold value may be, for example, the minimum value among the differences between different running times in the search space of all sub-models.

[0091] Step 1-8: When the sum Sum is less than the maximum number of computing resources allocated to the AI ​​model and the difference Dif is greater than the threshold, it indicates that some or all sub-models are allocated to more computing resources to reduce the running time generated when the sub-models are running. At this time, the determination device 300 can determine the first part of the space where the running time is less than the optimal time from the search space of each sub-model, so as to subsequently determine the final target running configuration of each sub-model based on the first part of the space of each sub-model and the maximum number of computing resources allocated to the AI ​​model. For example, Figure 3 As shown, the determining device 300 can use the existing intermediate value Mid as the new minimum running time Low, and calculate the new intermediate value Mid according to the maximum running time High of the running time generated by the sub-model and the new minimum running time Low, that is, the new intermediate value Mid = (High + new minimum running time Low) / 2, as shown in FIG. Figure 3 As shown, the new intermediate value Mid can be used as the new optimal time consumption, thereby increasing the amount of computing resources allocated to the sub-model by increasing the requirement for the running time consumption. Then, the determination device 300 returns to execute steps 1-3 and subsequent steps.

[0092] Step 1-9: When the sum Sum is greater than the maximum number of computing resources allocated to the AI ​​model and the difference Dif is greater than the threshold, it indicates that the number of computing resources allocated to some or all sub-models is too much, and the number of computing resources allocated to some or all sub-models needs to be reduced. At this time, the determination device 300 can determine the second part of the search space of each sub-model whose running time is greater than the optimal time, so as to subsequently determine the target running configuration of each sub-model based on the second part of the space of each sub-model and the maximum number of computing resources allocated to the AI ​​model. For example, Figure 3 As shown, the determining device 300 can use the existing intermediate value Mid as the new maximum running time High, and calculate the new intermediate value Mid according to the minimum running time Low of the running time generated by the sub-model and the new maximum running time High, that is, the new intermediate value Mid = (Low + new maximum running time High) / 2, as shown in FIG. Figure 3 As shown, the new intermediate value Mid can be used as the new optimal time consumption, thereby reducing the amount of computing resources allocated to the sub-model by relaxing the requirement for the running time consumption. Then, the determination device 300 returns to execute steps 1-3 and subsequent steps.

[0093] Step 1-10: For each sub-model, the determining device 300 can determine the number of computing resources used when the generated running time is within the value interval [Low, High) of the running time corresponding to the sub-model, thereby obtaining the number of computing resources allocated to each sub-model, such as Figure 3 For example, for sub-model 1, the determining device 300 may determine the running time within the value range in the search space of sub-model 1, and thus may determine the amount of computing resources allocated to sub-model 1 according to the running time.

[0094] Furthermore, when the running time or parallel strategy combination under each resource quantity exists in the search space of the sub-model is "None", all parallel strategy combinations representing the resource quantity are pruned. At this time, the determination device 300 can skip the process of calculating the computing resource quantity. For example, when determining that the computing resource quantity used when the generated running time is within the value range [Low, High), the computing resource quantity with a running time of "None" can be skipped.

[0095] In this way, by looping through the above steps, the determination device 300 can obtain the number of computing resources allocated to each sub-model in the AI ​​model and the parallel strategy combination adopted by looping through a binary search, that is, determine the target operation configuration of each sub-model, thereby obtaining the target configuration combination corresponding to the AI ​​model. It can be understood that since the parallel strategy in the target operation configuration of each sub-model is a parallel strategy that generates the minimum operation time when the sub-model runs based on the computing resources indicated by the target operation configuration, the sum of the operation times corresponding to each target operation configuration in the target configuration combination is the smallest. In addition, the determination device 300 generates a target configuration combination based on the maximum number of computing resources, which makes the total amount of computing resources in the target operation configurations of multiple sub-models in the target configuration combination less than or equal to the maximum number of computing resources.

[0096] In actual application, the determination device 300 may also calculate the sum of the number of computing resources allocated to the multiple sub-models finally determined, and further verify whether the sum is greater than the maximum number of computing resources allocated to the AI ​​model. If it is less, the determination device 300 may generate a target configuration combination based on this; if it is greater, the determination device 300 may determine that there is currently no available running configuration, and may output a corresponding prompt message to the user through the client, such as the prompt message "There is currently no available running configuration."

[0097] It can be understood that the above-mentioned binary search to obtain the target configuration combination is only an implementation example and is not intended to be limiting. For example, in other possible implementations, when calculating the intermediate value Mid, it can also be calculated by the intermediate value Mid = (High + Low) / 3 or other applicable formulas. For another example, in other possible implementations, the probability that the total sum Sum is the same as the maximum number of computing resources allocated to the AI ​​model is small. At this time, the determination device 300 may also not need to perform steps 1-6, etc. For another example, the maximum running time High in step 1-1 may also be a value greater than the maximum value of the running time generated by all sub-models, or the minimum running time Low in step 1-1 may also be a value less than the minimum value of the running time generated by all sub-models, or the minimum running time Low in step 1-1 may also be 0, etc.

[0098] In the implementation shown in the above steps S201 to S203, the search space records the operation configuration and the operation time as an example for explanation. In other embodiments, the determination device 300 can use the proxy model as the search space of each sub-model, without calculating the operation time generated by each sub-model based on the various possible numbers of computing resources allocated and the various parallel strategies that may be adopted. In this way, the cost of constructing the search space of each sub-model can be reduced. At this time, the search space of each sub-model can include some functions (or some network layers) in the proxy model, and at least one parallel strategy adopted by each sub-model for the minimum operation time generated by the number of allocated computing resources (such as can be inferred through multiple operation records of the AI ​​model).

[0099] Among them, the proxy model can be used to fit the mapping relationship between the number of computing resources allocated to each sub-model and the running time generated when the sub-model runs based on the number of allocated computing resources when the total number of computing resources allocated to each sub-model in the AI ​​model is less than or equal to the maximum number of computing resources allocated to the AI ​​model.

[0100] In a specific implementation, the determining device 300 constructs proxy models for multiple sub-models in the AI ​​model based on the maximum number of computing resources allocated to the AI ​​model. Since the proxy model's reasoning accuracy may be low, the mapping relationship between the number of computing resources simulated by the proxy model and the execution time may be less accurate. Therefore, the determining device 300 can iteratively train the proxy model using multiple execution records to improve the proxy model's reasoning accuracy.

[0101] Specifically, the determination device 300 can iteratively update the parameters in the proxy model, and, during the iteration process, the determination device 300 can use the proxy model to determine the first resource allocation result and the predicted results of the running time generated when each sub-model runs based on the computing resources allocated as indicated by the first resource allocation result, such as the running time generated when each sub-model runs based on the number of computing resources allocated to it can be inferred based on the mapping relationship fitted by the proxy model. Among them, the first resource allocation result is used to indicate the number of computing resources allocated to each sub-model. In addition, the determination device 300 can also determine the actual result of the minimum running time generated when each sub-model runs based on the computing resources allocated as indicated by the first resource allocation result based on multiple operation records of the AI ​​model. Thus, the determination device 300 can update the parameters in the proxy model based on the predicted results and the actual results.

[0102] The determining device 300 can then perform inference again based on the updated proxy model to obtain a second resource allocation result, which indicates the amount of computing resources allocated to each sub-model. The determining device 300 can then determine whether the proxy model satisfies the iteration termination condition based on the first and second resource allocation results. Furthermore, when the updated proxy model satisfies the iteration termination condition, the determining device 300 can terminate training of the proxy model. If the updated proxy model still does not satisfy the iteration termination condition, the determining device 300 continues to iteratively update the parameter values ​​in the proxy model in the same manner as described above until the proxy model satisfies the iteration termination condition.

[0103] In addition, the determination device 300 can also use multiple operation records of the AI ​​model to determine at least one parallel strategy adopted by each sub-model to generate the minimum operating time when running based on each amount of computing resources. For example, the above-mentioned performance modeling method can be used to determine at least one parallel strategy adopted by each sub-model to generate the minimum operating time when running based on each amount of computing resources. For details, please refer to the above-mentioned relevant descriptions and will not be repeated here.

[0104] To facilitate understanding, the following is an example explanation of how to implement the proxy model.

[0105] The proxy model may include a function y that maps the amount of computing resources allocated to each sub-model in the AI ​​model to the running time generated by the sub-model running based on the allocated computing resources when the total number of computing resources is limited. n =f(x n ) model. Where n refers to the nth sub-model, the AI ​​model includes m sub-models, and m is a positive integer greater than n; x n Refers to the amount of computing resources allocated to the nth sub-model; y n The nth sub-model is based on the number x assigned to it. n The running time generated by the computing resources. At this time, the relationship between the number of computing resources in the search space of each sub-model and the running time can be obtained through the function y in the proxy model n =f(x n ) model to replace it.

[0106] For example, the following is combined Figure 4 , introduces the implementation process of the determination device 300 realizing the search space and determining the target configuration combination by iteratively updating the proxy model.

[0107] Step 2-1: The determining device 300 constructs an agent model. In this embodiment, the determining device 300 can n Expanded into a Laurent function approximation consisting of k terms Among them, a0, ..., a k-1 is the parameter of the proxy model, and k is a positive integer greater than 1.

[0108] At this point, the proxy model can include the following sets of formulas:

[0109]

[0110] Here, N is the maximum number of computing resources allocated to the AI ​​model, for example, it can be the total number of computing nodes allocated to the AI ​​model.

[0111] Step 2-2: The determining device 300 assigns values ​​to various parameters in the proxy model, such as setting random values ​​for various parameters, to initialize the proxy model. 1_0、 a m_k-1 wait.

[0112] Step 2-3: If Figure 4 As shown, given the maximum number N of computing resources allocated to the AI ​​model, the determining device 300 uses the proxy model to determine x1 to x1 using a binary search method. mThe value of y1 to y2 is obtained, that is, the amount of computing resources allocated to each sub-model. m The value of , that is, the predicted running time of each sub-model based on the currently allocated number of computing resources (that is, the above-mentioned prediction result).

[0113] Among them, determine x1 to x m In the process of taking values, the determination device 300 can calculate the maximum running time and the minimum running time of each sub-model when the value of x does not exceed N, based on the parameter values ​​in the proxy model, under the restriction of the maximum number N of computing resources allocated to the AI ​​model. Then, the determination device 300 can determine the maximum value High from all the maximum running times, and select a minimum value as Low from all the minimum running times. Then, the determination device 300 can calculate the intermediate value Mid based on the High and Low, and based on the intermediate value Mid, the determination device 300 can use the proxy model to calculate the minimum number of computing resources that can be allocated when the running time of each sub-model does not exceed the Mid, thereby obtaining x1 to x m The determination device 300 uses a binary search method to determine the value of x1 to x m For the value of Figure 3 The description of the relevant aspects of the illustrated implementation manner will not be repeated here.

[0114] Step 2-4: The determination device 300 determines the actual result of the minimum running time generated by each sub-model based on the number of computing resources currently allocated based on multiple running records of the AI ​​model, such as Figure 4 As shown. For example, the determination device 300 can perform performance modeling for each sub-model based on multiple running records of the AI ​​model, and use the mathematical model obtained by the modeling to infer the running time generated when each sub-model is running based on the currently allocated number of computing resources and the best parallel strategy combination that can be adopted. Among them, the best parallel strategy combination refers to the parallel strategy combination adopted by the sub-model to run and generate the minimum running time when the number of computing resources is fixed, and when the sub-model runs based on the best parallel strategy combination, there will be no memory overflow (OOM), that is, the memory resources required to be used will not exceed the maximum memory resources supported by the number of computing resources.

[0115] Step 2-5: If Figure 4 As shown, the determining device 300 determines the actual result and the predicted result (i.e., y1 to y2) of the running time generated by each sub-model based on the number of computing resources currently allocated. m), update the parameter values ​​in the proxy model. Specifically, the actual result and predicted result (y1) corresponding to the first sub-model are used to update the parameter a in the proxy model. 1_0 to a 1_k-1 The value of the nth sub-model is used to compare the actual result with the predicted result (y n ), update a in the proxy model n_0 to a n_k-1 The values ​​of the other parameters are deduced in the same way. For example, in the process of updating the parameter values, the determination device 300 may calculate the difference between the actual result and the predicted result, and calculate the gradient data corresponding to the parameter in the proxy model based on the difference, thereby updating the parameter value in the proxy model using the gradient data.

[0116] Step 2-6: The determining device 300 determines whether the proxy model satisfies the iteration termination condition. If so, the determining device 300 executes step 2-7. If not, the determining device 300 may re-execute steps 2-3 to 2-6 using the updated proxy model.

[0117] Step 2-7: The determining device 300 uses the currently trained proxy model to determine the maximum number N of computing resources allocated to the AI ​​model using a binary search method. m The value of x1 to x m The value of is taken as the number of computing resources allocated to each sub-model in the final AI model, and the optimal parallel strategy combination calculated by the mathematical model obtained by modeling based on the number of computing resources currently allocated to the sub-model is determined as the parallel strategy combination finally adopted by the sub-model, so as to obtain the target operation configuration of each sub-model, and thus obtain the target configuration combination corresponding to the AI ​​model, such as Figure 4 shown.

[0118] Among them, x1 to x are determined in steps 2-7. m The implementation process of the value of is the same as that of determining x1 to x in steps 2-3 above. m The implementation process of obtaining the value of is similar. For details, please refer to the description of the relevant parts above and will not be repeated here.

[0119] For example, the iteration termination condition may be a convergence condition of the proxy model, such as the resource allocation results calculated by the proxy model in two consecutive rounds (or more than two rounds) are the same, such as the first resource allocation result and the second resource allocation result are the same, that is, the values ​​of x1 to x2 calculated by the proxy model in two consecutive rounds are the same. mThe value of is the same. At this time, the resource allocation result obtained by the last calculation of the proxy model can be the second resource allocation result. Accordingly, the target configuration combination includes x1 to x obtained by the last calculation of the proxy model. m The value of , and the combination of parallel strategies used by each sub-model to produce the minimum running time calculated by the mathematical model obtained by modeling. In this way, when there is less training data, overfitting of the surrogate model can be avoided.

[0120] Alternatively, the iteration termination condition can be the value of the last calculation from x1 to x2 using the proxy model. m The actual result of the running time corresponding to the value of (such as the second resource allocation result) is worse than the value of x1 to x2 obtained by the proxy model in the penultimate calculation. m The actual result of the running time corresponding to the value of (such as the first resource allocation result above), such as the value of x1 to x2 obtained in the last calculation using the agent model m The maximum running time corresponding to the value of is greater than the value of x1 to x calculated by the proxy model in the second to last m The maximum value of the running time corresponding to the value of . Accordingly, the target configuration combination includes x1 to x obtained by the penultimate calculation of the above agent model. m The optimal parallel strategy combination for each sub-model is calculated using the mathematical model derived from the modeling. This prevents overfitting of the surrogate model when training data is limited. Alternatively, the iteration termination condition can be achieved through other means and is not limited to this.

[0121] In the above embodiment, the proxy model may include m*k parameters during the initialization phase. In actual application scenarios, excessive number of parameters in the proxy model during the initialization phase may easily lead to slow training of the proxy model, thereby affecting the overall efficiency of the determination device 300 in determining the target configuration combination. Therefore, by limiting the number of parameters in the proxy model during the initialization phase, the iterative update efficiency of the parameters in the proxy model can be improved, thereby improving the overall efficiency of determining the target configuration combination. For details, please refer to the following Figure 5 Another implementation process shown is to realize the search space and determine the target configuration combination by iteratively updating the agent model.

[0122] like Figure 5 As shown, the determining device 300 can determine the target configuration combination by iteratively updating the parameters in the agent model based on the following steps.

[0123] Step 3-1: Determine that the device 300 constructs an agent model. In this embodiment, the implementation of step 3-1 is similar to the implementation of step 2-1 above. For details, please refer to the relevant description above and will not be repeated here.

[0124] Step 3-2: The determining device 300 determines the number of parameters in the proxy model. In this embodiment, only a portion of the parameters in the proxy model may be retained to quickly train a usable proxy model. For example, for each sub-model function in the proxy model, the determining device 300 may retain one parameter in the function. In this case, the multiple sets of formulas included in the proxy model are as follows:

[0125]

[0126] Here, N is the maximum number of computing resources allocated to the AI ​​model, for example, it can be the total number of computing nodes allocated to the AI ​​model.

[0127] Step 3-3: If Figure 5 As shown, the determining device 300 initializes the proxy model, specifically, it can be initialized a1_1, ..., a m_l The values ​​of other parameters are 0, that is, the m parameters in the proxy model are initialized.

[0128] For example, assuming that the number of computing resources allocated to the AI ​​model is specifically the number of computing nodes, when initializing the proxy model, for the formula corresponding to each sub-model, the determination device 300 can start with the number of computing nodes being 1, and based on multiple run records of the AI ​​model, while ensuring that there is no memory overflow (OOM), calculate whether there is an available parallel strategy combination when the sub-model is running based on the 1 computing node allocated to it. If so, the determination device 300 can infer the running time generated when the sub-model is running based on the 1 computing node allocated to it and adopting the parallel strategy combination based on the multiple run records, and assign values ​​to the parameters in the formula corresponding to the sub-model based on the running time and the number of computing nodes. If not, the determination device 300 increases the number of computing resources allocated to the sub-model to 2, and based on multiple run records of the AI ​​model, while ensuring that there is no memory overflow, calculate whether there is an available parallel strategy combination when the sub-model is running based on the 2 computing nodes allocated to it. If so, the determination device 300 can calculate, based on multiple run records, the runtime of the sub-model when running on the two allocated computing nodes and using the parallel strategy combination, and assign values ​​to the parameters in the formula corresponding to the sub-model based on the runtime and the number of computing nodes. Similarly, the parameters in the formula corresponding to each sub-model in the proxy model can be assigned values, thereby completing the initialization of the proxy model. Because the first two terms in the Laurent function approximation have a relatively large impact on the accuracy of the proxy model, prioritizing the updating of the parameter values ​​in the terms with greater impact on the results can quickly obtain a usable proxy model and improve the overall efficiency of proxy model training.

[0129] Step 3-4: Given the maximum number N of computing resources allocated to the AI ​​model, the determining device 300 uses the proxy model to determine x1 to x1 using a binary search method. m The value of y1 to y2 is obtained, that is, the amount of computing resources allocated to each sub-model. m The value of , that is, the predicted running time of each sub-model when running based on the number of computing resources currently allocated (that is, the above-mentioned prediction result).

[0130] Step 3-5: The determination device 300 determines the actual result of the minimum running time generated by each sub-model based on the number of computing resources currently allocated based on multiple running records of the AI ​​model.

[0131] Step 3-6: The determination device 300 determines the actual and predicted results of the running time generated by each sub-model based on the currently allocated number of computing resources (i.e., y1 to y m), update the parameter values ​​in the proxy model.

[0132] Step 3-7: The determining device 300 determines whether the proxy model satisfies the iteration termination condition. If so, the determining device 300 executes step 3-8. If not, the determining device 300 may re-execute steps 3-4 to 3-7 using the updated proxy model. Figure 5 shown.

[0133] Step 3-8: Figure 5 As shown, the determining device 300 uses the currently trained proxy model to determine the maximum number N of computing resources allocated to the AI ​​model using a binary search method. m The value of x1 to x m The value of is used as the number of computing resources allocated to each sub-model in the final AI model, and the optimal parallel strategy combination calculated by the mathematical model obtained by modeling based on the number of computing resources currently allocated to the sub-model is determined as the parallel strategy combination finally adopted by the sub-model, so as to obtain the target operation configuration of each sub-model, and thus obtain the target configuration combination corresponding to the AI ​​model.

[0134] The specific implementation of steps S304 to S308 is similar to the specific implementation of steps 2-3 to 2-7 above, and reference may be made to the above-mentioned related descriptions, which will not be repeated here.

[0135] It should be noted that, since in steps 3-2 and 3-3, the formula corresponding to each sub-model only retains one parameter in the Laurent function approximation formula for initialization, when updating the parameter values ​​in the formula in multiple rounds, not only can the current parameter values ​​be updated, but other parameters in the formula can also be gradually added, that is, the values ​​of other parameters can be gradually updated from 0 to non-zero values. For example, in the first round of updating the formula corresponding to sub-model 1, not only can a be updated, but also 1_1 The value of a 1_2 The value of is updated from 0 to a non-zero value, and the values ​​of the other parameters in the formula remain 0. When updating the formula corresponding to sub-model 1 in the second round, not only can the existing parameter a be updated 1_1 and a 1_2 The value of a 1_0 The value of is updated from 0 to a non-zero value, and the values ​​of the other parameters in the formula remain 0. When updating the formula corresponding to sub-model 1 in the third round, not only can the existing parameter a be updated 1_0 、a 1_1 and a 1_2 The value of a 1_3The value of is updated from 0 to a non-zero value, and the values ​​of the other parameters in the formula remain 0, and so on. Before the proxy model is updated for the k-1th round of parameters, if the proxy model has satisfied the iteration termination condition, the determination device 300 can update the values ​​of x1 to x2 obtained by the current proxy model. m The value of is used as the number of computing resources allocated to each sub-model in the final AI model, and the parallel strategy combination adopted by each sub-model is determined. When the proxy model is updated in the k-1th round and subsequent rounds, the formula corresponding to each sub-model already includes the parameter k. At this time, the determination device 300 only needs to update the values ​​of the k parameters in the formula corresponding to each sub-model without adding new parameters to the formula. Alternatively, in other implementations, when updating the parameter values ​​in the proxy model in each round of iteration, 2 or more parameter values ​​can be updated, and this is not limited.

[0136] It should be noted that the above two implementations are only examples of how to use the proxy model to determine the target configuration combination and are not intended to be limiting. For example, in other implementations, the function y n It can also be expanded into an approximate formula of other functions, that is, the formula corresponding to each sub-model is not limited to the above example. For example, in other implementations, when retaining some parameters in the proxy model and initializing them, the determination device 300 can also assign values ​​to each parameter without calculating the running time, but can assign the parameter randomly or manually based on expert experience, etc., and there is no limitation on this. For example, in other implementations, the determination device 300 can also use the proxy model to determine x1 to x2 in other ways. m If other search algorithms are used to determine the value of x1 to x m The value of is not limited to the above binary search method. For example, in other possible implementations, due to the additional requirements on the amount of computing resources allocated to some sub-models in actual application scenarios, such as the fact that there may not be a suitable parallel strategy when some sub-models are running based on the allocated computing resources, the determination device 300 can also use the proxy model in combination with the pruning strategy to determine the number of x1 to x2. m For example, the value of x1 corresponding to sub-model 1 must meet the value requirements indicated by the pruning strategy. For example, in other possible implementations, a proxy model corresponding to the AI ​​model can be pre-built and trained. When determining the target configuration combination for the AI ​​model, the trained proxy model can be used to determine the target operating configuration for each sub-model in the AI ​​model through binary search or other methods, thereby obtaining the target configuration combination corresponding to the AI ​​model.

[0137] In actual application, this embodiment may further include the following steps.

[0138] S204 : The determination device 300 outputs the target configuration combination to the client 101 .

[0139] In this way, the client 101 can present the target configuration combination to the user to recommend to the user the amount of computing resources that are suitable for allocation to each sub-model in the AI ​​model and at least one parallel strategy that is suitable for adoption.

[0140] S205 : The client 101 provides the target configuration combination confirmed by the user to the scheduler 100 .

[0141] S206: The scheduler 100 schedules the AI ​​model to the corresponding computing node for execution according to the target configuration combination to achieve distributed training of the AI ​​model.

[0142] In this way, during the training of the AI ​​model, the user does not need to manually determine the amount of computing resources used to train each sub-model and at least one parallel strategy to be adopted for the AI ​​model. This can not only effectively reduce the manpower cost required in the AI ​​model training process, but also, based on the automatically searched target configuration combination, the distributed training of the AI ​​model can ensure that the overall training efficiency of the AI ​​model can reach a high level.

[0143] It should be noted that Figure 2 The method for determining the operating configuration of the AI ​​model shown is only an example and is not intended to be limiting.

[0144] For example, in other embodiments, the determination device 300 may also execute step S202 first and then execute step S201, or the user may configure the maximum number of computing resources allocated to the AI ​​model and the configuration parameters on the client 101 at the same time, so that the determination device 300 can simultaneously obtain the maximum number of computing resources allocated to the AI ​​model and the configuration parameters, etc.

[0145] For example, in other embodiments, after determining the target configuration combination, the determination device 300 can send it directly to the scheduler 100, and the scheduler 100 can schedule the AI ​​model to run on multiple computing nodes according to the target configuration combination without the need for manual determination by the user, thereby achieving fully automated training for the AI ​​model and reducing the operational requirements for the user.

[0146] It is worth noting that other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0147] Combination of the above Figures 1 to 5 The method for determining the operating configuration of the AI ​​model provided in an embodiment of the present application is introduced. Next, the structure of the device for determining the operating configuration of the AI ​​model and the computing device provided in an embodiment of the present application are introduced in conjunction with the accompanying drawings.

[0148] See also Figure 6 , shows a schematic diagram of the structure of a device for determining the operating configuration of an AI model. Figure 6 As shown, the apparatus 600 includes:

[0149] Acquisition module 601 is configured to acquire a search space for each of multiple sub-models included in the AI ​​model, where the search space for each sub-model includes multiple operating configurations for each sub-model and the operating time consumed when the sub-model is run based on each operating configuration. The operating configuration includes the number of computing resources and a parallelization strategy; and acquire a maximum number of computing resources allocated to the AI ​​model.

[0150] A selection module 602 is used to select a target operating configuration for each sub-model in each search space based on the operating time of each sub-model under each operating configuration and the maximum number of computing resources in the search space of each sub-model. The target operating configurations of multiple sub-models constitute a target configuration combination. In the target configuration combination, the total amount of computing resources in the target operating configurations of the multiple sub-models is less than the maximum number of computing resources, and the sum of the operating times corresponding to the various target operating configurations in the target configuration combination is the smallest.

[0151] In one possible implementation, the selection module 602, when selecting a target running configuration for each sub-model in each search space based on the running time of each sub-model in each running configuration and the maximum number of computing resources in the search space of each sub-model, is specifically configured to:

[0152] Determine the optimal running time based on the running time of each sub-model in the search space;

[0153] Among the running configurations of each sub-model whose time consumption is less than the optimal one, the running configuration whose total number of computing resources is less than or equal to the maximum number of computing resources is selected as the target running configuration of each sub-model.

[0154] In a possible implementation, when the selection module 602 determines the optimal time consumption based on the running time consumption in the search space of each sub-model, it is specifically configured to:

[0155] Traverse the search space of each sub-model to obtain the maximum and minimum running times among the running times of multiple sub-models;

[0156] Determine the optimal running time according to the maximum running time and the minimum running time, where the optimal running time is between the maximum running time and the minimum running time;

[0157] The selection module 602 selects, from the running configurations of each sub-model that are less than the optimal time consumption, the running configuration in which the total number of computing resources is less than or equal to the maximum number of computing resources as the target running configuration of each sub-model, specifically for:

[0158] Determine the total amount of computing resources allocated to each sub-model based on the optimal time consumption;

[0159] When the total number is greater than the maximum number of computing resources, determine the first part of the space whose running time is less than the optimal time from the search space of each sub-model, and determine the target running configuration of each sub-model based on the first part of the space of each sub-model and the maximum number of computing resources;

[0160] When the total number is less than the maximum number of computing resources, a second part of the space where the running time is greater than the optimal time is determined from the search space of each sub-model, and the target running configuration of each sub-model is determined based on the second part of the space of each sub-model and the maximum number of computing resources.

[0161] In a possible implementation, when acquiring the search space of each sub-model in the multiple sub-models included in the AI ​​model, the acquisition module 601 is specifically configured to:

[0162] Obtain multiple run records of the AI ​​model. The multiple run records are used to indicate the run configuration and run time of each sub-model when the AI ​​model is running;

[0163] Based on multiple running records of the AI ​​model, the search space of each sub-model in multiple sub-models is constructed.

[0164] In one possible implementation, the multiple sub-models include a first sub-model, the search space of the first sub-model includes multiple operating configurations of the first sub-model, different operating configurations of the multiple operating configurations of the first sub-model include different amounts of computing resources, each operating configuration of the first sub-model includes at least one parallel strategy adopted to generate minimum operating time, the multiple operating configurations include a first operating configuration, and the first operating configuration includes a first amount of computing resources;

[0165] When constructing the search space of each sub-model in multiple sub-models based on multiple running records of the AI ​​model, the acquisition module 601 is specifically used to:

[0166] Determining a plurality of parallel strategy combinations that can be adopted when the first sub-model is run based on the first amount of computing resources;

[0167] According to multiple running records, the running time of the parallel computing first sub-model is respectively generated when the first number of computing resources is used and the combination of multiple parallel strategies is used;

[0168] During the parallel computing process, the multiple parallel strategy combinations adopted by the first sub-model based on the first number of computing resources are sorted according to the running time to obtain the minimum running time and the first running configuration, and the first running configuration produces the parallel strategy combination adopted with the minimum running time.

[0169] In one possible implementation, when constructing the search space of each sub-model in the plurality of sub-models based on multiple running records of the AI ​​model, the acquisition module 601 is specifically configured to:

[0170] Based on multiple running records of the AI ​​model, construct the initial search space of each sub-model in multiple sub-models;

[0171] Get the pruning strategy for the initial search space of each sub-model;

[0172] According to the pruning strategy, the initial search space of each sub-model is pruned to obtain the search space of each sub-model.

[0173] In one possible implementation, when constructing the search space of each sub-model in the plurality of sub-models based on multiple running records of the AI ​​model, the acquisition module 601 is specifically configured to:

[0174] Constructing proxy models for multiple sub-models based on the maximum number of computing resources. The proxy models are used to fit the mapping relationship between the number of computing resources allocated to each sub-model and the running time generated when each sub-model is running, when the total number of computing resources allocated to each sub-model is less than or equal to the maximum number of computing resources.

[0175] The proxy model is iteratively trained using multiple running records of the AI ​​model, and at least one parallel strategy is determined that produces the minimum running time when each sub-model is running based on each amount of computing resources.

[0176] In one possible implementation, when the acquisition module 601 uses multiple running records of the AI ​​model to iteratively train the proxy model, it is specifically configured to:

[0177] Determine, using the proxy model, a first resource allocation result and a predicted result of the running time generated when each sub-model runs the computing resources allocated based on the first resource allocation result, wherein the first resource allocation result is used to indicate the amount of computing resources allocated to each sub-model;

[0178] Determine, based on multiple running records of the AI ​​model, an actual result of the minimum running time generated when each sub-model runs the computing resources allocated based on the first resource allocation result indication;

[0179] Update the parameters in the proxy model based on the predicted results and actual results;

[0180] Using the updated proxy model, obtaining a second resource allocation result, where the second resource allocation result is used to indicate the amount of computing resources allocated to each model component;

[0181] When it is determined that the proxy model meets the iteration termination condition according to the first resource allocation result and the second resource allocation result, the iterative training of the proxy model is terminated.

[0182] In one possible implementation, when acquiring multiple running records of the AI ​​model, the acquisition module 601 is specifically configured to:

[0183] Get configuration parameters, including the number of computing resources used by multiple sub-models and the parallelization strategy when running the AI ​​model;

[0184] In the process of running the AI ​​model based on the configuration parameters, multiple running records of the AI ​​model are collected.

[0185] because Figure 6 The device 600 shown corresponds to the above Figure 3 The determining device 300 in the embodiment shown, therefore Figure 6 For the specific implementation of the device 600 and its technical effects, see the above Figure 3 The description of the relevant parts in the illustrated embodiment will not be repeated here.

[0186] Figure 7 This is a hardware structure diagram of a computing device 700 provided in this application. The computing device 700 can, for example, implement the above Figures 2 to 5 The method steps etc. executed by the determination device 300 in the illustrated embodiment.

[0187] like Figure 7 As shown, the computing device 700 includes a processor 701, a memory 702, and a communication interface 703. The processor 701, the memory 702, and the communication interface 703 communicate via a bus 704, and may also communicate via other means such as wireless transmission. The memory 702 is used to store instructions, and the processor 701 is used to execute the instructions stored in the memory 702. Furthermore, the computing device 700 may also include a memory unit 705, and the memory unit 705 may be connected to the processor 701, the storage medium 702, and the communication interface 703 via a bus 704. The memory 702 stores program code, and the processor 701 may use the program code stored in the memory 702 to perform the following operations:

[0188] Obtaining the search space of each of the multiple sub-models included in the AI ​​model. The search space of each sub-model includes multiple running configurations of each sub-model and the running time of the sub-model based on each running configuration. The running configuration includes the number of computing resources and parallelization strategy.

[0189] Get the maximum amount of computing resources allocated to the AI ​​model;

[0190] Based on the running time of each sub-model under each running configuration and the maximum number of computing resources in the search space of each sub-model, a target running configuration is selected for each sub-model in each search space. The target running configurations of multiple sub-models constitute a target configuration combination. In the target configuration combination, the total amount of computing resources in the target running configurations of multiple sub-models is less than or equal to the maximum number of computing resources, and the sum of the running times corresponding to the target running configurations in the target configuration combination is minimized.

[0191] It should be understood that in this embodiment, the processor 701 may be a CPU, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete device components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0192] The memory 702 may include a read-only memory and a random access memory, and provides instructions and data to the processor 701. The memory 702 may also include a nonvolatile random access memory.

[0193] The memory 702 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0194] The communication interface 703 is used to communicate with other devices connected to the computing device 700. The bus 704 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 704 in the figure.

[0195] It should be understood that the computing device 700 according to the embodiment of the present application may correspond to the determining device 300 in the embodiment of the present application, and may correspond to the execution of the process according to the embodiment of the present application. Figures 2 to 5 The method performed by the determining device 300 and the above and other operations and / or functions implemented by the computing device 700 are respectively to achieve Figures 2 to 5 For the sake of brevity, the process of the corresponding method in will not be repeated here.

[0196] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned method for determining the operating configuration of the AI ​​model.

[0197] The present application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the computer program product fully or partially generates the process or function described in the present application.

[0198] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0199] The computer program product may be a software installation package, which may be downloaded and executed on a computing device when any of the aforementioned methods for determining the operating configuration of an AI model is required.

[0200] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0201] The terms used in the above embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; the character " / " generally indicates that the objects before and after are in an "or" relationship. In the embodiments of the present application. "Simultaneously" refers to the same time period, including the situation at the same moment.

[0202] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0203] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for determining the operating configuration of an artificial intelligence (AI) model, characterized in that: The method comprises: Obtaining a search space for each of the multiple sub-models included in the AI ​​model, where the search space for each sub-model includes multiple operating configurations of each sub-model and the operating time consumed when the sub-model is run based on each operating configuration, where the operating configuration includes the number of computing resources and a parallel strategy; Obtaining a maximum amount of computing resources allocated to the AI ​​model; According to the running time of each sub-model under each running configuration in the search space of each sub-model and the maximum number of computing resources, a target running configuration is selected for each sub-model in each search space, and the target running configurations of the multiple sub-models constitute a target configuration combination. In the target configuration combination, the total amount of computing resources in the target running configurations of the multiple sub-models is less than or equal to the maximum number of computing resources, and the sum of the running times corresponding to the target running configurations in the target configuration combination is the smallest.

2. The method according to claim 1, characterized in that The selecting a target running configuration for each sub-model in each search space according to the running time of each sub-model in each running configuration in the search space of each sub-model and the maximum number of computing resources includes: Determining the optimal time consumption according to the running time consumption in the search space of each sub-model; Among the running configurations of each sub-model that have a shorter time consumption than the optimal one, the running configuration whose total number of computing resources is less than or equal to the maximum number of computing resources is selected as the target running configuration of each sub-model.

3. The method according to claim 2, characterized in that Determining the optimal time consumption according to the operation time consumption in the search space of each sub-model includes: Traversing the search space of each sub-model to obtain the maximum running time and the minimum running time among the running times of the multiple sub-models; Determining the optimal running time according to the maximum running time and the minimum running time, where the optimal running time is between the maximum running time and the minimum running time; The step of selecting, from the running configurations of each sub-model that are less than the optimal time consumption, a running configuration in which the sum of the number of computing resources is less than or equal to the maximum number of computing resources as the target running configuration of each sub-model includes: Determining the total amount of computing resources allocated to each sub-model according to the optimal time consumption; When the sum of the quantities is greater than the maximum quantity of the computing resources, determining a first portion of the search space of each sub-model whose running time is less than the optimal time, and determining a target running configuration of each sub-model according to the first portion of the search space of each sub-model and the maximum quantity of the computing resources; When the sum of the quantities is less than the maximum number of computing resources, a second portion of the space in which the running time is greater than the optimal time is determined from the search space of each sub-model, and the target running configuration of each sub-model is determined based on the second portion of the space of each sub-model and the maximum number of computing resources.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining of the search space of each sub-model in the plurality of sub-models included in the AI ​​model includes: Obtain multiple running records of the AI ​​model, where the multiple running records are used to indicate the running configuration and running time adopted by each sub-model when the AI ​​model is running; According to multiple running records of the AI ​​model, a search space of each sub-model in the multiple sub-models is constructed.

5. The method according to claim 4, characterized in that The multiple sub-models include a first sub-model, the search space of the first sub-model includes multiple running configurations of the first sub-model, different running configurations of the multiple running configurations of the first sub-model include different amounts of computing resources, each running configuration of the first sub-model includes at least one parallel strategy adopted to generate minimum running time, the multiple running configurations include a first running configuration, and the first running configuration includes a first amount of computing resources; The step of constructing a search space for each of the multiple sub-models based on multiple running records of the AI ​​model includes: Determining a plurality of parallel strategy combinations that can be adopted when the first sub-model is run based on a first number of computing resources; According to the multiple running records, parallel calculation of the running time respectively generated when the first sub-model is run based on the first number of computing resources and the combination of the multiple parallel strategies; During the parallel computing process, the multiple parallel strategy combinations adopted by the first sub-model based on the first number of computing resources are sorted according to the running time to obtain the minimum running time and the first running configuration, and the first running configuration generates the parallel strategy combination adopted for the minimum running time.

6. The method according to claim 4 or 5, characterized in that The step of constructing a search space for each of the multiple sub-models based on multiple running records of the AI ​​model includes: Constructing an initial search space for each of the multiple sub-models based on multiple running records of the AI ​​model; Obtaining a pruning strategy for the initial search space of each sub-model; According to the pruning strategy, the initial search space of each sub-model is pruned to obtain the search space of each sub-model.

7. The method according to claim 4, characterized in that The step of constructing a search space for each of the multiple sub-models based on multiple running records of the AI ​​model includes: Constructing a proxy model for the multiple sub-models according to the maximum number of computing resources, the proxy model being used to fit a mapping relationship between the number of computing resources allocated to each sub-model and the running time generated when each sub-model is running, when the total number of computing resources allocated to each sub-model is less than or equal to the maximum number of computing resources; The proxy model is iteratively trained using multiple running records of the AI ​​model, and at least one parallel strategy adopted by each sub-model to generate the minimum running time when running based on each amount of computing resources is determined.

8. The method according to claim 7, characterized in that The iterative training of the proxy model using multiple running records of the AI ​​model includes: Determine, by using the proxy model, a first resource allocation result and a predicted result of the running time generated when each sub-model runs the computing resources allocated based on the first resource allocation result, wherein the first resource allocation result is used to indicate the amount of computing resources allocated to each sub-model; Determine, based on multiple running records of the AI ​​model, an actual result of minimum running time generated when each sub-model runs the computing resources allocated based on the first resource allocation result indication; updating parameters in the proxy model according to the predicted result and the actual result; Obtaining a second resource allocation result using the updated proxy model, where the second resource allocation result is used to indicate the amount of computing resources allocated to each model component; When it is determined that the proxy model meets the iteration termination condition according to the first resource allocation result and the second resource allocation result, the iterative training of the proxy model is terminated.

9. The method according to any one of claims 4 to 8, characterized in that The acquisition of multiple running records of the AI ​​model includes: Obtaining configuration parameters, where the configuration parameters include the number of computing resources used by each of the multiple sub-models when running the AI ​​model and a parallel strategy; In the process of running the AI ​​model based on the configuration parameters, multiple running records of the AI ​​model are collected.

10. A device for determining the operating configuration of an artificial intelligence (AI) model, characterized in that: The device comprises: an acquisition module, configured to acquire a search space for each of a plurality of sub-models included in the AI ​​model, wherein the search space for each sub-model includes multiple operating configurations for each sub-model and an operating time consumed when the sub-model is run based on each operating configuration, wherein the operating configuration includes the number of computing resources and a parallel strategy; and to acquire a maximum number of computing resources allocated to the AI ​​model; A selection module is used to select a target operating configuration for each sub-model in each search space based on the operating time of each sub-model under each operating configuration in the search space of each sub-model and the maximum number of computing resources. The target operating configurations of the multiple sub-models constitute a target configuration combination. In the target configuration combination, the total amount of computing resources in the target operating configurations of the multiple sub-models is less than the maximum number of computing resources, and the sum of the operating times corresponding to the target operating configurations in the target configuration combination is the smallest.

11. The device according to claim 10, characterized in that The selection module is specifically configured to select a target running configuration for each sub-model in each search space according to the running time of each sub-model in each running configuration in the search space of each sub-model and the maximum number of computing resources: Determining the optimal time consumption according to the running time consumption in the search space of each sub-model; Among the running configurations of each sub-model that have a shorter time consumption than the optimal one, the running configuration whose total number of computing resources is less than or equal to the maximum number of computing resources is selected as the target running configuration of each sub-model.

12. The device according to claim 11, characterized in that When the selection module determines the optimal time consumption based on the operation time consumption in the search space of each sub-model, it is specifically used to: Traversing the search space of each sub-model to obtain the maximum running time and the minimum running time among the running times of the multiple sub-models; Determining the optimal running time according to the maximum running time and the minimum running time, where the optimal running time is between the maximum running time and the minimum running time; When the selection module selects, from the running configurations of each sub-model that are less than the optimal time consumption, a running configuration in which the total number of computing resources is less than or equal to the maximum number of computing resources as the target running configuration of each sub-model, the selection module is specifically configured to: Determining the total amount of computing resources allocated to each sub-model according to the optimal time consumption; When the sum of the quantities is greater than the maximum quantity of the computing resources, determining a first portion of the search space of each sub-model whose running time is less than the optimal time, and determining a target running configuration of each sub-model according to the first portion of the search space of each sub-model and the maximum quantity of the computing resources; When the sum of the quantities is less than the maximum number of computing resources, a second portion of the space in which the running time is greater than the optimal time is determined from the search space of each sub-model, and the target running configuration of each sub-model is determined based on the second portion of the space of each sub-model and the maximum number of computing resources.

13. The device according to any one of claims 10 to 12, characterized in that When acquiring the search space of each sub-model in the multiple sub-models included in the AI ​​model, the acquisition module is specifically used to: Obtain multiple running records of the AI ​​model, where the multiple running records are used to indicate the running configuration and running time adopted by each sub-model when the AI ​​model is running; According to multiple running records of the AI ​​model, a search space of each sub-model in the multiple sub-models is constructed.

14. The device according to claim 13, characterized in that The multiple sub-models include a first sub-model, the search space of the first sub-model includes multiple running configurations of the first sub-model, different running configurations of the multiple running configurations of the first sub-model include different amounts of computing resources, each running configuration of the first sub-model includes at least one parallel strategy adopted to generate minimum running time, the multiple running configurations include a first running configuration, and the first running configuration includes a first amount of computing resources; When constructing the search space of each sub-model in the multiple sub-models based on multiple running records of the AI ​​model, the acquisition module is specifically used to: Determining a plurality of parallel strategy combinations that can be adopted when the first sub-model is run based on a first number of computing resources; According to the multiple running records, parallel calculation of the running time respectively generated when the first sub-model is run based on the first number of computing resources and the combination of the multiple parallel strategies; During the parallel computing process, the multiple parallel strategy combinations adopted by the first sub-model based on the first number of computing resources are sorted according to the running time to obtain the minimum running time and the first running configuration, and the first running configuration generates the parallel strategy combination adopted for the minimum running time.

15. The device according to claim 13 or 14, characterized in that When constructing the search space of each sub-model in the multiple sub-models based on multiple running records of the AI ​​model, the acquisition module is specifically used to: Constructing an initial search space for each of the multiple sub-models based on multiple running records of the AI ​​model; Obtaining a pruning strategy for the initial search space of each sub-model; According to the pruning strategy, the initial search space of each sub-model is pruned to obtain the search space of each sub-model.

16. The device according to claim 13, characterized in that When constructing the search space of each sub-model in the multiple sub-models based on multiple running records of the AI ​​model, the acquisition module is specifically used to: Constructing a proxy model for the multiple sub-models according to the maximum number of computing resources, the proxy model being used to fit a mapping relationship between the number of computing resources allocated to each sub-model and the running time generated when each sub-model is running, when the total number of computing resources allocated to each sub-model is less than or equal to the maximum number of computing resources; The proxy model is iteratively trained using multiple running records of the AI ​​model, and at least one parallel strategy adopted by each sub-model to generate the minimum running time when running based on each amount of computing resources is determined.

17. The device according to claim 16, characterized in that When the acquisition module uses multiple running records of the AI ​​model to iteratively train the proxy model, it is specifically used to: Determine, by using the proxy model, a first resource allocation result and a predicted result of the running time generated when each sub-model runs the computing resources allocated based on the first resource allocation result, wherein the first resource allocation result is used to indicate the amount of computing resources allocated to each sub-model; Determine, based on multiple running records of the AI ​​model, an actual result of minimum running time generated when each sub-model runs the computing resources allocated based on the first resource allocation result indication; updating parameters in the proxy model according to the predicted result and the actual result; Obtaining a second resource allocation result using the updated proxy model, where the second resource allocation result is used to indicate the amount of computing resources allocated to each model component; When it is determined that the proxy model meets the iteration termination condition according to the first resource allocation result and the second resource allocation result, the iterative training of the proxy model is terminated.

18. The device according to any one of claims 13 to 17, characterized in that When acquiring multiple running records of the AI ​​model, the acquisition module is specifically used to: Obtaining configuration parameters, where the configuration parameters include the number of computing resources used by each of the multiple sub-models when running the AI ​​model and a parallel strategy; In the process of running the AI ​​model based on the configuration parameters, multiple running records of the AI ​​model are collected.

19. A computer-readable storage medium, characterized in that The method comprises instructions which, when executed on at least one computing device, cause the at least one computing device to perform the steps of the method according to any one of claims 1 to 9.

20. A computer program product comprising instructions, characterized in that When the method is executed on at least one computing device, the method enables the at least one computing device to execute the method according to any one of claims 1 to 9.

Images