Method and device for determining operator shape of AI model and related equipment

Abstract

The invention discloses a method and device for determining an operator shape of an AI model and related equipment, and relates to the technical field of artificial intelligence. Determining the degree of influence of each parallel strategy on the shape of a first operator included in the AI model in a plurality of parallel strategies provided by a computing system running the AI model; obtaining a target parallel configuration of the AI model, wherein the target parallel configuration comprises a configuration value of each parallel strategy in a plurality of parallel strategies; and determining the shape of the first operator under the target parallel configuration according to the configuration value of each parallel strategy in the plurality of parallel strategies, the basic shape of the first operator and the influence degree of each parallel strategy on the shape of the first operator. Therefore, according to the influence degree of each parallel strategy on the shape of the operator in the AI model, the shape of the operator under the target parallel configuration can be determined without depending on the specific structure of the AI model, so that the generalization and universality of determining the shape of the operator can be effectively improved, and meanwhile, the efficiency of determining the shape of the operator can also be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and related equipment for determining the operator shape of an AI model. Background Art

[0002] With the development of artificial intelligence (AI) technology, the scale of parameters in AI models continues to increase. At this time, multiple computing nodes can be used to adopt one or more parallel strategies to perform distributed training on AI models. Among them, the parallel strategy refers to a strategy that decomposes the training tasks of the AI ​​model and distributes them to multiple computing nodes for simultaneous execution to improve the overall training efficiency of the AI ​​model. For example, it can be tensor parallelism (TP), context parallelism (CP), etc. Each computing node executes a corresponding operator sequence based on the assigned training task to participate in the training of the AI ​​model. The operator sequences executed by different computing nodes may be different.

[0003] Normally, using different parallel strategies for distributed training of AI models will result in different computing times, resulting in differences in the overall training efficiency of the AI ​​model. Therefore, it is usually possible to calculate the shape of each operator executed by each computing node through theoretical analysis based on the structure of the AI ​​model, and then calculate the total time taken for each computing node to execute the operator sequence based on the shape of the operator. In this way, when configuring the AI ​​model in parallel (that is, configuring the parallel strategy used for training the AI ​​model), for different parallel configurations, the total time taken for each computing node to execute the operator sequence can be calculated through theoretical analysis, thereby calculating the overall time taken to train the AI ​​model, thereby determining the parallel configuration that produces the minimum overall time, and configuring the corresponding parallel strategy for the AI ​​model based on this parallel configuration.

[0004] However, determining the operator shape through theoretical analysis requires the user to provide the specific structure of the AI ​​model. This makes it impossible to determine the shape of the operator executed when the compute node trains an AI model with a different structure, resulting in low generalizability of the operator shape determination. Furthermore, providing the specific structure of the AI ​​model is time-consuming, which affects the efficiency of determining the operator shape and, consequently, the overall efficiency of training the AI ​​model. Summary of the Invention

[0005] This application provides a method for determining the operator shape of an AI model to improve the generalization of the operator shape executed by the computing node and improve the overall training efficiency of the AI ​​model. In addition, this application also provides a corresponding apparatus for determining the operator shape of the AI ​​model, a computing device, a computer-readable storage medium, and a computer program product.

[0006] In a first aspect, the present application provides a method for determining the shape of an operator of an AI model, which can be executed by a corresponding determination device. Specifically, the determination device determines the degree of influence of each parallel strategy on the shape of a first operator included in the AI ​​model among the multiple parallel strategies provided by the computing system running the AI ​​model; illustratively, the multiple parallel strategies provided by the computing system can be, for example, data parallelism, tensor parallelism, pipeline parallelism, expert parallelism, and context parallelism. In addition, the determination device also obtains the target parallel configuration of the AI ​​model, which includes the configuration value of each parallel strategy in the multiple parallel strategies. Thus, the determination device can determine the shape of the first operator under the target parallel configuration based on the configuration value of each parallel strategy in the multiple parallel strategies, the basic shape of the first operator, and the degree of influence of each parallel strategy on the shape of the first operator. Similarly, for the shapes of the remaining operators that the computing node needs to execute during the operation of the AI ​​model, the determination device can also refer to the above-mentioned similar method to determine.

[0007] Since the determination device can determine the shape of the operator under the target parallel configuration based on the degree of influence of each parallel strategy on the shape of the operator in the AI ​​model, without relying on the specific structure of the AI ​​model, this can effectively improve the generalization and versatility of determining the shape of the operator. Moreover, without the user having to provide the specific structure of the AI ​​model, the determination device automatically determines the shape of the operator based on the degree of influence of each parallel strategy on the shape of the operator in the AI ​​model, which can effectively improve the efficiency of determining the shape of the operator, thereby helping to improve the overall efficiency of training the AI ​​model. At the same time, for any target parallel configuration, the determination device can quickly determine the shape of the operator that needs to be executed when the computing system adopts the target parallel configuration to run the AI ​​model based on the degree of influence of each parallel strategy on the shape of the operator in the AI ​​model, thereby improving the efficiency of determining the shape of the operator under multiple parallel configurations.

[0008] In one possible implementation, when determining the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model among the multiple parallel strategies provided by the computing system running the AI ​​model, the determination device may specifically obtain the operator shape corresponding to each parallel configuration when the first operator of the AI ​​model is executed under multiple parallel configurations, and under each parallel configuration, the configuration value of each parallel strategy in the multiple parallel strategies is partially different or completely different; then, the determination device may determine the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under the multiple parallel configurations. In this way, by actually running the AI ​​model and collecting the operator shape under each parallel configuration, the accuracy of the degree of influence of each parallel strategy on the shape of the first operator of the determination device can be guaranteed, thereby helping to improve the accuracy of the subsequent determination of the shape of the first operator under the target parallel configuration.

[0009] In one possible embodiment, the first operator belongs to an operator sequence executed by a computing node in the computing system when the computing system runs an AI model. The first operator is located at the first position in the operator sequence, and the shape of the first operator includes a first component parameter and a second component parameter. Then, when the determination device determines the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, it can be specifically determined according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter are determined, and the values ​​of the model parameters in the first sub-model are used to indicate the degree of influence of the parallel strategy on the first component parameter included in the shape of the first operator. In addition, the determination device will also determine the values ​​of the model parameters corresponding to the multiple parallel strategies in the second sub-model corresponding to the second component parameter according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, and the values ​​of the model parameters in the second sub-model are used to indicate the degree of influence of the parallel strategy on the second component parameter included in the shape of the first operator. In this way, the determination device can fit the mapping relationship between the values ​​of each component parameter in the shape of the operator and the parallel configuration through a mathematical model, so as to determine the shape of the first operator under other types of parallel configurations without relying on the specific structure of the AI ​​model, thereby improving the generalization and versatility of determining the shape of the operator.

[0010] In one possible embodiment, the determination device can also determine the basic shape parameters in the first sub-model and the basic shape parameters in the second sub-model based on the operator shapes corresponding to each parallel configuration under multiple parallel configurations. The basic shape parameters in the first sub-model are used to indicate the values ​​of the first component parameters when the computing system does not adopt a parallel strategy to run the AI ​​model, and the basic shape parameters in the second sub-model are used to indicate the values ​​of the second component parameters when the computing system does not adopt a parallel strategy to run the AI ​​model. In this way, based on the actual operation of the AI ​​model and the operator shapes under each parallel configuration, the basic shape parameters in the sub-model corresponding to each component parameter in the shape of the first operator are determined, thereby improving the accuracy of the mapping relationship between the values ​​of each component parameter in the shape of the operator fitted by the mathematical model and the parallel configuration, thereby improving the accuracy of the subsequent determination of the shape of the first operator under the target parallel configuration.

[0011] In one possible embodiment, the multiple parallel strategies include a first parallel strategy, and the degree of influence of the first parallel strategy on the shape of the first operator included in the AI ​​model includes: when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model; or, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is the value of the basic shape parameter in the first sub-model; or, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model. Similarly, for the second parallel strategy among the multiple parallel strategies, its degree of influence on the shape of the first operator includes multiplying, multiplying, or remaining unchanged based on the value of the basic shape parameter.

[0012] In one possible embodiment, the multiple parallel configurations include a first partial parallel configuration and a second partial parallel configuration. When the computing system runs the AI ​​model using the first partial parallel configuration, the first number of component parameters of the shape of the first operator executed by the computing node differs from the second number of component parameters of the shape of the first operator executed by the computing node when the computing system runs the AI ​​model using the second partial parallel configuration. Thus, when determining the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under the multiple parallel configurations, the determination device may specifically compare the first number of the first partial parallel configurations with the second number of the second partial parallel configurations. When the first number of the first partial parallel configurations is greater than the second number of the second partial parallel configurations, the determination device determines the degree of influence of each parallel strategy on each of the first number of component parameters included in the shape of the first operator based on the operator shape corresponding to each parallel configuration under the first partial parallel configuration. In this way, if the number of component parameters in the shape of the first operator is not uniform, the parameter values ​​in the first sub-model are determined based on the parallel configuration with the largest number. This can minimize the possibility of being unable to determine the parameter values ​​in each sub-model of the first operator, thereby determining the parameter values ​​in the first sub-model corresponding to the first operator.

[0013] In one possible embodiment, the first operator belongs to a difference operator sequence, which is an operator sequence other than a basic operator sequence executed by a computing node when a computing system adopts a parallel strategy to run an AI model, and a basic operator sequence is an operator sequence that is executed by a computing node when the computing system runs an AI model based on each parallel strategy; then, when the determination device determines the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, it can specifically determine a target parallel strategy that does not affect the value of the first component parameter according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, and determine that the value of the model parameter corresponding to the target parallel strategy in the first sub-model is 0, and determine the values ​​of the remaining parameters in the first sub-model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations. In this way, by setting the values ​​of the model parameters corresponding to the parallel strategies that do not affect the shape of the operators to 0, the number of model parameters with unknown values ​​in the first sub-model can be reduced, thereby determining the parameter values ​​in the first sub-model.

[0014] In one possible implementation, when determining the degree of influence of each parallel strategy on the shape of a first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations, the determination device may specifically obtain a priori conditions for the degree of influence of each parallel strategy on the shape of the first operator, and determine the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the priori conditions and the operator shape corresponding to each parallel configuration under the multiple parallel configurations. In this way, by determining the degree of influence of each parallel strategy on the shape of the first operator in combination with the prior conditions, the accuracy of determining the degree of influence of the parallel strategy on the shape of the first operator can be improved.

[0015] In second aspect, the present application provides a device for determining the operator shape of an AI model, the device comprising: an influence degree determination module for determining the degree of influence of each parallel strategy provided by a computing system running the AI ​​model on the shape of a first operator included in the AI ​​model; an acquisition module for acquiring a target parallel configuration of the AI ​​model, the target parallel configuration including the configuration value of each parallel strategy in the multiple parallel strategies; a shape determination module for determining the shape of the first operator under the target parallel configuration based on the configuration value of each parallel strategy in the multiple parallel strategies, the basic shape of the first operator and the degree of influence of each parallel strategy on the shape of the first operator.

[0016] In one possible implementation, when determining the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model among the multiple parallel strategies provided by the computing system running the AI ​​model, the influence degree determination module is specifically used to: obtain the operator shape corresponding to each parallel configuration when the first operator of the AI ​​model is executed under multiple parallel configurations, and under each parallel configuration, the configuration value of each parallel strategy in the multiple parallel strategies is partially different or completely different; determine the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under the multiple parallel configurations.

[0017] In one possible embodiment, the first operator belongs to an operator sequence executed by a computing node in the computing system when the computing system runs the AI ​​model. The first operator is located at the first position in the operator sequence, and the shape of the first operator includes a first component parameter and a second component parameter. The influence degree determination module is specifically used to determine the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations: according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, determine the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter, and the values ​​of the model parameters in the first sub-model are used to indicate the influence degree of the parallel strategy on the first component parameter included in the shape of the first operator; according to the operator shape corresponding to each parallel configuration under multiple parallel configurations, determine the values ​​of the model parameters corresponding to the multiple parallel strategies in the second sub-model corresponding to the second component parameter, and the values ​​of the model parameters in the second sub-model are used to indicate the influence degree of the parallel strategy on the second component parameter included in the shape of the first operator.

[0018] In one possible embodiment, the impact degree determination module is also used to determine the basic shape parameters in the first sub-model and the basic shape parameters in the second sub-model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations. The basic shape parameters in the first sub-model are used to indicate the value of the first component parameter when the computing system does not adopt a parallel strategy to run the AI ​​model, and the basic shape parameters in the second sub-model are used to indicate the value of the second component parameter when the computing system does not adopt a parallel strategy to run the AI ​​model.

[0019] In one possible embodiment, the multiple parallel strategies include a first parallel strategy, and the degree of influence of the first parallel strategy on the shape of the first operator included in the AI ​​model includes: when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model; or, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is the value of the basic shape parameter in the first sub-model; or, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model.

[0020] In one possible embodiment, multiple parallel configurations include a first-part parallel configuration and a second-part parallel configuration. When the computing system adopts the first-part parallel configuration to run the AI ​​model, the first number of constituent parameters included in the shape of the first operator executed by the computing node is different from the second number of constituent parameters included in the shape of the first operator executed by the computing node when the computing system adopts the second-part parallel configuration to run the AI ​​model. The influence degree determination module determines the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations. Specifically, it is used to: compare the first number of the first-part parallel configurations with the second number of the second-part parallel configurations; when the first number of the first-part parallel configurations is greater than the second number of the second-part parallel configurations, determine the influence degree of each parallel strategy on each constituent parameter of the first number of constituent parameters included in the shape of the first operator according to the operator shape corresponding to each parallel configuration under the first-part parallel configuration.

[0021] In one possible implementation, the first operator belongs to a difference operator sequence, which is an operator sequence other than a basic operator sequence executed by a computing node when a computing system adopts a parallel strategy to run an AI model. The basic operator sequence is an operator sequence that will be executed by the computing node when the computing system runs an AI model based on each parallel strategy. The impact degree determination module is specifically used to determine the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter according to the operator shape corresponding to each parallel configuration under multiple parallel configurations: determine the target parallel strategy that does not affect the numerical value of the first component parameter according to the operator shape corresponding to each parallel configuration under multiple parallel configurations; determine that the value of the model parameter corresponding to the target parallel strategy in the first sub-model is 0, and determine the values ​​of the remaining parameters in the first sub-model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations.

[0022] In one possible implementation, when determining the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations, the influence degree determination module is specifically used to: obtain a priori conditions for the degree of influence of each parallel strategy on the shape of the first operator; and determine the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the prior conditions and the operator shape corresponding to each parallel configuration under multiple parallel configurations.

[0023] The device for determining the operator shape of the AI ​​model provided in the second aspect corresponds to the method for determining the operator shape of the AI ​​model provided in the first aspect. Therefore, the technical effects of any implementation method in the second aspect can be referred to the technical effects of the corresponding implementation method in the above-mentioned first aspect, and will not be repeated here.

[0024] In a third aspect, the present application provides a computing device comprising a processor and a memory. The processor and the memory communicate with each other. The processor is used to execute instructions stored in the memory so that the computing device executes the method for determining the operator shape of the AI ​​model in the first aspect or any one of the implementations of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computing device, the computing device executes the operating steps of the method for determining the operator shape of the AI ​​model described in the first aspect or any implementation of the first aspect.

[0026] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when run on a computing device, enables the computing device to execute the operational steps of the method for determining the operator shape of the AI ​​model described in the first aspect or any one of the implementations of the first aspect.

[0027] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the structure of an exemplary computing system provided in this application;

[0029] Figure 2 A flowchart of a method for determining the operator shape of an AI model provided in this application;

[0030] Figure 3 A schematic diagram of the structure of a device for determining the operator shape of an AI model provided in this application;

[0031] Figure 4 A schematic diagram of the hardware structure of a computing device provided in this application. DETAILED DESCRIPTION

[0032] The terms "first," "second," and so on, in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate and are merely used to describe the manner in which objects with the same attributes are distinguished in the embodiments of this application.

[0033] The technical solution in this application will be described below in conjunction with the drawings provided in this application.

[0034] See also Figure 1 , is a schematic diagram of the structure of an exemplary computing system 10. Figure 1 As shown, the computing system 10 includes a scheduler 100 and a plurality of computing nodes. Figure 1 The computing system 10 including computing nodes 201 to 204 is used as an example for description. Different computing nodes may be connected via a bus, for example, a peripheral component interconnect express (PCIe) bus, a compute express link (CXL) bus, a unified bus (UB or Ubus) or other types of buses. Alternatively, different computing nodes may be connected via a switching node, such as Figure 1 As shown, computing nodes 201 to 204 may be connected via switching node 210 and switching node 211 .

[0035] Scheduler 100 is used for task scheduling, such as scheduling AI models or other AI tasks to one or more computing nodes in computing system 10 for execution. Scheduler 100 can be implemented in software or hardware. When implemented in software, scheduler 100 can be program code running on hardware, such as a process running on a processor or controller. When implemented in hardware, scheduler 100 can be implemented by a processor or a computing device including a processor.

[0036] Computing node 201 (other computing nodes are similar) refers to a node with data computing capability, which can be implemented by a processor, or can be implemented by a computing device including a processor, such as a server. Exemplarily, a processor can be, for example, a central processing unit (CPU), an accelerator, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system on chip (SoC), a software-defined infrastructure (SDI) chip, an AI processing chip, a data processing unit (DPU), or any other processor or any combination thereof. Among them, an accelerator can be a neural-network processing unit (NPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a deep-learning processing unit (DPU), etc.

[0037] Switching node 210 (similar to switching node 211) refers to a node with data exchange capabilities, such as an electrical switching node or an optical switching node. An electrical switching node refers to a switching node that uses electrical signals for data exchange, and these electrical signals can be transmitted via twisted pair or coaxial cables, such as an electrical switch, a network card, or a router. An optical switching node refers to a switching node that uses optical signals for data exchange, such as an optical cross-connect (OXC) device, and these optical signals can be transmitted via optical fiber. In actual applications, a switching node may also be other types of devices, such as a network card or a high-performance network card, and this is not limited to this.

[0038] In actual application, such as Figure 1As shown, computing system 10 may also provide a client 101, and client 101 may interact with a user, such as receiving an AI model input by the user or a configuration file for the AI ​​model. For example, client 101 may be an application running on a user's terminal device, or a web browser provided by computing system 10, etc., without limitation.

[0039] In the process of scheduling multiple computing nodes to perform distributed training on the AI ​​model, the shape of each operator in the operator sequence executed by the computing node when a certain parallel configuration is used to train the AI ​​model can be determined, and the total time consumed by the computing node to execute the operator sequence can be calculated based on the shape of each operator. The parallel configuration that can minimize the total time consumed by the computing node to execute the operator sequence can be determined from multiple parallel configurations, thereby improving the overall training efficiency of the AI ​​model. At this time, if the shape of each operator executed by the computing node when a certain parallel configuration is used to train the AI ​​model is determined by theoretical analysis, it usually takes a long time for the user to provide the specific structure of the AI ​​model (such as manually writing a file of the model structure by the user, etc.), which not only affects the efficiency of determining the operator shape, thereby affecting the overall efficiency of training the AI ​​model, but also, based on the specific AI model structure, it is impossible to automatically analyze the shape of the operator executed by the computing node when training AI models of other structures, resulting in low generalization of the operator shape.

[0040] Based on this, Figure 1 The computing system 10 shown may also include a determination device 300 for determining the shape of the operator of the AI ​​model. The determination device 300 uses the degree of influence of each parallel strategy on the shape of the operator included in the AI ​​model to determine the shape of the operator under the target parallel configuration, thereby improving the accuracy of determining the shape of the operator. The determination device 300 can be deployed on the same physical node device as the scheduler 100, or it can be deployed independently.

[0041] In specific implementation, taking the determination of the shape of one of the operators included in the AI ​​model as an example, the operator is referred to as the first operator below, and the determination device 300 determines the degree of influence of each parallel strategy on the shape of the first operator among the multiple parallel strategies provided by the computing system 10 that runs the AI ​​model. For example, the parallel strategies provided by the computing system 10 may be data parallelism, tensor parallelism, and pipeline parallelism. In addition, the determination device 300 also obtains the target parallel configuration of the AI ​​model, which includes the configuration value of each parallel strategy in the multiple parallel strategies. In this way, the determination device 300 can determine the shape of the first operator under the target parallel configuration based on the configuration value of each parallel strategy in the multiple parallel strategies, the basic shape of the first operator, and the degree of influence of each parallel strategy on the shape of the first operator.

[0042] Since the determination device 300 determines the shape of the operator in the AI ​​model according to the degree of influence of each parallel strategy on the shape of the operator, it can determine the shape of the operator under the target parallel configuration without relying on the specific structure of the AI ​​model. This can effectively improve the generalization and versatility of determining the shape of the operator.

[0043] Moreover, without the user providing the specific structure of the AI ​​model, the determination device 300 automatically determines the shape of the operator based on the degree of influence of each parallel strategy on the shape of the operator in the AI ​​model. This can effectively improve the efficiency of determining the shape of the operator, thereby helping to improve the overall efficiency of training the AI ​​model.

[0044] At the same time, for any target parallel configuration, the determination device 300 can quickly determine the shape of the operator that needs to be executed when the computing system 10 adopts the target parallel configuration to run the AI ​​model based on the degree of influence of each parallel strategy on the shape of the operator in the AI ​​model, thereby improving the efficiency of determining the operator shape under multiple parallel configurations.

[0045] Exemplarily, the determining device 300 may be implemented by software or hardware.

[0046] When the determining device 300 is implemented via software, the determining device 300 may be an application program running on hardware, such as a process or software running on a processor or other hardware. When implemented via hardware, the determining device 300 may be a processor or a computing device including a processor, such as a server; or the determining device 300 may be a device implemented using an ASIC or a PLD.

[0047] It is worth noting that the above Figure 1 The computing system 10 shown is only used as an exemplary illustration and is not intended to be limiting. For example, in other possible implementations, the computing system 10 may include a larger number of computing nodes or switching nodes, and the multiple switching nodes in the computing system 10 may be deployed into multiple levels. Taking deployment into two levels as an example, some switching nodes may be deployed as leaf nodes in the first level to connect multiple switching nodes, and another part of the switching nodes may be deployed as spine nodes in the second level to forward communication data between different leaf nodes. For another example, in other possible implementations, the computing system 10 may also include other types of nodes, such as storage nodes for storing data generated by the training AI model during the training process, or management nodes for managing the various nodes in the computing system 10.

[0048] in addition, Figure 1The example in which a computing system is composed of multiple computing nodes and switching nodes is used for explanation. In other implementations, Figure 1 Multiple computing nodes and switching nodes in the computing system can also constitute a superpod in the computing system. In this case, the computing system 10 can include multiple superpods, and different superpods can communicate with each other through a bus or a network.

[0049] For ease of understanding, an embodiment of the method for determining the operator shape of the AI ​​model provided in this application is described below in conjunction with the accompanying drawings.

[0050] See also Figure 2 , Figure 2 A flow chart of a method for determining the operator shape of an AI model provided in an embodiment of the present application, which can be applied to Figure 1 The computing system 10 described above may be applied to other applicable computing systems. Figure 1 The computing system 10 shown is taken as an example for illustrative purposes.

[0051] in, Figure 2 The method for determining the operator shape of the AI ​​model shown may specifically include the following steps.

[0052] S201: The determining device 300 determines the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model among multiple parallel strategies provided by the computing system 10 running the AI ​​model.

[0053] Exemplarily, the parallel strategy provided by the computing system 10 may be data parallelism (DP), tensor parallelism (TP), pipeline parallelism (PP), expert parallelism (EP), or context parallelism (CP), etc., or may be other types of parallel strategies, which are not limited to this.

[0054] Among them, data parallelism refers to dividing the training data set into several blocks and distributing them to multiple groups of different computing nodes involved in training the AI ​​model. Each group of computing nodes runs the same AI model to process the assigned training data. Simply put, in data parallelism, the training data set is divided into several data blocks, and each data block is distributed to a group of computing nodes. Each computing node will hold a complete model and perform model training based on the allocated data block. In addition, during the model training process, after the back propagation of the model is completed, the model gradients obtained on different computing nodes will be aggregated so that the model parameters on different computing nodes can be kept synchronized.

[0055] Tensor parallelism refers to splitting a tensor calculation across multiple compute nodes involved in training an AI model, and merging the calculation results of multiple compute nodes into a complete tensor. In other words, tensor parallelism actually divides the input data (i.e., tensor) in a single operation into N parts along a specific dimension, calculates only one part on each compute node, and finally obtains the output data by merging the calculation results of each compute node.

[0056] Pipeline parallelism involves assigning different parts of a model (such as different neural network layers) to different compute nodes for execution. Because neural network layers within a model often have data dependencies, different compute nodes often have a specific order of execution. Both pipeline parallelism and tensor parallelism can be considered types of model parallelism, differing only in the dimensions along which they are sliced. Pipeline parallelism can be considered inter-layer parallelism, while tensor parallelism can be considered intra-layer parallelism.

[0057] Expert parallelism is a parallel training technique for Mixture of Experts (MoE) models. Since MoE models typically include multiple expert models, expert parallelism effectively involves distributing these expert models across multiple compute nodes involved in training the AI ​​model.

[0058] Contextual parallelism involves splitting input data and activation values ​​along the sequence dimension and distributing them to different compute nodes involved in training the AI ​​model. This allows each compute node to process only a portion of the sequence, while multiple compute nodes can collaboratively process longer contexts. This not only improves the efficiency of parallel computing but also effectively reduces memory usage within the compute nodes, resolving memory bottlenecks in long-sequence training.

[0059] In actual applications, the AI ​​model may include multiple operators, and when the computing system 10 runs the AI ​​model, the computing nodes in the computing system 10 may execute the multiple operators. Specifically, the computing node may execute an operator sequence, where the operator sequence includes multiple operators executed sequentially by the computing node. For ease of understanding, this embodiment uses the example of determining the shape of one of the multiple operators included in the AI ​​model, hereinafter referred to as the first operator. Therefore, the implementation method for determining the shape of the remaining operators in the AI ​​model is similar to the implementation method for determining the shape of the first operator.

[0060] Typically, for each parallel strategy provided by the computing system 10, when the computing system 10 uses the parallel strategy to run the AI ​​model, the shape of the first operator may or may not be changed. In this embodiment, the determining device 300 can obtain the degree of influence of each parallel strategy provided by the computing system 10 on the shape of the first operator, so as to subsequently determine the shape of the first operator under any parallel configuration based on the degree of influence.

[0061] Among them, the parallel configuration may include one or more parallel strategies. Moreover, the parallel strategies included in different parallel configurations may be different. For example, different parallel configurations may include the same type of parallel strategies, but the configuration values ​​in the included parallel strategies are different. Among them, the configuration value in the parallel strategy may be, for example, the number of channels for parallel computing. For example, the configuration value in data parallelism may be the number of AI model copies running in parallel. When the configuration value is 4, it represents the use of 4 copies of the AI ​​model to process different data subsets at the same time, that is, the number of channels for parallel computing is 4. In actual application, when the configuration value in the parallel strategy is 1 (or other numerical values), it can indicate that the AI ​​model is not run using this parallel strategy. Therefore, in some scenarios, different parallel configurations may also include different numbers or types of parallel strategies. At this time, the determination device 300 can determine that the value of the parallel strategy not included in the parallel configuration is 1.

[0062] In this embodiment, the determining device 300 may determine the degree of influence of each parallel strategy on the shape of the first operator by running an AI model.

[0063] In one possible implementation, the determination device 300 can obtain the operator shape corresponding to each parallel configuration when the first operator of the AI ​​model is executed under multiple parallel configurations. Under each parallel configuration, the configuration value of each parallel strategy in the multiple parallel strategies is partially different or completely different, that is, the configuration values ​​of some or all parallel strategies are different between different parallel configurations.

[0064] For example, the scheduler 100 can schedule the AI ​​model to run on multiple computing nodes in the computing system 10 based on multiple parallel configurations. The multiple computing nodes in the computing system 10 can run the AI ​​model based on one parallel configuration at a time. During the execution of the AI ​​model, the determination device 300 can collect and obtain the operation record of the AI ​​model. The operation record can include the operator shape corresponding to each parallel configuration when the first operator is executed under multiple parallel configurations.

[0065] For example, the client 101 can present an interactive interface to the user, and the user can specify multiple parallel configurations used when the AI ​​model is running in the interactive interface. Each parallel configuration can include all types of parallel strategies that the computing system 10 can provide, but the configuration values ​​of the parallel strategies in different parallel configurations are different. For example, the data parallel configuration value in the user-specified parallel configuration A is 4, and the data parallel configuration value in the user-specified parallel configuration B is 1 (equivalent to the computing system 10 not running the AI ​​model based on data parallelism). Accordingly, the client 101 can generate a configuration file including multiple parallel configurations specified by the user based on the configuration operation performed by the user, and send the configuration file to the determination device 300, such as Figure 2 As shown. Then, the determination device 300 can instruct the scheduler 100 to schedule the AI ​​model to run on multiple computing nodes using each parallel configuration according to the configuration file. When the AI ​​model runs based on each parallel configuration, the determination device 300 can collect the shape information of the operators executed by each computing node to obtain the operation record of the AI ​​model, that is, to obtain the operator shape corresponding to each parallel configuration when each operator (including the first operator) included in the AI ​​model is executed under multiple parallel configurations, such as Figure 2 In actual application scenarios, the operation records collected by the determination device 300 may be profiling data, or other data that can be used for performance analysis.

[0066] In actual application, the complete AI model can be run in the computing system 10, and the determination device 300 can collect the operator shapes corresponding to each parallel configuration when each operator is executed in multiple parallel configurations. Alternatively, a model with the same architecture as the AI ​​model but with a smaller data volume can also be run in the computing system 10. For example, the AI ​​model in S201 can be a teacher model, and the AI ​​model running in the computing system 10 can be a student model; or, the AI ​​model running in the computing system 10 can be an AI model constructed by part of the network layers in the AI ​​model in S201; or, the AI ​​model running in the computing system 10 can be obtained by scaling down the AI ​​model in S201. For example, by trimming the repeated neural network layers in the AI ​​model in S201, an AI model including fewer neural network layers that needs to be run in the computing system 10 is obtained, wherein the number of types of neural network layers included in the AI ​​model running in the computing system 10 is the same as the number of types of neural network layers included in the AI ​​model in S201.

[0067] The shape of the operator may include the shape of the tensor that serves as the input data of the operator. A tensor can be regarded as a one-dimensional or multi-dimensional array, so the shape of the operator may include multiple component parameters, each component parameter being used to describe the size of the data volume of the tensor of the input data of the operator in one direction. For example, the shape of the operator may be [4096, 2048], which represents that the operator includes two component parameters, the value of the first component parameter is 4096, and the value of the second component parameter is 2048, so that the tensor of the input data of the operator is 4096×2048. Accordingly, the shape information of the operator in the operation record may specifically be the values ​​of multiple component parameters in the shape of the operator. Alternatively, the shape of the operator may include the shape of the tensor that serves as the output data of the operator, or may include the shapes of the tensors that serve as the input data and output data of the operator at the same time.

[0068] In this embodiment, an example is taken of actually running the AI ​​model in the computing system 10 to obtain the operator shape corresponding to each parallel configuration when the operators included in the AI ​​model are executed under multiple parallel configurations. In other embodiments, the determination device 300 may also obtain the operator shape corresponding to each parallel configuration when the operators included in the AI ​​model are executed under multiple parallel configurations through other methods, such as by the user sending it to the determination device 300 through the client 101, etc., and this is not limited to this.

[0069] Then, the determination device 300 can determine the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations.

[0070] Exemplarily, the determination device 300 can determine the parameter values ​​in the mathematical model corresponding to the first operator based on the operator shape corresponding to each parallel configuration under multiple parallel configurations. The mathematical model is used to fit the mapping relationship between the shape of the first operator and the parallel configuration adopted when the AI ​​model is running. The mathematical model includes parameters corresponding to multiple parallel strategies, so that the values ​​of the parameters corresponding to each parallel strategy can be used to indicate the degree of influence of the parallel strategy on the shape of the first operator included in the AI ​​model.

[0071] In a specific implementation, the determining device 300 models the mapping relationship between the shape of the first operator and the parallel configuration according to the operator shape corresponding to each parallel configuration under multiple parallel configurations. Since the shape of the first operator may include multiple component parameters, and the values ​​of different component parameters may be different, the determining device 300 can use the component parameters as the granularity to model the mapping relationship between the value of each component parameter of the first operator and the parallel configuration, thereby achieving modeling of the mapping relationship between the shape of the entire first operator and the parallel configuration. Taking the example that the first operator includes two component parameters, namely the first component parameter and the second component parameter, the determining device 300 can model the mapping relationship between the value of the first component parameter and the parallel configuration to obtain a first sub-model, and model the mapping relationship between the value of the second component parameter and the parallel configuration to obtain a second sub-model.

[0072] Generally, when running an AI model based on different parallel configurations, the operator sequences executed by the computing nodes may be different, and the shapes of the same operator at different positions in the operator sequence may also be different. Therefore, the determination device 300 can also perform modeling in combination with the position of the first operator in the operator sequence, that is, the determination device 300 can model the same operator at different positions separately. Among them, the operator sequence can include two parts, namely the basic operator sequence and the difference operator sequence, wherein the basic operator sequence refers to the operator sequence that the computing node will execute when the computing system 10 runs the AI ​​model based on each parallel strategy; the difference operator sequence refers to the operator sequence other than the basic operator sequence executed by the computing node when the computing system 10 adopts a parallel strategy to run the AI ​​model. In actual application scenarios, the basic operator sequence accounts for a large proportion of the entire operator sequence, such as the proportion of the basic operator sequence can reach more than 97%, that is, the proportion of the difference operator sequence in the entire operator sequence may be less than 3%.

[0073] In practical applications, the determination device 300 can model each component parameter of the shape of the operator at each position in the operator sequence. For ease of understanding, the following description uses the modeling of the first component parameter of the shape of the first operator at the first position in the operator sequence as an example. The resulting model is the first sub-model in the aforementioned mathematical model. The first operator is an operator in the operator sequence, and the first component parameter is one of the at least one component parameter included in the shape of the first operator.

[0074] Exemplarily, the first sub-model constructed by the determination device 300 for the first component parameter in the shape of the first operator may be as shown in the following formula (1).

[0075]

[0076] Among them, shape is the value of the first component parameter included in the shape of the first operator; a0 is the basic shape parameter, which is used to indicate the value of the first component parameter included in the shape of the first operator when the computing node does not adopt a parallel configuration to run the AI ​​model, that is, the basic shape of the first component parameter in the first operator; x1, ..., x n There are n different parallel strategies, and each parallel configuration may include any one or more of the n parallel strategies; a1, ..., a n The model parameters corresponding to n different parallel strategies are used to indicate the degree of influence of the computing node using the parallel strategy on the value of the first component parameter of the shape of the first operator when running the AI ​​model. The parameters in the first sub-model include the basic shape parameters and model parameters.

[0077] For example, a1, ..., a n The value of can be a value in {0, 1, -1}. Taking the model parameter a1 as an example (the other model parameters are similar), when the value of a1 is 0, the numerical value of the first component parameter in the shape of the first operator is not affected when the characterization computing system 10 adopts the parallel strategy corresponding to the model parameter to run the AI ​​model. When the value of a1 is 1, the numerical value of the first component parameter in the shape of the first operator will increase when the characterization computing system 10 adopts the parallel strategy corresponding to the model parameter to run the AI ​​model, such as the numerical value of the first component parameter increases proportionally, that is, data fusion occurs in the tensor direction indicated by the first component parameter in the shape of the first operator. For example, when the computing system 10 adopts experts to run the AI ​​model in parallel, the numerical value of the first component parameter in the shape of the first operator may be multiplied due to the use of multiple expert models for calculation. When the value of a1 is -1, it indicates that when the computing node uses the parallel strategy corresponding to the model parameter to run the AI ​​model, the value of the first component parameter in the shape of the first operator will be reduced, such as the amount of data of the first component parameter is proportionally reduced, that is, the tensor direction indicated by the first component parameter in the shape of the first operator will be split. For example, when the computing system 10 uses tensor parallelism to run the AI ​​model, the first component parameter in the shape of the first operator can be a multi-head attention parameter, so that the value of the multi-head attention parameter will be multiplied due to tensor parallelism.

[0078] That is to say, taking the first parallel strategy as an example among the multiple parallel strategies that the computing system 10 can provide, the degree of influence of the first parallel strategy on the shape of the first operator included in the AI ​​model includes three types. The first type: when the computing system 10 adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model. The second type: when the computing system 10 adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is the value of the basic shape parameter in the first sub-model. The third type: when the computing system 10 adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model. Similarly, for the second parallel strategy and the remaining parallel strategies among the multiple parallel strategies, the degree of influence on the shape of the first operator can also include multiplication, subtraction, or unchanged based on the value of the basic shape parameter.

[0079] Then, the determining device 300 can determine the parameter values ​​in the first sub-model obtained by modeling based on the operator shape corresponding to each parallel configuration under the multiple parallel configurations, specifically based on the value of the first component parameter in the shape of the first operator located at the first position under each parallel configuration. Assuming that the determining device 300 can obtain k sets of data, that is, obtain k sets of values ​​of the first component parameter in the shape of the first operator located at the first position under k parallel configurations, where k is an integer greater than 1, the determining device 300 can obtain multiple relationship expressions shown in the following formula (2).

[0080]

[0081] Among them, x 11 ,…,x 1n is the configuration value of n different parallel strategies included in the first parallel configuration among k parallel configurations. When the first parallel configuration does not include some parallel strategies, the configuration value corresponding to the partial parallel strategy is 1 (or other values). For example, assuming that the parallel configuration does not include x 11 The parallel strategy belongs to, then x 11 The value of shape1 is 1. shape1 is the value of the first component parameter in the shape of the first operator when the computing node adopts the first parallel configuration to run the AI ​​model. k1 ,…,x kn The configuration value of n different parallel strategies included in the k-th parallel configuration among k parallel configurations. k is the value of the first component parameter in the shape of the first operator when the computing node adopts the kth parallel configuration to run the AI ​​model. The determining device 300 can determine shape1, ..., shape in formula (2) based on the operation record.k and x 11 ,…,x kn The value of; accordingly, a0, a1…, a in formula (2) n It is an unknown number.

[0082] Then, the determining device 300 can solve formula (2) to obtain a0, a1, ..., a in the first sub-model. n The value of .

[0083] As an implementation example of solving parameter values, the determination device 300 can take logarithms on both sides of the equation in formula (2) to obtain multiple relationship expressions shown in the following formula (3).

[0084]

[0085] Formula (3) can be solved as a linear equation system Ax = b to obtain:

[0086]

[0087] When the linear equation system is well-posed (i.e., the number of equations in the linear equation system is equal to the number of unknowns) or over-determined (i.e., the number of equations in the linear equation system is greater than the number of unknowns and the column is full rank), the determination device 300 can directly solve the linear equation system Ax=b, and the calculated solution is x=(A 2 A) 31 A 2 b, that is, obtaining the values ​​of each model parameter in the first sub-model.

[0088] In most cases, the determining device 300 can collect the values ​​of the first component parameters of the shape of the first operator under k parallel configurations, so that the matrix A 6 A is usually reversible. However, when there are some special cases, the linear equation system may be underdetermined (that is, the number of equations in the linear equation system is less than the number of unknowns). In this case, the square matrix A 6 A is irreversible. In this case, if the determination device 300 directly solves the linear equation system Ax=b, it may not be possible to obtain the values ​​of the various model parameters in the first sub-model. In this embodiment, the following special cases that cause the linear equation system to be underdetermined are provided, as well as several implementation examples for solving the linear equation system, which are respectively described below.

[0089] Special case 1: The number of constituent parameters of the same operator at the same position in the operator sequence may change under different parallel configurations. For example, under parallel configuration A, the shape of the first operator may be [2048, 2048], which includes 2 constituent parameters; however, under parallel configuration B, the shape of the first operator may be [(1024, 2), (1024, 2)], which includes 4 constituent parameters. For example, for the ConcatD operator, when the tensor parallel configuration value in parallel configuration A is 8, the number of constituent parameters in the shape of the ConcatD operator may be 16; and when the tensor parallel configuration value in parallel configuration B is 4, the number of constituent parameters in the shape of the ConcatD operator may be 8. In this case, if the shape of the operator is determined by default according to the maximum number of constituent parameters included in the shape of the first operator, the solution of the linear equation system may be affected because the amount of data in the parallel configuration that enables the operator to have the maximum number of constituent parameters is small, that is, the solution of the parameter values ​​in the first sub-model corresponding to the first operator may be affected.

[0090] Based on this, in an implementation example, the determination device 300 can compare the number of the first part parallel configurations with the number of the second part parallel configurations. Among them, the number P of constituent parameters in the shape of the first operator executed by the computing node when the computing system 10 adopts the first part parallel configuration to run the AI ​​model is different from the number Q of constituent parameters in the shape of the first operator executed by the computing node when the computing system 10 adopts the second part parallel configuration to run the AI ​​model, and both P and Q are positive integers. When the number P of the first part parallel configurations is greater than the number Q of the second part parallel configurations, the number of constituent parameters in the shape of the first operator executed by the computing node under the majority parallel configuration is P. Therefore, the determination device 300 can determine the parameter values ​​in each sub-model corresponding to the first operator based on the operator shape corresponding to each parallel configuration under the first part parallel configuration, specifically based on the shape of the first operator corresponding to each parallel configuration under the first part parallel configuration. For example, for each of the P component parameters included in the shape of the first operator, the determination device 300 can construct a linear equation system for each component parameter in the shape of the first operator according to the operator shape corresponding to each parallel configuration under the first part of the parallel configuration, and solve the linear equation system to obtain the values ​​of each parameter in the first sub-model. Conversely, when the number P of the first part of the parallel configuration is less than the number Q of the second part of the parallel configuration, the number of component parameters in the shape of the first operator executed by the computing node under the majority of the parallel configurations is Q. Therefore, the determination device 300 can determine the parameter values ​​in each sub-model corresponding to the first operator according to the operator shape corresponding to each parallel configuration under the second part of the parallel configuration, specifically according to the shape of the first operator corresponding to each parallel configuration under the second part of the parallel configuration.

[0091] It will be appreciated that the above implementation example uses two different component parameters in the shape of the first operator as an example for illustration. In actual applications, under different parallel configurations, the number of component parameters in the shape of the first operator may be three or more (including three). Therefore, the determination device 300 can count the number of parallel configurations that result in the shape of the first operator generating various numbers of component parameters, and determine the parallel configuration with the largest number of components, thereby determining the values ​​of each parameter in the first sub-model based on the shape of the first operator corresponding to each parallel configuration under the parallel configuration adopted by the computing node.

[0092] In this way, when the number of constituent parameters in the shape of the first operator is not uniform, the parameter values ​​in the first sub-model are determined according to the parallel configuration with the largest number, which can avoid the underdetermination of the linear equation group constructed for the first operator as much as possible, thereby determining the parameter values ​​in the first sub-model corresponding to the first operator.

[0093] Special case 2: Some types of operators y will only appear in the process of the computing system 10 using a partial parallel strategy C to run the AI ​​model, and will not appear in the process of the computing system using another partial parallel strategy D to run the AI ​​model. For example, for the grouped matrix multiplication (grouped matmul) operator, it will usually be executed by the computing node when the computing system 10 uses expert parallelism to run the AI ​​model, but will not be executed by the computing node when the computing system 10 uses other parallel strategies to run the AI ​​model. When the first operator is the operator y of this part of the type in the operator sequence, in the process of determining the parameter values ​​in the first sub-model based on the above-mentioned method of constructing a linear equation system, the solution of the linear equation system may be affected due to the small number of parallel configurations including the parallel strategy C, that is, the solution of the parameter values ​​in the first sub-model used to determine the shape of the first operator. In actual application scenarios, the operator y of this part of the type is usually located in the difference operator sequence.

[0094] Based on this, in an implementation example, the determination device 300 can determine a target parallel strategy that does not affect the numerical value of the first component parameter included in the shape of the first operator based on the operator shape corresponding to each parallel configuration under multiple parallel configurations, specifically based on the shape of the first operator corresponding to each parallel configuration under multiple parallel configurations. Then, the determination device 300 can determine that the value of the model parameter corresponding to the target parallel strategy in the first sub-model is 0, and based on the operator shape corresponding to each parallel configuration under multiple parallel configurations, specifically based on the shape of the first operator corresponding to each parallel configuration under multiple parallel configurations, determine the values ​​of the remaining parameters in the first sub-model except for the model parameter, such as by continuing to solve the above-mentioned linear equation group.

[0095] For example, the determining device 300 may construct a linear equation system based on the first component parameters in the shape of the first operator according to the above formulas (1) to (3), assuming that the matrix Since all the values ​​in the second column of matrix A are the same, this indicates that a parallel strategy corresponding to the second column exists in all different parallel configurations, and thus is not helpful in determining the value a2 corresponding to the second column. In this case, it can be considered that the parallel strategy has no effect on the value of the first component parameter in the shape of the first operator, and thus the determination device 300 can determine that the value a2 is 0. Similarly, since all the values ​​in the third column of matrix A are the same, the determination device 300 can determine that the value a3 is 0. The first column of matrix A is used to calculate the basic shape parameter. Since the value of the basic shape parameter is the same in each parallel configuration, the value of the first column of matrix A must remain consistent.

[0096] So, in determining When the parameter a1 and parameter a2 in are 0, the parameter in x is Thus, the determination device 300 solves the linear equations again to obtain the values ​​of the remaining parameters in x. For example, when the values ​​of the parameters a1 and a2 are determined to be 0, the determination device 300 can be based on Solve The value of the parameter in .

[0097] In this way, when the linear system of equations is underdetermined, by setting the model parameter values ​​corresponding to the parallel strategy that does not affect the shape of the operator to 0 (specifically, setting the model parameter values ​​corresponding to the parallel strategy that does not affect the numerical values ​​of the component parameters in the shape of the operator to 0), the number of unknowns in the linear system of equations can be reduced, making the linear system of equations well-posed or overdetermined, thereby determining the parameter values ​​in the first sub-model.

[0098] Special Case 3: After processing the linear equation system using the two aforementioned methods, the processed linear equation system may still be underdetermined (i.e., matrix A still does not satisfy column full rank). For example, the operator shapes corresponding to each of the multiple parallel configurations obtained by the determination device 300 may contain some data missing (e.g., excluding the shape information of the operators executed by the computing nodes when the computing system 10 uses a partial parallel configuration to run the AI ​​model).

[0099] At this time, in an implementation example, the determination device 300 can also combine the priori conditions to solve the approximate solutions of various parameters in the first sub-model.

[0100] Specifically, the determination device 300 can obtain a priori conditions for the degree of influence of each parallel strategy on the shape of the first operator. When determining the value of the first component parameter in the shape of the first operator, the priori conditions obtained by the determination device 300 can specifically be a priori conditions for the parameter values ​​in the first sub-model. The priori conditions can be, for example, the value ranges of the various parameters in the first sub-model. The prior conditions can be pre-set by a technician, configured by a user through the client 101, or independently determined by the determination device 300, etc., and there is no limitation on this. Therefore, the determination device 300 can determine the values ​​of the various parameters in the first sub-model based on the prior conditions and the operation records.

[0101] For example, assuming that the prior conditions are the parameters a1∈{0, 1}, a2∈{-1, 0}, and a3∈{-1, 0} in the first sub-model, then, for the linear equations shown in formula (3), the generalized inverse solution x=A can be obtained. @ b=(A 6 A) @ A 6 b, get x 6 =[3,0.333,0.333,0.333], that is, log(a0)=3 is obtained by generalized inverse solution, and the values ​​of a1, a2, and a3 are all 0.333. Since the priori condition corresponding to parameter a2 is a2∈{-1,0}, and the priori condition corresponding to parameter a3 is a3∈{-1,0}, and the values ​​of a2 and a3 do not fall within the value range indicated by the priori condition (that is, the interval [0,1]), but are closest to the value 0 within the value range, the determination device 300 can determine that the values ​​of parameters a2 and a3 are both 0 based on the priori condition. The value of parameter a1 is 0.333, which falls within the value range indicated by the priori condition corresponding to parameter a1. Then, the determination device 300 can re-solve the above linear equations after determining that the values ​​of a2 and a3 are both 0, and obtain the value of parameter a1 as 1. Then, the determination device 300 finally solves and obtains x 6 =[3,1,0,0], and obtain the values ​​of each parameter in the first sub-model.

[0102] In actual application, the determination device 300 can obtain a priori conditions for the parameter values ​​in the mathematical model corresponding to each operator. Based on the a priori conditions and the operator shapes corresponding to each parallel configuration under multiple parallel configurations, the parameter values ​​in the mathematical model corresponding to each operator can be determined. In other words, the parameter values ​​in the sub-models corresponding to the component parameters included in the shape of each operator can be determined. In other words, the degree of influence of each parallel strategy on the shape of each operator (including the first operator) can be determined. Optionally, when the system of linear equations is overdetermined or well-posed, the determination device 300 can also determine the parameter values ​​in the first sub-model corresponding to the first operator in combination with the a priori conditions.

[0103] Among them, the range of values ​​indicated by the prior condition can be determined by the probability of the parameter value. For example, when the probability of the parameter a1 taking the value of 1 is 0.6, the probability of taking the value of 0 is 0.3, and the probability of taking the value of -1 is 0.1, the determination device 300 can determine the prior condition of the parameter a1 as a1∈{0, 1} based on the probability of each value, and the value range of the parameter a1 indicated by the prior condition is [0, 1]. Exemplarily, the probability of each value of the parameter a1 can be determined by counting the number of values ​​of the parameter a1 in the historical records, and the historical records refer to the value records of each parameter in the first sub-model collected when the AI ​​model or other AI models were run in the past period of time. At this time, the probability of the parameter a1 taking the value of 1 can be the ratio between the number of times the parameter a1 takes the value of 1 and the total number of various values ​​of the parameter a1 in the historical records.

[0104] Furthermore, if it is determined that based on the current prior conditions (i.e., the value ranges of the various parameters), it is still impossible to solve the values ​​of the various parameters in the first sub-model, the determination device 300 can also continue to try to solve the values ​​of the various parameters in the first sub-model by adjusting some of the prior conditions (i.e., adjusting the value ranges of some parameters).

[0105] For example, assuming that the probability of parameter a1 taking the value of 1 is 0.6, the probability of taking the value of 0 is 0.3, and the probability of taking the value of -1 is 0.1, the probability of parameter a2 taking the value of 1 is 0.2, the probability of taking the value of 0 is 0.3, and the probability of taking the value of -1 is 0.5, and the probability of parameter a3 taking the value of 1 is 0.3, the probability of taking the value of 0 is 0.3, and the probability of taking the value of -1 is 0.4. Then, in the first solution process, the determination device 300 can determine the prior conditions as parameters a1∈{0,1}, a2∈{-1,0}, and a3∈{-1,0} based on the value probabilities of the parameters. Assuming that the determination device 300 has not yet solved the values ​​of the various parameters in the first sub-model based on the current prior conditions, the determination device 300 can also compare the remaining value probabilities of the various parameters, where the remaining probability of the value of parameter a1 being -1 is 0.1, the remaining probability of the value of parameter a2 being 1 is 0.2, and the remaining probability of the value of parameter a3 being 1 is 0.3. Then, the determination device 300 can determine that among the remaining value probabilities of the various parameters, the remaining probability of the value of parameter a3 being 1 is the largest, and adjust the value range of parameter a3 from [0, -1] to [0, 1], that is, adjust the prior conditions. At this time, the adjusted prior conditions are parameters a1∈{0, 1}, a2∈{-1, 0}, and a3∈{0, 1}. Then, the determination device 300 can refer to the above method and continue to try to solve the values ​​of the various parameters in the first sub-model based on the adjusted prior conditions. Similarly, through one or more solving processes, the values ​​of each parameter in the first sub-model can be finally calculated.

[0106] In actual application scenarios, after determining that the linear system of equations is underdetermined, the determination device 300 can first determine whether the cause of the underdetermination of the linear system of equations is the above-mentioned special case 1 or special case 2. If so, the determination device 300 can solve the linear system of equations using the processing method for the above-mentioned special case 1 or special case 2. If not, or if the linear system of equations is still underdetermined when solving the linear system of equations based on the processing method for the above-mentioned special case 1 or special case 2, the determination device 300 can solve the linear system of equations using the implementation method for the above-mentioned special case 3, thereby improving the robustness of determining the parameter values ​​in the first sub-model.

[0107] Similarly, for the second component parameter in the shape of the first operator at the first position, the determination device 300 may also refer to the above-described similar method and determine the values ​​of the model parameters corresponding to the multiple parallel strategies in the second sub-model corresponding to the second component parameter based on the operator shapes corresponding to each parallel configuration under the multiple parallel configurations. Specifically, based on the shapes of the first operator corresponding to each parallel configuration under the multiple parallel configurations, the values ​​of the model parameters in the second sub-model are used to indicate the degree of influence of the parallel strategy on the second component parameter included in the shape of the first operator. In this way, the determination device 300 can determine the degree of influence of each parallel strategy on the shape of the first operator.

[0108] In practical applications, the determination device 300 can refer to the above method to construct a sub-model for each component parameter of the shape of an operator at any position in the operator sequence, and determine the model parameters in each sub-model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations. After determining the values ​​of each parameter in the mathematical model corresponding to each operator, the mapping relationship between the operator shape and the parallel configuration can be fitted.

[0109] It should be noted that the above-mentioned implementation method of obtaining the degree of influence of each parallel strategy on the shape of the first operator is only an exemplary description and is not intended to be limiting. In other embodiments, the determination device 300 may also use other methods to obtain the degree of influence of each parallel strategy on the shape of the first operator.

[0110] S202: The determination device 300 obtains a target parallel configuration of the AI ​​model, where the target parallel configuration includes a configuration value of each parallel strategy among a plurality of parallel strategies.

[0111] S203: The determining device 300 determines the shape of the first operator under the target parallel configuration according to the configuration value of each parallel strategy among the multiple parallel strategies included in the target parallel configuration, the basic shape of the first operator and the degree of influence of each parallel strategy on the shape of the first operator.

[0112] The target parallel configuration may be different from the multiple parallel configurations used by the computing system 10 when running the AI ​​model. In actual application scenarios, some parallel configurations may be selected to run the AI ​​model so as to collect the operation records and use them to construct the mathematical model corresponding to each operator. For example, Figure 2 As shown, the target parallel configuration may be provided by the user through the client 101, or may be selected by the determination device 300 from a search space corresponding to the parallel configuration as the target parallel configuration, etc., which is not limited.

[0113] Then, the determining device 300 can determine the shape of the first operator under the target parallel configuration based on the configuration value of each parallel strategy among the multiple parallel strategies included in the target parallel configuration, the basic shape of the first operator, and the degree of influence of each parallel strategy on the shape of the first operator. For example, the determining device 300 can use the mathematical model corresponding to the first operator to calculate the shape of the first operator executed by the computing node when the computing system 10 adopts the target parallel configuration to run the AI ​​model according to the target parallel configuration. In this way, the determining device 300 can use the mathematical models corresponding to each operator to calculate the shape of each operator executed by the computing node when the computing system 10 adopts the target parallel configuration to run the AI ​​model.

[0114] Furthermore, the determining device 300 can calculate the computational time taken by the computing node to execute the operator based on the shape of the operator executed by the computing node when the determined computing system 10 adopts the target parallel configuration to run the AI ​​model. Thus, the determining device 300 can calculate the total time taken by the operator sequence executed by each computing node participating in the training of the AI ​​model based on the computational time taken by each operator, thereby determining the total time taken by the computing system 10 for a single round of training of the AI ​​model. For example, the determining device 300 can obtain the basic operator sequence and the difference operator sequence corresponding to each parallel strategy, and determine the complete operator sequence that the computing node needs to execute when the AI ​​model is trained using this parallel configuration based on the parallel strategy included in each parallel configuration, thereby calculating the total time taken by the computing node to execute the complete operator sequence based on the computational time taken by each operator.

[0115] When the total time consumption of the AI ​​model meets the configuration conditions, the determination device 300 can set the parallel configuration as the parallel configuration ultimately used for training the AI ​​model.

[0116] In actual application scenarios, since the computing system 10 trains the AI ​​model based on different parallel configurations, the training efficiency of the AI ​​model will be different. Therefore, before training the AI ​​model, the determination device 300 can use the mathematical models corresponding to each operator to determine the shape of each operator executed by the computing node when the computing system 10 uses multiple parallel configurations to run the AI ​​model respectively, so that the total time consumed by the computing system 10 to train the AI ​​model based on each parallel configuration (the time consumed for one round of model training) can be further calculated based on the determined operator shape. In this way, the determination device 300 can determine the parallel configuration with the smallest total time consumption from multiple parallel configurations based on the total time consumption of training the AI ​​model corresponding to each parallel configuration, and use the parallel configuration as the parallel configuration adopted for the final training of the AI ​​model. For example, when the total time consumption of the training AI model corresponding to the target parallel configuration is the smallest, the determination device 300 can use the target parallel configuration as the parallel configuration adopted for the final training of the AI ​​model.

[0117] It is worth noting that other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0118] Combination of the above Figure 1 as well as Figure 2 The method for determining the operator shape of the AI ​​model provided in the embodiment of the present application is introduced. Next, the structure of the determination device and computing equipment for determining the operator shape of the AI ​​model provided in the embodiment of the present application is introduced in conjunction with the accompanying drawings.

[0119] See also Figure 3 , shows a schematic diagram of the structure of a device for determining the operator shape of an AI model. Figure 3 As shown, the determining means 310 includes:

[0120] An impact degree determination module 301 is configured to determine the impact degree of each of multiple parallel strategies provided by a computing system running the AI ​​model on the shape of a first operator included in the AI ​​model;

[0121] An acquisition module 302 is configured to acquire a target parallel configuration of the AI ​​model, where the target parallel configuration includes a configuration value for each of the multiple parallel strategies.

[0122] The shape determination module 303 is configured to determine the shape of the first operator under the target parallel configuration according to the configuration value of each parallel strategy among the multiple parallel strategies, the basic shape of the first operator, and the degree of influence of each parallel strategy on the shape of the first operator.

[0123] In one possible implementation, when determining the degree of influence of each of the multiple parallel strategies provided by the computing system running the AI ​​model on the shape of the first operator included in the AI ​​model, the impact degree determination module 301 is specifically configured to:

[0124] Obtain the operator shape corresponding to each parallel configuration when the first operator of the AI ​​model is executed under multiple parallel configurations, where the configuration values ​​of each parallel strategy in the multiple parallel strategies are partially or completely different under each parallel configuration;

[0125] According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model is determined.

[0126] In one possible implementation, the first operator belongs to an operator sequence executed by a computing node in the computing system when the computing system runs the AI ​​model. The first operator is located at the first position in the operator sequence. The shape of the first operator includes a first component parameter and a second component parameter.

[0127] The influence degree determination module 301 is specifically configured to determine the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations:

[0128] Determining, based on the operator shapes corresponding to each of the multiple parallel configurations, values ​​of model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter, the values ​​of the model parameters in the first sub-model being used to indicate the degree of influence of the parallel strategy on the first component parameter included in the shape of the first operator;

[0129] According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the values ​​of the model parameters corresponding to the multiple parallel strategies in the second sub-model corresponding to the second component parameters are determined. The values ​​of the model parameters in the second sub-model are used to indicate the degree of influence of the parallel strategy on the second component parameters included in the shape of the first operator.

[0130] In one possible embodiment, the impact degree determination module 301 is also used to determine the basic shape parameters in the first sub-model and the basic shape parameters in the second sub-model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations. The basic shape parameters in the first sub-model are used to indicate the value of the first component parameter when the computing system does not adopt a parallel strategy to run the AI ​​model, and the basic shape parameters in the second sub-model are used to indicate the value of the second component parameter when the computing system does not adopt a parallel strategy to run the AI ​​model.

[0131] In one possible embodiment, the multiple parallel strategies include a first parallel strategy, and the degree of influence of the first parallel strategy on the shape of the first operator included in the AI ​​model includes: when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model; or, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is the value of the basic shape parameter in the first sub-model; or, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model.

[0132] In one possible implementation, the multiple parallel configurations include a first partial parallel configuration and a second partial parallel configuration, and when the computing system uses the first partial parallel configuration to run the AI ​​model, a shape of a first operator executed by the computing node includes a first number of constituent parameters that differs from a shape of the first operator executed by the computing node when the computing system uses the second partial parallel configuration to run the AI ​​model.

[0133] The influence degree determination module 301 is specifically configured to determine the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations:

[0134] comparing a first number of the first portion of parallel configurations with a second number of the second portion of parallel configurations;

[0135] When the first number of parallel configurations of the first part is greater than the second number of parallel configurations of the second part, the degree of influence of each parallel strategy on each component parameter of the first number of component parameters included in the shape of the first operator is determined according to the operator shape corresponding to each parallel configuration under the first part of the parallel configuration.

[0136] In one possible implementation, the first operator belongs to a difference operator sequence, where the difference operator sequence is an operator sequence other than a basic operator sequence executed by a computing node when the computing system adopts a parallel strategy to run the AI ​​model. The basic operator sequence is an operator sequence that the computing node will execute when the computing system runs the AI ​​model based on each parallel strategy.

[0137] The influence degree determination module 301 is specifically configured to determine the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter according to the operator shape corresponding to each parallel configuration under the multiple parallel configurations:

[0138] Determining a target parallel strategy that does not affect the value of the first component parameter according to an operator shape corresponding to each parallel configuration under multiple parallel configurations;

[0139] The value of the model parameter corresponding to the target parallel strategy in the first sub-model is determined to be 0, and the values ​​of the remaining parameters in the first sub-model are determined according to the operator shape corresponding to each parallel configuration under multiple parallel configurations.

[0140] In one possible implementation, the impact degree determination module 301 is specifically configured to determine the impact degree of each parallel strategy on the shape of the first operator included in the AI ​​model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations:

[0141] Obtaining a priori conditions on the degree of influence of each parallel strategy on the shape of the first operator;

[0142] According to the prior conditions and the operator shapes corresponding to each parallel configuration under multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model is determined.

[0143] because Figure 3 The determining means 310 shown corresponds to the above Figure 2 The determining device 300 in the embodiment shown, therefore Figure 3 For the specific implementation of the determination device 310 and its technical effects, see the above Figure 2 The description of the relevant parts in the illustrated embodiment will not be repeated here.

[0144] Figure 4 This is a hardware structure diagram of a computing device 400 provided in this application. The computing device 400 can, for example, implement the above Figure 2 The determining device 300 in the illustrated embodiment, etc.

[0145] like Figure 4 As shown, the computing device 400 includes a processor 401, a memory 402, and a communication interface 403. The processor 401, the memory 402, and the communication interface 403 communicate via a bus 404, and may also communicate via other means such as wireless transmission. The memory 402 is used to store instructions, and the processor 401 is used to execute the instructions stored in the memory 402. Furthermore, the computing device 400 may also include a memory unit 405, which may be connected to the processor 401, the storage medium 402, and the communication interface 403 via a bus 404. The memory 402 stores program code, and the processor 401 may use the program code stored in the memory 402 to perform the following operations:

[0146] Determining the extent to which each of a plurality of parallel strategies provided by a computing system running the AI ​​model affects the shape of a first operator included in the AI ​​model;

[0147] Obtaining a target parallel configuration of the AI ​​model, where the target parallel configuration includes a configuration value for each of the multiple parallel strategies;

[0148] The shape of the first operator under the target parallel configuration is determined according to a configuration value of each parallel strategy among the multiple parallel strategies, a basic shape of the first operator, and an influence degree of each parallel strategy on the shape of the first operator.

[0149] It should be understood that in this embodiment, the processor 401 may be a CPU, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete device components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0150] The memory 402 may include a read-only memory and a random access memory, and provides instructions and data to the processor 401. The memory 402 may also include a nonvolatile random access memory.

[0151] The memory 402 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0152] The communication interface 403 is used to communicate with other devices connected to the computing device 400. In addition to the data bus, the bus 404 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 404 in the figure.

[0153] It should be understood that the computing device 400 according to the embodiment of the present application may correspond to the above Figure 2 The determining device 300 in the embodiment shown may correspond to executing the above Figure 2The method performed by the determining device 300 and the above-mentioned and other operations and / or functions implemented by the computing device 400 are respectively to achieve Figure 2 For the sake of brevity, the process of the corresponding method in will not be repeated here.

[0154] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the method for determining the operator shape of the above-mentioned AI model.

[0155] The present application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the computer program product fully or partially generates the process or function described in the present application.

[0156] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0157] The computer program product may be a software installation package, which may be downloaded and executed on a computing device when any of the methods for determining the operator shape of the aforementioned AI model is required.

[0158] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0159] The terms used in the above embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; the character " / " generally indicates that the objects before and after are in an "or" relationship. In the embodiments of the present application. "Simultaneously" refers to the same time period, including the situation at the same moment.

[0160] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0161] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for determining the shape of an operator of an artificial intelligence (AI) model, characterized in that: The method comprises: Determining the extent to which each of a plurality of parallel strategies provided by a computing system running the AI ​​model affects the shape of a first operator included in the AI ​​model; Obtaining a target parallel configuration of the AI ​​model, where the target parallel configuration includes a configuration value for each of the multiple parallel strategies; The shape of the first operator under the target parallel configuration is determined according to a configuration value of each parallel strategy among the multiple parallel strategies, a basic shape of the first operator, and an influence degree of each parallel strategy on the shape of the first operator.

2. The method according to claim 1, characterized in that In determining the multiple parallel strategies provided by the computing system running the AI ​​model, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model includes: Obtaining an operator shape corresponding to each parallel configuration when a first operator of the AI ​​model is executed under multiple parallel configurations, wherein under each parallel configuration, a configuration value of each parallel strategy in the multiple parallel strategies is partially or completely different; According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model is determined.

3. The method according to claim 2, characterized in that The first operator belongs to an operator sequence executed by a computing node in the computing system when the computing system runs the AI ​​model, the first operator is located at the first position in the operator sequence, and the shape of the first operator includes a first component parameter and a second component parameter; Determining, based on the operator shapes corresponding to each parallel configuration under the multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model includes: Determining, based on the operator shapes corresponding to each of the multiple parallel configurations under the multiple parallel configurations, values ​​of model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter, wherein the values ​​of the model parameters in the first sub-model are used to indicate the degree of influence of the parallel strategy on the first component parameter included in the shape of the first operator; According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the values ​​of the model parameters corresponding to the multiple parallel strategies in the second sub-model corresponding to the second component parameter are determined, and the values ​​of the model parameters in the second sub-model are used to indicate the degree of influence of the parallel strategy on the second component parameters included in the shape of the first operator.

4. The method according to claim 3, characterized in that The method further comprises: According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the basic shape parameters in the first sub-model and the basic shape parameters in the second sub-model are determined. The basic shape parameters in the first sub-model are used to indicate the values ​​of the first component parameters when the computing system does not adopt a parallel strategy to run the AI ​​model, and the basic shape parameters in the second sub-model are used to indicate the values ​​of the second component parameters when the computing system does not adopt a parallel strategy to run the AI ​​model.

5. The method according to claim 4, characterized in that The multiple parallel strategies include a first parallel strategy, and the degree of influence of the first parallel strategy on the shape of the first operator included in the AI ​​model includes: When the computing system runs the AI ​​model using the first parallel strategy, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model; Alternatively, when the computing system runs the AI ​​model using the first parallel strategy, the value of the first component parameter is the value of the basic shape parameter in the first sub-model; Alternatively, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model.

6. The method according to any one of claims 2 to 5, characterized in that The multiple parallel configurations include a first partial parallel configuration and a second partial parallel configuration, and when the computing system uses the first partial parallel configuration to run the AI ​​model, a shape of the first operator executed by the computing node includes a first number of constituent parameters, which is different from a shape of the first operator executed by the computing node when the computing system uses the second partial parallel configuration to run the AI ​​model. Determining, based on the operator shapes corresponding to each parallel configuration under the multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model includes: comparing a first number of the first portion of parallel configurations with a second number of the second portion of parallel configurations; When the first number of parallel configurations of the first part is greater than the second number of parallel configurations of the second part, the degree of influence of each parallel strategy on each component parameter of the first number of component parameters included in the shape of the first operator is determined according to the operator shape corresponding to each parallel configuration under the first part of the parallel configuration.

7. The method according to claim 4 or 5, characterized in that The first operator belongs to a difference operator sequence, where the difference operator sequence is an operator sequence executed by the computing node when the computing system adopts a parallel strategy to run the AI ​​model, excluding a basic operator sequence. The basic operator sequence is an operator sequence that the computing node will execute when the computing system runs the AI ​​model based on each parallel strategy; Determining, based on the operator shape corresponding to each parallel configuration under the multiple parallel configurations, the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter includes: determining, according to an operator shape corresponding to each of the plurality of parallel configurations, a target parallel strategy that does not affect a value of the first component parameter; The value of the model parameter corresponding to the target parallel strategy in the first sub-model is determined to be 0, and the values ​​of the remaining parameters in the first sub-model are determined according to the operator shape corresponding to each parallel configuration under multiple parallel configurations.

8. The method according to any one of claims 2 to 7, characterized in that Determining, based on the operator shapes corresponding to each parallel configuration under the multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model includes: Obtaining a priori conditions for the degree of influence of each parallel strategy on the shape of the first operator; According to the prior conditions and the operator shapes corresponding to each parallel configuration under the multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model is determined.

9. A device for determining the shape of an operator of an artificial intelligence (AI) model, characterized in that: The device comprises: an impact degree determination module, configured to determine the impact degree of each parallel strategy, among multiple parallel strategies provided by a computing system running the AI ​​model, on the shape of a first operator included in the AI ​​model; An acquisition module, configured to acquire a target parallel configuration of the AI ​​model, wherein the target parallel configuration includes a configuration value of each of the multiple parallel strategies; A shape determination module is used to determine the shape of the first operator under the target parallel configuration based on the configuration value of each parallel strategy in the multiple parallel strategies, the basic shape of the first operator and the degree of influence of each parallel strategy on the shape of the first operator.

10. The device according to claim 9, characterized in that The impact degree determination module is specifically configured to, when determining the impact degree of each parallel strategy on the shape of the first operator included in the AI ​​model among multiple parallel strategies provided by the computing system running the AI ​​model,: Obtaining an operator shape corresponding to each parallel configuration when a first operator of the AI ​​model is executed under multiple parallel configurations, wherein under each parallel configuration, a configuration value of each parallel strategy in the multiple parallel strategies is partially or completely different; According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model is determined.

11. The device according to claim 10, characterized in that The first operator belongs to an operator sequence executed by a computing node in the computing system when the computing system runs the AI ​​model, the first operator is located at the first position in the operator sequence, and the shape of the first operator includes a first component parameter and a second component parameter; The influence degree determination module is specifically configured to determine the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations: Determining, based on the operator shapes corresponding to each of the multiple parallel configurations under the multiple parallel configurations, values ​​of model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter, wherein the values ​​of the model parameters in the first sub-model are used to indicate the degree of influence of the parallel strategy on the first component parameter included in the shape of the first operator; According to the operator shape corresponding to each parallel configuration under multiple parallel configurations, the values ​​of the model parameters corresponding to the multiple parallel strategies in the second sub-model corresponding to the second component parameter are determined, and the values ​​of the model parameters in the second sub-model are used to indicate the degree of influence of the parallel strategy on the second component parameters included in the shape of the first operator.

12. The device according to claim 11, characterized in that The impact degree determination module is also used to determine the basic shape parameters in the first sub-model and the basic shape parameters in the second sub-model based on the operator shape corresponding to each parallel configuration under multiple parallel configurations. The basic shape parameters in the first sub-model are used to indicate the numerical value of the first component parameter when the computing system does not adopt a parallel strategy to run the AI ​​model, and the basic shape parameters in the second sub-model are used to indicate the numerical value of the second component parameter when the computing system does not adopt a parallel strategy to run the AI ​​model.

13. The device according to claim 12, characterized in that The multiple parallel strategies include a first parallel strategy, and the degree of influence of the first parallel strategy on the shape of the first operator included in the AI ​​model includes: When the computing system runs the AI ​​model using the first parallel strategy, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model; Alternatively, when the computing system runs the AI ​​model using the first parallel strategy, the value of the first component parameter is the value of the basic shape parameter in the first sub-model; Alternatively, when the computing system adopts the first parallel strategy to run the AI ​​model, the value of the first component parameter is multiplied based on the value of the basic shape parameter in the first sub-model.

14. The device according to any one of claims 10 to 13, characterized in that The multiple parallel configurations include a first partial parallel configuration and a second partial parallel configuration, and when the computing system uses the first partial parallel configuration to run the AI ​​model, a shape of the first operator executed by the computing node includes a first number of constituent parameters, which is different from a shape of the first operator executed by the computing node when the computing system uses the second partial parallel configuration to run the AI ​​model. The influence degree determination module is specifically configured to determine the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations: comparing a first number of the first portion of parallel configurations with a second number of the second portion of parallel configurations; When the first number of parallel configurations of the first part is greater than the second number of parallel configurations of the second part, the degree of influence of each parallel strategy on each component parameter of the first number of component parameters included in the shape of the first operator is determined according to the operator shape corresponding to each parallel configuration under the first part of the parallel configuration.

15. The device according to claim 12 or 13, characterized in that The first operator belongs to a difference operator sequence, where the difference operator sequence is an operator sequence executed by the computing node when the computing system adopts a parallel strategy to run the AI ​​model, excluding a basic operator sequence. The basic operator sequence is an operator sequence that the computing node will execute when the computing system runs the AI ​​model based on each parallel strategy; The influence degree determination module is specifically configured to determine, based on the operator shape corresponding to each parallel configuration under the multiple parallel configurations, the values ​​of the model parameters corresponding to the multiple parallel strategies in the first sub-model corresponding to the first component parameter: determining, according to an operator shape corresponding to each of the plurality of parallel configurations, a target parallel strategy that does not affect a value of the first component parameter; The value of the model parameter corresponding to the target parallel strategy in the first sub-model is determined to be 0, and the values ​​of the remaining parameters in the first sub-model are determined according to the operator shape corresponding to each parallel configuration under multiple parallel configurations.

16. The device according to any one of claims 11 to 15, characterized in that The influence degree determination module is specifically configured to determine the influence degree of each parallel strategy on the shape of the first operator included in the AI ​​model according to the operator shape corresponding to each parallel configuration under multiple parallel configurations: Obtaining a priori conditions for the degree of influence of each parallel strategy on the shape of the first operator; According to the prior conditions and the operator shapes corresponding to each parallel configuration under the multiple parallel configurations, the degree of influence of each parallel strategy on the shape of the first operator included in the AI ​​model is determined.

17. A computer-readable storage medium, characterized in that The method comprises instructions which, when executed on a computing device, cause the computing device to perform the steps of the method according to any one of claims 1 to 7.

18. A computer program product comprising instructions, characterized in that When the method is executed on a computing device, the computing device is caused to execute the method according to any one of claims 1 to 7.

Images