Hardware deployment method, related devices and media for learning models

Abstract

The present disclosure provides a hardware deployment method, related devices and media for a learning model. The hardware deployment method includes: obtaining the initial pipeline parallel number and the initial tensor parallel number for the processing unit deployment of the target learning model; locating the initial deployment plan node where the initial pipeline parallel number and the initial tensor parallel number are located on the deployment plan node jump relationship diagram; inputting the deployment plan node jump relationship diagram and the initial deployment plan node into the reinforcement learning model to obtain a target deployment plan node that optimizes the reasoning performance of the target learning model, the target deployment plan node including the target pipeline parallel number and the target tensor parallel number; based on the target pipeline parallel number and the target tensor parallel number, deploying the processing unit for the target learning model. The embodiments of the present disclosure can expand the scope of application of the deployment method and improve deployment efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of model deployment, and in particular to a hardware deployment method, related devices and media for a learning model. Background Art

[0002] In the field of deep learning, the size of the learning model can be as high as tens of billions or even hundreds of billions of parameters, which makes the computational workload and memory requirements of the learning model continue to increase. As the computational workload and memory requirements continue to increase, a method for parallel deployment of models is proposed. This method uses the parallel processing capabilities of the hardware architecture to distribute the model's calculations to multiple computing cores or devices, thereby achieving substantial acceleration in the inference process.

[0003] In order to further improve the performance of learning models and the resource utilization of hardware architecture, automatic parallelization of model deployment has also received extensive attention. However, the automatic parallelization methods in the existing technology are mainly designed for graphics processing unit (GPU) accelerators, which means that the methods can only support algorithm models deployed on GPU accelerators, while the deployment and operation of other models on artificial intelligence (AI) accelerators often require manual adjustment of parallel strategies, which has low universality and low deployment efficiency. Summary of the invention

[0004] The embodiments of the present disclosure provide a hardware deployment method, related devices and media for a learning model, which can improve the universality of the deployment method and improve the deployment efficiency.

[0005] According to one aspect of the present disclosure, a hardware deployment method for a learning model is provided, comprising: obtaining an initial pipeline parallelism number and an initial tensor parallelism number for processing unit deployment of a target learning model; locating an initial deployment scheme node where the initial pipeline parallelism number and the initial tensor parallelism number are located on a deployment scheme node jump relationship graph, wherein each deployment scheme node in the deployment scheme node jump relationship graph includes a pipeline parallelism number and a tensor parallelism number; inputting the deployment scheme node jump relationship graph and the initial deployment scheme node into a reinforcement learning model to obtain a target deployment scheme node that optimizes the reasoning performance of the target learning model, wherein the target deployment scheme node includes a target pipeline parallelism number and a target tensor parallelism number, wherein the reinforcement learning model specifies a label deployment scheme node with the best reasoning performance by jumping a relationship graph for each sample scheme node in a plurality of sample scheme node jump relationship graphs, and starts from a sample initial deployment scheme node in the sample scheme node jump relationship graph, jumping according to the principle of maximizing the reward value until jumping to the label deployment scheme node for training; and deploying a processing unit for the target learning model based on the target pipeline parallelism number and the target tensor parallelism number.

[0006] According to one aspect of the present disclosure, a hardware deployment device for a learning model is provided, comprising: a first acquisition unit, used to acquire an initial pipeline parallel number and an initial tensor parallel number for processing unit deployment of a target learning model; a positioning unit, used to locate an initial deployment scheme node where the initial pipeline parallel number and the initial tensor parallel number are located on a deployment scheme node jump relationship graph, wherein each deployment scheme node of the deployment scheme node jump relationship graph includes a pipeline parallel number and a tensor parallel number; an input unit, used to input the deployment scheme node jump relationship graph and the initial deployment scheme node into a reinforcement learning model to obtain a target learning model. The target deployment solution node with the best reasoning performance is optimized, and the target deployment solution node includes a target pipeline parallel number and a target tensor parallel number, wherein the reinforcement learning model specifies a label deployment solution node with the best reasoning performance by jumping a relationship graph for each sample solution node in a plurality of sample solution node jump relationship graphs, and starts from the sample initial deployment solution node of the sample solution node jump relationship graph, and jumps according to the principle of maximizing the reward value until jumping to the label deployment solution node for training; a first deployment unit is used to deploy a processing unit for the target learning model based on the target pipeline parallel number and the target tensor parallel number.

[0007] According to one aspect of the present disclosure, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the hardware deployment method of the learning model as described above when executing the computer program.

[0008] According to one aspect of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the hardware deployment method of the learning model as described above is implemented.

[0009] According to one aspect of the present disclosure, a computer program product is provided, which includes a computer program, and the computer program is read and executed by a processor of a computer device, so that the computer device executes the hardware deployment method of the learning model as described above.

[0010] The disclosed embodiment obtains the initial pipeline parallelism and the initial tensor parallelism, which are the initial parallelism values ​​for the processing unit deployment of the target learning model. In addition, each deployment scheme node in the deployment scheme node jump relationship diagram includes the corresponding pipeline parallelism and tensor parallelism, so the initial deployment scheme node where the initial pipeline parallelism and the initial tensor parallelism are located can be located on the deployment scheme node jump relationship diagram. The reinforcement learning model is trained based on multiple sample scheme node jump relationship diagrams, sample initial deployment scheme nodes, and label deployment scheme nodes. Specifically, the reinforcement learning model starts from the sample initial deployment scheme node of the sample scheme node jump relationship diagram, and jumps according to the principle of maximizing the reward value until it jumps to the label deployment scheme node, and the label deployment scheme node is the node with the most optimized reasoning performance in the sample scheme node jump relationship diagram. Then, the deployment plan node jump relationship diagram and the initial deployment plan node are used as the input of the trained reinforcement learning model. The reinforcement learning model can jump from the initial deployment plan node to the target deployment plan node with the best reasoning performance in the deployment plan node jump relationship diagram, and further determine the target pipeline parallelism and target tensor parallelism in the target deployment plan node. Since the target deployment plan node corresponding to the target pipeline parallelism and target tensor parallelism is the node with the best reasoning performance, the processing unit is deployed for the target learning model based on the target pipeline parallelism and target tensor parallelism, so that the reasoning performance of the deployed target learning model is optimized to meet the requirements of low latency and high throughput. In addition, the hardware deployment process of the target learning model only inputs the deployment plan node jump relationship diagram and the initial deployment plan node into the reinforcement learning model to obtain the deployment plan with the best reasoning performance. It is applicable to a variety of AI accelerators on the basis of realizing automatic parallelization, which improves the universality of the deployment method and the deployment efficiency.

[0011] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or understood by practicing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to provide further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation on the technical solution of the present disclosure.

[0013] Figure 1 is an architecture diagram of a system to which a method for hardware deployment of a learning model according to an embodiment of the present disclosure is applied;

[0014] Figure 2A and Figure 2BIt is a comparative schematic diagram of the hardware deployment method of the learning model of the related technology and the embodiment of the present disclosure in which the neural network algorithm is deployed on the neural network processor;

[0015] Figure 3 An overall flow chart of a hardware deployment method for a learning model according to an embodiment of the present disclosure;

[0016] FIG. 4A to FIG. 4C is a schematic diagram of pipeline parallelism and tensor parallelism according to an embodiment of the present disclosure;

[0017] Figure 5A It is a schematic diagram of a deployment solution node jump relationship diagram and an initial deployment solution node according to an embodiment of the present disclosure;

[0018] Figure 5B is a schematic diagram of an initial deployment solution node and a target deployment solution node according to an embodiment of the present disclosure;

[0019] Figure 6 yes Figure 3 Step 310 is a flow chart of obtaining the initial pipeline parallel number and the initial tensor parallel number;

[0020] Figure 7 yes Figure 6 A flow chart of determining the storage capacity occupied by a single layer of processing in step 610;

[0021] Figure 8 yes Figure 6 A flow chart of determining the initial pipeline parallelism number and the initial tensor parallelism number in step 640;

[0022] Fig. 9 yes Figure 3 In step 330, the deployment scheme node jump relationship diagram and the initial deployment scheme node are input into a flow chart of the reinforcement learning model;

[0023] Fig.10 yes Fig. 9 A schematic diagram of determining candidate deployment solution nodes in ;

[0024] Fig.11 yes Fig. 9 A schematic diagram of inputting the deployment plan node jump relationship diagram and the initial deployment plan node into the reinforcement learning model;

[0025] Fig.12 yes Fig. 9 A flow chart of selecting a candidate deployment solution node in step 910;

[0026] Fig.13 yes Fig. 9 A flow chart of selecting a jump strategy in step 920;

[0027] Fig.14 is a flow chart for determining a minimum number of processing units according to an embodiment of the present disclosure;

[0028] Fig.15 It is a flow chart of generating a deployment solution node jump relationship diagram according to an embodiment of the present disclosure;

[0029] Fig.16 is a flow chart of a training process of a reinforcement learning model provided according to an embodiment of the present disclosure;

[0030] Fig.17 is a schematic diagram of a node jump relationship diagram of a sample solution provided according to an embodiment of the present disclosure;

[0031] Fig.18 is an overall flow chart of the training process of the reinforcement learning model provided according to an embodiment of the present disclosure;

[0032] Fig.19 yes Fig.16 A flow chart of determining a tag deployment solution node in step 1610;

[0033] Fig. 20 yes Fig.19 A flowchart of determining the inference duration of the sample deployment solution node in step 1920;

[0034] Fig.21 yes Fig. 20 A flow chart of determining the sample model calculation duration in step 2030;

[0035] Fig. 22 yes Fig. 20 A flow chart of determining the sample model access time in step 2040;

[0036] Fig.23 yes Fig. 20 A flow chart of determining the communication duration of the sample model in step 2050;

[0037] Fig.24 yes Fig.16 A flow chart of selecting a sample jump strategy in step 1630;

[0038] Fig.25 yes Fig.24 A flow chart of calculating the reward value in step 2410;

[0039] Fig.26 yes Fig.16 Another flow chart of determining a tag deployment solution node in step 1610;

[0040] Fig. 27is a block diagram of a hardware deployment apparatus for a learning model according to an embodiment of the present disclosure;

[0041] Fig.28 is a terminal structure diagram of a hardware deployment method for executing a learning model according to an embodiment of the present disclosure;

[0042] Fig.29 It is a server structure diagram of a hardware deployment method for executing a learning model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0044] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations:

[0045] Deep Learning (DL): is a type of Machine Learning (ML), which is the only way to achieve artificial intelligence. Deep learning is to learn the inherent laws and representation levels of sample data. The information obtained in the learning process is very helpful for interpreting data such as text, images, and sounds. Its concept originated from the study of artificial neural networks. Multilayer perceptrons with multiple hidden layers are a type of deep learning structure. Deep learning combines low-level features to form more abstract high-level representation attribute categories or features to discover distributed feature representations of data.

[0046] Learning model: deep learning model, usually composed of multiple layers of neural networks, each layer contains multiple neurons, these neurons are connected by weights to transmit and process information. Deep learning models have achieved great success in image recognition, speech recognition, natural language processing and other fields, and have become one of the hot technologies in the field of artificial intelligence today.

[0047] Model parallelism: It is a common parallel computing strategy for training large deep learning models. It aims to accelerate the training process and handle large amounts of data and complex models through distributed computing. In model parallelism, a large learning model is split into multiple parts, each running on different hardware. Each hardware is responsible for calculating a part of the model and passing its results to other hardware, which are finally merged to obtain the overall model output.

[0048] Graphics Processing Unit (GPU) accelerator: A hardware device specifically used to process graphics and image-related calculations, with highly parallel computing capabilities. It is suitable for processing large-scale graphics data and complex computing tasks, such as scientific computing, machine learning, and artificial intelligence.

[0049] Artificial Intelligence (AI) accelerator: also known as AI chip. AI chip is a chip specially designed for accelerating AI algorithms, that is, a functional module specially used to handle a large number of computing tasks in AI applications. AI accelerators include Accelerated Processing Units (APU), Central Processing Unit (CPU), GPU, Data Processing Unit (DPU), Neural Processing Unit (NPU), Tensor Processing Unit (TPU), etc. Different AI accelerators are suitable for different algorithms.

[0050] In the field of deep learning, the size of the learning model can be as high as tens of billions or even hundreds of billions of parameters, which makes the computational workload and memory requirements of the learning model continue to increase. As the computational workload and memory requirements continue to increase, a method for parallel deployment of models is proposed. This method uses the parallel processing capabilities of the hardware architecture to distribute the model's calculations to multiple computing cores or devices, thereby achieving substantial acceleration in the inference process.

[0051] In order to further improve the performance of learning models and the resource utilization of hardware architecture, automatic parallelization of model deployment has also received extensive attention. However, the automatic parallelization methods in related technologies are mainly designed for graphics processing unit (GPU) accelerators, which means that the methods can only support algorithm models deployed on GPU accelerators, while the deployment and operation of other models on artificial intelligence (AI) accelerators often require manual adjustment of parallel strategies, which has a narrow scope of application and low deployment efficiency.

[0052] Based on this, the embodiments of the present disclosure provide a hardware deployment method, related devices and media for a learning model, which can expand the universality of the deployment method and improve the deployment efficiency.

[0053] System architecture of the application of the disclosed embodiments

[0054] Figure 11 is a system architecture diagram used in the hardware deployment method of the learning model according to the embodiment of the present disclosure, which includes an object terminal 110, the Internet 120, a gateway 130, and a server 140.

[0055] The object terminal 110 is a device for displaying multiple learning models for determining the target learning model. In addition, the object terminal 110 can also display the target pipeline parallelism and target tensor parallelism for the processing unit deployment of the target learning model for the object to determine the deployment parameters. The object terminal 110 includes various forms such as desktop computers, laptops, PDAs (personal digital assistants), mobile phones, car terminals, home theater terminals, dedicated terminals, digital tablets, etc. In addition, it can be a single device or a collection of multiple devices. For example, multiple devices are connected through a local area network and share a display device to work together to form a terminal. The terminal can also communicate with the Internet 120 in a wired or wireless manner to exchange data.

[0056] The gateway 130 is also called an internetwork connector or a protocol converter. The gateway 130 realizes network interconnection at the transport layer and is a computer system or device that acts as a converter. The gateway 130 is a translator between two systems that use different communication protocols, data formats or languages, or even completely different architectures. At the same time, the gateway 130 can also provide filtering and security functions. The message sent by the object terminal 110 to the server 140 is sent to the corresponding server 140 through the gateway 130. The message sent by the server 140 to the object terminal 110 is also sent to the corresponding object terminal 110 through the gateway 130.

[0057] The server 140 refers to a computer system that can provide hardware deployment services of the learning model to the object terminal 110. Compared with the object terminal 110, the server 140 has higher requirements in terms of stability, security, performance, etc. The server 140 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a part of a high-performance computer (such as a virtual machine), a combination of parts of multiple high-performance computers (such as virtual machines), etc. The server 140 can also communicate with the Internet 120 in a wired or wireless manner to exchange data.

[0058] The embodiments of the present disclosure can be applied in various scenarios, such as text analysis, image recognition, etc. Correspondingly, the learning model can be a text analysis model, an image recognition model, a large language model, etc.

[0059] Figure 2AThis is a schematic diagram of the deployment of a neural network algorithm learning model on an NPU accelerator in the related art. When the related art deploys the learning model on an NPU accelerator, the parallel strategy needs to be manually adjusted. In the related art, the number of NPU accelerators used for the learning model is 5, the deployment time of the learning model takes 36 seconds, and the delay time of the model during the inference process is 48 milliseconds.

[0060] Figure 2B It is a schematic diagram of the deployment of the neural network algorithm learning model in the NPU accelerator in the embodiment of the present disclosure. The learning model is processed by the hardware part method of the learning model provided by the embodiment of the present disclosure, and the number of NPU accelerators used for the learning model is 4. The whole process takes 2 seconds, and the delay of the model during the reasoning process is 12 milliseconds. It can be clearly seen that the embodiment of the present disclosure is suitable for the NPU accelerator, the time required for deployment is reduced, and the reasoning delay of the model is reduced, that is, the reasoning performance of the model is improved, so the embodiment of the present disclosure can expand the scope of application of the deployment method and improve the deployment efficiency.

[0061] It should be understood that the above content only illustrates some application scenarios of the present disclosure. The business scenarios to which the present disclosure can be applied may include but are not limited to the specific embodiments listed above.

[0062] General description of the disclosed embodiments

[0063] It should be emphasized that the embodiments of the present disclosure can be applied to a variety of scenarios, such as text analysis, image recognition, etc. Correspondingly, the learning model can be a text analysis model, an image recognition model, a large language model, etc. The relevant technology can only support algorithm models deployed on GPU accelerators, while the deployment and operation of other models on artificial intelligence (AI) accelerators often require manual adjustment of parallel strategies, which have a narrow scope of application and low deployment efficiency. Some embodiments of the present disclosure provide a hardware deployment method, related devices and media for a learning model, which can expand the scope of application of the deployment method and improve deployment efficiency.

[0064] The hardware deployment method of the learning model is a method of processing the target learning model so as to deploy a processing unit for the target learning model. The method can expand the scope of application of the deployment method and improve the deployment efficiency.

[0065] The hardware deployment method of the learning model of the embodiment of the present disclosure may be executed on the object terminal 110 , may be executed on the server 140 , or may be executed partially on the object terminal 110 and the other partially on the server 140 .

[0066] like Figure 3 As shown, according to one embodiment of the present disclosure, the hardware deployment method of the learning model includes:

[0067] Step 310: Obtain the initial pipeline parallelism and initial tensor parallelism of the processing unit deployment for the target learning model;

[0068] Step 320: Locate the initial deployment solution node where the initial pipeline parallelism number and the initial tensor parallelism number are located on the deployment solution node jump relationship graph, wherein each deployment solution node in the deployment solution node jump relationship graph includes the pipeline parallelism number and the tensor parallelism number;

[0069] Step 330: Input the deployment solution node jump relationship graph and the initial deployment solution node into the reinforcement learning model to obtain a target deployment solution node that optimizes the reasoning performance of the target learning model, wherein the target deployment solution node includes a target pipeline parallel number and a target tensor parallel number, wherein the reinforcement learning model specifies a label deployment solution node with the best reasoning performance by jumping a relationship graph for each sample solution node in a plurality of sample solution node jump relationship graphs, and starts from the sample initial deployment solution node of the sample solution node jump relationship graph, jumping according to the principle of maximizing the reward value until jumping to the label deployment solution node for training;

[0070] Step 340: Deploy a processing unit for the target learning model based on the target pipeline parallelism and the target tensor parallelism.

[0071] Steps 310 to 340 are described in detail below.

[0072] In step 310, the initial pipeline parallelism and initial tensor parallelism of the processing unit deployment for the target learning model are obtained.

[0073] The target learning model is a deep learning model. Since the target learning model is too large to fit into a single hardware memory, multiple AI accelerators need to be deployed for the target learning model. The target learning model can be a text analysis model, an image recognition model, a large language model, and other models. Figure 2B , the target learning model is a neural network learning model.

[0074] A processing unit refers to a hardware device used to deploy a target learning model. A processing unit is specifically an AI accelerator, such as APU, CPU, GPU, DPU, NPU, and TPU.

[0075] The initial pipeline parallelism and initial tensor parallelism are initial parameters for the processing unit deployment of the target learning model. The initial pipeline parallelism is the initial value of the pipeline parallelism. The initial tensor parallelism is the initial value of the tensor parallelism.

[0076] Pipeline parallelism is a value related to pipeline parallelism (PP). Pipeline parallelism is a form of model parallelism, which can be regarded as inter-layer parallelism. For example, Figure 4A The learning model is divided into two parts in a pipeline parallel manner, one part is deployed in processing unit 1, and the other part is deployed in processing unit 2. Processing unit 1 and processing unit 2 are operated in sequence to realize the operation of the learning model.

[0077] Tensor parallelism is a value related to tensor parallelism (TP). Tensor parallelism is a form of model parallelism, which can be regarded as intra-layer parallelism. Figure 4B , tensor parallelism divides the learning model into two parts within the layer, one part is deployed on processing unit 1, and the other part is deployed on processing unit 2. Unlike pipeline parallelism, processing unit 1 and processing unit 2 in tensor parallelism need to run at the same time to realize the operation of the learning model. In addition, tensor parallelism is to split the specific matrix operations within the layer. From the perspective of splitting matrix operations, tensor parallelism is divided into row parallelism and column parallelism. Figure 4C , assuming that the calculation of one layer of the learning model is expressed as X·A=Y, where X is the input vector, Y is the output vector, and A is the weight matrix. Column parallelism only requires column division of weights, and column parallelism can be expressed as XA=X[A1,A2]=[XA1,XA2]=Y. Row parallelism divides the weights into rows, so the input matrix must be divided into columns, and row parallelism can be expressed as

[0078] In step 320, the initial deployment scheme node where the initial pipeline parallelism number and the initial tensor parallelism number are located is located on the deployment scheme node jump relationship graph, wherein each deployment scheme node in the deployment scheme node jump relationship graph includes the pipeline parallelism number and the tensor parallelism number.

[0079] The deployment plan node jump relationship diagram is a diagram including multiple deployment plan nodes. Each deployment plan node corresponds to a deployment plan, which specifically includes pipeline parallelism and tensor parallelism, and the pipeline parallelism and tensor parallelism corresponding to multiple deployment plan nodes are different. In addition, the deployment plan node jump relationship diagram can correspond to any learning model. Regardless of the scenario or algorithm that the target learning model is suitable for, the corresponding deployment plan node jump relationship diagram can be the same diagram, except that the initial deployment plan nodes corresponding to different learning models and the processing in the subsequent deployment process are different. For example, the deployment plan node jump relationship diagram corresponding to the text analysis learning model is Figure 5A As shown in the figure, the deployment solution node jump relationship diagram corresponding to the image recognition learning model can also be Figure 5A Figure shown.

[0080] The number of pipeline parallelisms corresponding to the initial deployment plan node is the initial pipeline parallelism, and the number of tensor parallelisms corresponding to the initial deployment plan node is the initial tensor parallelism. Figure 5A The deployment scheme node jump relationship diagram shows that the initial pipeline parallelism of the initial deployment scheme node is 5, and the initial tensor parallelism is 1.

[0081] In step 330, the deployment solution node jump relationship graph and the initial deployment solution node are input into the reinforcement learning model to obtain a target deployment solution node that optimizes the reasoning performance of the target learning model, wherein the target deployment solution node includes a target pipeline parallel number and a target tensor parallel number, wherein the reinforcement learning model is trained by specifying a label deployment solution node with optimized reasoning performance for each sample solution node jump relationship graph in a plurality of sample solution node jump relationship graphs, and starting from the sample initial deployment solution node of the sample solution node jump relationship graph, jumping according to the principle of maximizing the reward value until jumping to the label deployment solution node.

[0082] The reinforcement learning model is a trained model. The input of the reinforcement learning model is the deployment plan node jump relationship diagram and the initial deployment plan node. The output of the reinforcement learning model is the target pipeline parallelism and the target tensor parallelism. The target deployment plan node refers to the node where a deployment plan for the processing unit deployment of the target learning model is located, and the reasoning performance of the target learning model is optimized under this deployment plan.

[0083] The reinforcement learning model is trained based on multiple sample solution node jump relationship graphs and label deployment solution nodes in each sample solution node jump relationship graph. The label deployment solution node corresponds to the target deployment solution node, which is the node with the best reasoning performance in the sample solution node jump relationship graph. The reinforcement learning model starts from the sample initial deployment solution node in the sample solution node jump relationship graph, and jumps according to the principle of maximizing the reward value until it jumps to the label deployment solution node. It can be said that the training goal of the reinforcement learning model is to find the node with the best reasoning performance in the sample solution node jump relationship graph. Therefore, by inputting the deployment solution node jump relationship graph and the initial deployment solution node into the trained reinforcement learning model, the target deployment solution node with the best reasoning performance can be obtained.

[0084] It should be noted that in the training process of the reinforcement learning model, each sample solution node jump relationship diagram is provided with a sample initial deployment solution node and a label deployment solution node. The sample initial deployment solution node and the label deployment solution node correspond to the sample learning model and the processing unit for sample learning model deployment.

[0085] Figure 5AThe deployment scheme node jump relationship diagram shown in the figure contains the initial deployment scheme node corresponding to the target learning model. The initial pipeline parallelism of the target learning model is 5 and the initial tensor parallelism is 1. Figure 5A The deployment plan node jump relationship diagram and the initial deployment plan node input reinforcement learning model shown in the figure. The reinforcement learning model starts from the initial deployment plan node and jumps to Figure 5B In addition, refer to the target deployment scenario node shown in Figure 5B , the inference time of the initial deployment solution node is 1451, while the inference times of the three nodes to which the initial deployment solution node can jump are 1000, 1300, and 1100. Therefore, the reinforcement learning model finally identifies the node with an inference time of 1000, that is, the node with the best inference performance, as the target deployment solution node.

[0086] In step 340, based on the target pipeline parallelism and the target tensor parallelism, a processing unit is deployed for the target learning model.

[0087] After determining the target pipeline parallelism and target tensor parallelism, the target learning model is divided within the layers based on the target pipeline parallelism, and the target learning model is divided between the layers based on the target tensor number, and the divided parts of the target learning model are deployed to the processing unit for work.

[0088] It should be noted that the number of processing units deployed for the target learning model is the product of the target pipeline parallelism and the target tensor parallelism. Figure 5B , if the target pipeline parallelism is 5 and the target tensor parallelism is 3, then the number of processing units deployed for the target learning model is 15.

[0089] The embodiments of the above steps 310 to 340 obtain the initial pipeline parallelism and the initial tensor parallelism, which are the initial parallelism values ​​for the processing unit deployment of the target learning model. In addition, each deployment scheme node in the deployment scheme node jump relationship diagram includes the corresponding pipeline parallelism and tensor parallelism, so the initial deployment scheme node where the initial pipeline parallelism and the initial tensor parallelism are located can be located on the deployment scheme node jump relationship diagram. The reinforcement learning model is trained based on multiple sample scheme node jump relationship diagrams, sample initial deployment scheme nodes, and label deployment scheme nodes. Specifically, the reinforcement learning model starts from the sample initial deployment scheme node of the sample scheme node jump relationship diagram, and jumps according to the principle of maximizing the reward value until it jumps to the label deployment scheme node, and the label deployment scheme node is the node with the most optimized reasoning performance in the sample scheme node jump relationship diagram. Then, the deployment plan node jump relationship diagram and the initial deployment plan node are used as the input of the trained reinforcement learning model. The reinforcement learning model can jump from the initial deployment plan node to the target deployment plan node with the best reasoning performance in the deployment plan node jump relationship diagram, and further determine the target pipeline parallelism and target tensor parallelism in the target deployment plan node. Since the target deployment plan node corresponding to the target pipeline parallelism and target tensor parallelism is the node with the best reasoning performance, the processing unit is deployed for the target learning model based on the target pipeline parallelism and target tensor parallelism, so that the reasoning performance of the deployed target learning model is optimized to meet the requirements of low latency and high throughput. In addition, the hardware deployment process of the target learning model only inputs the deployment plan node jump relationship diagram and the initial deployment plan node into the reinforcement learning model to obtain the deployment plan with the best reasoning performance. It is applicable to a variety of AI accelerators on the basis of realizing automatic parallelization, which improves the applicability of the deployment method, expands the scope of application, and improves the deployment efficiency.

[0090] The above is a general description of step 310 to step 340. Since step 320 and step 340 have been described in detail above, only the specific implementation process of step 310 and step 330 will be described in detail below.

[0091] Detailed description of step 310

[0092] In step 310, the initial pipeline parallelism and initial tensor parallelism of the processing unit deployment for the target learning model are obtained.

[0093] In one embodiment, referring to Figure 6 , step 310 includes:

[0094] Step 610, determining the storage capacity occupied by single-layer processing of the target learning model;

[0095] Step 620: Obtain the maximum storage capacity of the processing unit;

[0096] Step 630, determining the greatest common divisor of the storage capacity occupied by single-layer processing and the maximum storage capacity;

[0097] Step 640: Determine the initial pipeline parallelism and the initial tensor parallelism based on the greatest common divisor.

[0098] Steps 610 to 640 are described in detail below.

[0099] In step 610, the storage capacity occupied by single-layer processing of the target learning model is determined.

[0100] The storage capacity occupied by a single layer of processing is the storage capacity occupied by each layer of the target learning model during the inference process. In addition to the storage capacity required to place the model, the target learning model also needs to occupy other storage space during the inference process. Therefore, the storage capacity occupied by a single layer of processing includes not only the storage capacity occupied by the target learning model itself, but also the storage capacity occupied by inference. Figure 4A and Figure 4B ,If the target learning model has 6 layers, then the storage capacity occupied by single-layer processing is the storage capacity occupied by one layer of the target learning model during the reasoning process.

[0101] In step 620, the maximum storage capacity of the processing unit is obtained.

[0102] The maximum storage capacity is the maximum value of the storage capacity of the processing unit, which corresponds to the processing unit. Different types of processing units have different maximum storage capacities, and multiple processing units of the same type may also have different maximum storage capacities. Figure 2B , the maximum storage capacity is the maximum value of the storage capacity of the NPU accelerator.

[0103] In step 630, the greatest common divisor of the storage capacity occupied by the single-layer processing and the maximum storage capacity is determined.

[0104] Assume that the single-layer processing capacity of the target learning model is (params+kv cache ) / Layers, the maximum storage capacity of the processing unit is memory chip , then the greatest common divisor of the storage capacity occupied by single-layer processing and the maximum storage capacity can be expressed as:

[0105] Val gcd =GCD((params+kv cache ) / Layers,Memory chip ),

[0106] Among them, Val gcdIt is the greatest common divisor of the storage capacity occupied by single-layer processing and the maximum storage capacity. GCD means solving the greatest common divisor.

[0107] In one embodiment, assuming that the storage capacity occupied by a single-layer processing of the target learning model is 20 and the maximum storage capacity of the processing unit is 15, then the greatest common divisor of the storage capacity occupied by the single-layer processing and the maximum storage capacity is 5.

[0108] In step 640 , an initial pipeline parallelism number and an initial tensor parallelism number are determined based on the greatest common divisor.

[0109] After determining the greatest common divisor, the initial deployment parameters of the target learning model, namely the initial pipeline parallelism number and the initial tensor number, can be further determined based on the greatest common divisor.

[0110] The embodiments of the above steps 610 and 640 determine the initial pipeline parallelism and the initial tensor parallelism based on the greatest common divisor of the storage capacity occupied by the single-layer processing of the target learning model and the maximum storage capacity of the processing unit. The initial pipeline parallelism and the initial tensor parallelism are determined based on the parameters of the target learning model and the parameters of the processing unit, so the initial pipeline parallelism and the initial tensor parallelism are more matched with the target learning model and the processing unit, reducing the search space from the initial deployment solution node to the target deployment solution node, reducing the time for the reinforcement learning model to determine the target deployment solution node, and thus improving the hardware deployment efficiency of the learning model.

[0111] The above is a general description of step 610 to step 640. Since step 620 and step 630 have been described in sufficient detail above, only the specific implementation process of step 610 and step 640 will be described in detail below.

[0112] In step 610, the storage capacity occupied by single-layer processing of the target learning model is determined.

[0113] In one embodiment, referring to Figure 7 , step 610 includes:

[0114] Step 710: Obtain the parameter quantity of the target learning model;

[0115] Step 720: Obtain the maximum key value cache size used in the target learning model inference;

[0116] Step 730: Determine the storage capacity occupied by single-layer processing based on the first sum of the parameter quantity and the maximum key-value cache size and the number of layers of the target learning model.

[0117] Steps 710 to 730 are described in detail below.

[0118] In step 710, the parameter quantity of the target learning model is obtained.

[0119] The number of parameters refers to the number of parameters that need to be learned in the target learning model. It is an important indicator to measure the complexity of the model. The more parameters there are, the stronger the expressive power of the model is, but it also means that the model requires more computing resources and data for training and reasoning. It can be said that the number of parameters is also used to indicate the size of the storage capacity occupied by the target learning model.

[0120] It should be noted that for the convolutional layer, the parameter quantity is the product of the number of input channels, the height of the convolution kernel, the width of the convolution kernel, and the number of output channels. For the fully connected layer, the parameter quantity is the product of the number of input nodes and the number of output nodes.

[0121] In step 720, the maximum key-value cache size used in the target learning model inference is obtained.

[0122] In the process of inferring the target learning model, such as the Transformer model, keys and values ​​are generated, and keys and values ​​need to occupy a certain amount of storage capacity. At this time, the maximum key-value cache size is the maximum value of the storage capacity occupied by the key or value.

[0123] In step 730, the storage capacity occupied by single-layer processing is determined based on the first sum of the parameter quantity and the maximum key-value cache size and the number of layers of the target learning model.

[0124] The first sum is the sum of the number of parameters and the maximum key-value cache size. The first sum can be understood as the maximum value of the storage capacity that the target learning model needs to occupy during the inference process.

[0125] The storage capacity occupied by a single layer of processing is the ratio of the first sum and the number of layers of the target learning model. The storage capacity occupied by a single layer of processing can be specifically expressed as (params+kv cache ) / Layers, where parrams is the number of parameters of the target learning model, kv cache is the maximum key-value cache size used in the target learning model inference, and Layers is the number of layers of the target learning model.

[0126] It should be noted that if the number of parameters of the target learning model is 1000, the maximum key-value cache size is 100, and the number of layers is 5, then the storage capacity occupied by single-layer processing is 220.

[0127] The embodiments of steps 710 to 730 above determine the storage capacity occupied by single-layer processing based on the first sum of the parameter quantity and the maximum key value cache size and the number of layers of the target learning model. The storage capacity occupied by single-layer processing is determined based on the parameter quantity and the maximum key value cache size, ensuring that the reasoning process of the target learning model has sufficient storage capacity and can run normally, improving the accuracy of the initial pipeline parallel number and the initial tensor parallel number, and thus improving the hardware deployment efficiency of the learning model.

[0128] In step 640 , an initial pipeline parallelism number and an initial tensor parallelism number are determined based on the greatest common divisor.

[0129] In one embodiment, referring to Figure 8 , step 640 includes:

[0130] Step 810: Determine the initial tensor parallel number based on the storage capacity occupied by single-layer processing and the greatest common divisor;

[0131] Step 820: Determine the initial pipeline parallel number based on the number of layers, maximum storage capacity, and greatest common divisor of the target learning model.

[0132] Step 810 and step 820 are described in detail below.

[0133] In step 810, the initial tensor parallel number is determined based on the storage capacity occupied by single-layer processing and the greatest common divisor.

[0134] The initial tensor parallel number is determined based on the single-layer storage capacity and the greatest common divisor. Specifically, the initial tensor parallel number is the ratio of the single-layer storage capacity to the greatest common divisor. If the single-layer storage capacity is (params+kv cache ) / Layers, the greatest common divisor is Val gcd , then the initial tensor parallel number can be expressed as:

[0135]

[0136] It should be noted that since the greatest common divisor is the storage capacity occupied by a single layer of processing and the maximum storage capacity, the initial tensor parallel number is an integer. Assuming that the storage capacity occupied by a single layer of processing is 20 and the greatest common divisor is 5, the initial tensor parallel number is 4.

[0137] In step 820, the initial pipeline parallelism number is determined based on the number of layers, maximum storage capacity, and greatest common divisor of the target learning model.

[0138] The initial pipeline parallelism number first needs to calculate the ratio of the maximum storage capacity and the greatest common divisor, and then calculate the ratio of the number of layers of the target learning model to this ratio. The initial pipeline parallelism number can be expressed as:

[0139] PP=Layers / (Memory chip / Val gcd ),

[0140] Among them, Layers is the number of layers of the target learning model, Memory chip is the maximum storage capacity, Val gcd is the greatest common divisor.

[0141] Assuming the greatest common divisor is 5, the maximum storage capacity is 15, and the number of model layers is 6, the initial pipeline parallel number is 2.

[0142] The embodiments of the above-mentioned steps 810 and 820 calculate the initial pipeline parallelism and tensor pipeline parallelism based on the greatest common divisor and the storage capacity occupied by single-layer processing of the target learning model, the number of layers, and the maximum storage capacity of the processing unit, so that the calculated initial pipeline parallelism and tensor pipeline parallelism are compatible with the target learning model and the processing unit, further improving the accuracy of the initial pipeline parallelism and tensor pipeline parallelism, and being able to improve the hardware deployment efficiency of the learning model.

[0143] Detailed description of step 330

[0144] In step 330, the deployment solution node jump relationship diagram and the initial deployment solution node are input into the reinforcement learning model to obtain the target deployment solution node that optimizes the reasoning performance of the target learning model.

[0145] In one embodiment, referring to Fig. 9 , step 330 includes:

[0146] Step 910: Input the deployment solution node jump relationship graph and the initial deployment solution node into the reinforcement learning model, so that the reinforcement learning model selects a candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units from the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph;

[0147] Step 920: Based on the direct strategy function and the indirect strategy function, a jump strategy is selected between jumping to any candidate deployment solution node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the deployment solution node after the candidate deployment solution node;

[0148] Step 930: When the selected jump strategy is to jump to a candidate deployment solution node, the candidate deployment solution node to which the jump is made is used as the initial deployment solution node, and the process returns to the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship diagram, and selects the candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units, until the selected jump strategy is not to jump, and the initial deployment solution node when not to jump is selected is used as the target deployment solution node.

[0149] Steps 910 to 930 are described in detail below.

[0150] In step 910, the deployment plan node jump relationship graph and the initial deployment plan node are input into the reinforcement learning model, so that the reinforcement learning model selects a candidate deployment plan node that meets the minimum number of processing units and the maximum number of processing units from the nodes adjacent to the initial deployment plan node in the deployment plan node jump relationship graph.

[0151] The candidate deployment solution node is a node adjacent to the initial deployment solution node and is a node to which the current initial deployment solution node may jump next.

[0152] The minimum number of processing units and the maximum number of processing units limit the number of processing units, thereby reducing the number of candidate deployment solution nodes and improving jump efficiency.

[0153] Assume that the maximum number of processing units is 15 and the minimum number of processing units is 5. Based on the minimum number of processing units and the number of processing units in Figure 5A The node jump relationship diagram of the deployment solution shown in FIG. 1 determines the candidate deployment solution node from the nodes adjacent to the initial deployment solution node, and deletes other nodes. The processed deployment solution node jump relationship diagram is shown in FIG. Fig.10 shown.

[0154] It should be noted that in the deployment solution node jump relationship diagram, adjacent nodes are connected by a single path. When deleting a node, the path connected to the node can be deleted together.

[0155] In step 920, based on the direct strategy function and the indirect strategy function, a jump strategy is selected between jumping to any candidate deployment solution node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the deployment solution node after the candidate deployment solution node.

[0156] The direct policy function reflects the reward value that can be obtained by jumping to the candidate deployment solution node itself, which can also be understood as the probability of choosing to jump to the candidate deployment solution node in the current state. The direct policy function can be expressed as πθ (a t |s t ), where π θ represents the direct policy function, θ represents the target learning model, and s t is the current state, which includes all nodes and path information from the initial deployment plan node to the current time step t, as well as the information of the (t-1) actions performed previously. t Indicates the action of selecting a candidate deployment solution node from the current state.

[0157] It should be noted that, refer to Fig.10 , for the initial deployment solution node with pipeline parallelism of 5 and tensor parallelism of 1, since it has not jumped to other nodes, the current state only includes the initial deployment solution node. In addition, the initial deployment solution node has two candidate deployment solution nodes, so there are two actions in total, one of which is to jump to candidate deployment solution node 1 with pipeline parallelism of 5 and tensor parallelism of 2, and the other is to jump to candidate deployment solution node 2 with pipeline parallelism of 5 and tensor parallelism of 3. The number of values ​​of the indirect policy function corresponds to the number of candidate deployment solution nodes, and the two indirect policy functions represent the probabilities of jumping from the initial deployment solution node to candidate deployment solution nodes 1 and 2, respectively.

[0158] The indirect policy function reflects the reward value that can be obtained by jumping to the deployment solution node after the candidate deployment solution node. It can also be understood as the probability of jumping to the deployment solution node after the candidate deployment solution node. The direct policy function focuses on the short-term jump probability, while the indirect policy function focuses on the long-term jump probability. The indirect policy function can be expressed as Q θ (a t ,s t ), where Q represents the indirect policy function. Fig.10 ,For the candidate deployment scheme nodes with pipeline parallelism of 5 and tensor parallelism of 2, the indirect policy function of the initial deployment scheme node also reflects the probability of jumping from the node to other adjacent nodes.

[0159] The jump strategy includes jumping to any candidate deployment solution node or not jumping. Fig.10 The initial deployment solution node shown has three jump strategies, namely, a strategy of jumping to two candidate deployment solution nodes and a strategy of not jumping.

[0160] In step 930, when the selected jump strategy is to jump to a candidate deployment solution node, the candidate deployment solution node to which it jumps is used as the initial deployment solution node, and the process returns to the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship diagram, and the step of selecting candidate deployment solution nodes that meet the minimum number of processing units and the maximum number of processing units is performed, until the selected jump strategy is not to jump, and the initial deployment solution node when not to jump is selected is used as the target deployment solution node.

[0161] When the selected jump strategy is to jump to a candidate deployment solution node, the candidate deployment solution node jumped to is used as the initial deployment solution node, and the process returns to the step of selecting the candidate deployment solution node to perform a new jump.

[0162] When the selected jump strategy is not to jump, the current initial deployment solution node can be considered as the point with the best reasoning performance, and the node is used as the target deployment solution node.

[0163] Reference Fig.10 In the deployment plan node jump relationship diagram, first jump from the initial deployment plan node to the candidate deployment plan node with a pipeline parallelism of 5 and a tensor parallelism of 3. Then, use the node as the new initial deployment plan node, and based on the direct strategy function and indirect strategy function of the corresponding candidate deployment plan node, choose not to jump. Then use the node with a pipeline parallelism of 5 and a tensor parallelism of 3 as the target deployment plan node.

[0164] Reference Fig.11 , the disclosed embodiment outputs the deployment scheme node jump relationship diagram and the initial deployment scheme node to the reinforcement learning model. The reinforcement learning model selects a candidate deployment scheme node from the nodes adjacent to the initial deployment scheme node based on the restrictions of the minimum number of processing units and the maximum number of processing units. Afterwards, based on the direct strategy function and the indirect strategy function, a jump strategy is selected between jumping to any candidate deployment scheme node and not jumping. If the jump strategy is to jump to a candidate deployment scheme node, then the candidate deployment scheme node is used as the new initial deployment scheme node, and returns to the step of selecting a candidate deployment scheme node from the nodes adjacent to the initial deployment scheme node based on the restrictions of the minimum number of processing units and the maximum number of processing units to perform a new jump. If the jump strategy is not to jump, then the current initial deployment scheme node is used as the target deployment scheme node.

[0165] The embodiments of the above steps 910 to 930 select the jump strategy of jumping to any candidate deployment solution node and not jumping based on the direct strategy function and the indirect strategy function, and when the jump strategy is to jump to a candidate deployment solution node, the candidate deployment solution node is used as the new initial deployment solution node, and the step of selecting the candidate deployment solution node is returned until the jump strategy is not to jump, and the current initial deployment solution node is used as the target deployment solution node. In the deployment solution node jump relationship diagram, the reinforcement learning model can automatically jump from the initial deployment solution node to the target deployment solution node based on the direct strategy function and the indirect strategy function, realizing the automatic parallelization of the hardware deployment of the learning model and improving the deployment efficiency.

[0166] The above is a general description of step 910 to step 930. Since step 930 has been described in detail above, only the specific implementation process of step 910 and step 920 will be described in detail below.

[0167] In step 910, a candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units is selected from the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph.

[0168] In one embodiment, referring to Fig.12 , step 910 includes:

[0169] Step 1210: for a single adjacent node adjacent to the initial deployment solution node, based on the pipeline parallelism and tensor parallelism in the single adjacent node, determine the number of processing unit occupancy of the single adjacent node;

[0170] Step 1220: If the number of occupied processing units is between the minimum number of processing units and the maximum number of processing units, a single adjacent node is used as a candidate deployment solution node.

[0171] Step 1210 and step 1220 are described in detail below.

[0172] In step 1210, for a single adjacent node adjacent to the initial deployment solution node, the number of processing unit occupancy of the single adjacent node is determined based on the pipeline parallelism and tensor parallelism in the single adjacent node.

[0173] The number of processing unit occupancy is the number of processing units deployed for the target learning model under the deployment strategy corresponding to the node.

[0174] The number of processing units occupied is the product of pipeline parallelism and tensor parallelism. Figure 5A, but for the initial deployment solution node, it contains three adjacent nodes. For a node with 4 pipeline parallelism and 1 tensor parallelism, the number of processing units occupied is 4. For a node with 5 pipeline parallelism and 2 tensor parallelism, the number of processing units occupied is 10. For a node with 5 pipeline parallelism and 3 tensor parallelism, the number of processing units occupied is 15.

[0175] In step 1220, if the number of occupied processing units is between the minimum number of processing units and the maximum number of processing units, a single adjacent node is used as a candidate deployment solution node.

[0176] The number of processing units occupied by the candidate deployment solution node must meet the minimum number of processing units and the maximum number of processing units. Therefore, if the number of processing units occupied by the node is between the minimum number of processing units and the maximum number of processing units, the node will be used as a candidate deployment solution node for the initial deployment solution node.

[0177] Reference Figure 5A , assuming that the maximum number of processing units is 15 and the minimum number of processing units is 5, then the adjacent nodes with processing unit occupancy of 10 and 15 are taken as candidate deployment solution nodes for the initial deployment solution nodes, thus obtaining Fig.10 The deployment scenario node jump relationship diagram in.

[0178] The embodiments of step 1210 and step 1220 above determine the number of processing unit occupancy of a single adjacent node based on the pipeline parallelism and tensor parallelism in the single adjacent node, and then use the adjacent node whose processing unit occupancy number is between the minimum number of processing units and the maximum number of processing units as the candidate deployment solution node. The minimum number of processing units and the maximum number of processing units limit the number of processing units occupied by the node, and then limit the number of candidate deployment solution nodes to be jumped, reducing the number of nodes to be processed and further improving the deployment efficiency.

[0179] In step 920, based on the direct policy function and the indirect policy function, a jump strategy is selected between jumping to any candidate deployment solution node and not jumping.

[0180] In one embodiment, referring to Fig.13 , step 920 includes:

[0181] Step 1310: Calculate the third sum of the jump strategy of jumping to any candidate deployment solution node and the jump strategy of not jumping, where the third sum is the sum of the direct strategy function and the indirect strategy function;

[0182] Step 1320: Determine the third and largest jump strategy as the selected jump strategy.

[0183] Step 1310 and step 1320 are described in detail below.

[0184] In step 1310, the third sum of the jump strategy to jump to any candidate deployment solution node and the jump strategy not to jump is calculated, and the third sum is the sum of the direct strategy function and the indirect strategy function.

[0185] The third sum is the sum of the direct policy function and the indirect policy function. Assuming that the direct policy function of jumping to candidate deployment solution node 1 is 0.4 and the indirect policy function is 0.2, the third sum is 0.6.

[0186] Reference Fig.10 , the third sum of jumping from the initial deployment solution node to the node with 5 pipeline parallelism and 3 tensor parallelism is 0.8. The third sum of jumping from the initial deployment solution node to the node with 5 pipeline parallelism and 2 tensor parallelism is 0.6. The third sum of not jumping is 0.4.

[0187] In step 1320, the third and largest jump strategy is determined as the selected jump strategy.

[0188] The larger the value of the third sum is, the greater the probability of selecting the jump strategy corresponding to the third sum is. Therefore, the jump strategy with the largest third sum is determined as the selected jump strategy. Fig.10 , jump from the initial deployment plan node to the third and largest node with a pipeline parallelism of 5 and a tensor parallelism of 3, so jump to the initial deployment plan node to jump to the node with a pipeline parallelism of 5 and a tensor parallelism of 3 as the selected jump strategy.

[0189] The embodiments of step 1310 and step 1320 above determine the third and largest jump strategy of the direct strategy function and the indirect strategy function as the selected jump strategy, ensuring that it is possible to quickly jump to the node with the most optimized reasoning performance, thereby improving the processing efficiency of the reinforcement learning model and improving the deployment efficiency.

[0190] Determination of the minimum number of processing units

[0191] In step 910, the deployment solution node jump relationship graph and the initial deployment solution node are input into the reinforcement learning model, so that the reinforcement learning model selects a candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units from the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph. In this case, in one embodiment, referring to Fig.14 , the steps for determining the minimum number of processing units include:

[0192] Step 1410: Obtain the parameter quantity of the target learning model;

[0193] Step 1420, obtaining the maximum storage capacity of the processing unit;

[0194] Step 1430: Determine the minimum number of processing units based on the parameter quantity and the maximum storage capacity.

[0195] It should be noted that the minimum number of processing units is the minimum number of processing units deployed for the target learning model. The minimum number of processing units is determined based on the parameters of the target learning model and the maximum storage capacity of the processing unit. Specifically, the minimum number of processing units is the ratio of the parameter amount to the maximum storage capacity. If the ratio is a decimal, the minimum number of processing units needs to be rounded up based on the ratio.

[0196] In addition, during the inference process of the target learning model, keys and values ​​that require a certain amount of storage capacity will be generated. At this time, the minimum number of processing units is determined based on the number of parameters, the maximum key value cache size, and the maximum storage capacity. Specifically, the minimum number of processing units is the ratio of the first sum of the number of parameters and the maximum key value cache size to the maximum storage capacity. The minimum number of processing units can be expressed as:

[0197] (params+kv cache ) / Memory chip ,

[0198] Among them, parrams is the parameter amount of the target learning model, kv cache The maximum key-value cache size used in the target learning model inference, Memory chip is the maximum storage capacity. If the number of parameters of the target learning model is 1000, the maximum key-value cache size is 100, and the maximum storage capacity is 400, then the minimum number of processing units is 3.

[0199] The above-mentioned embodiments of steps 1410 to 1430 determine the minimum number of processing units based on parameter quantities and maximum storage capacity, thereby ensuring the accuracy of the minimum number of processing units. In addition, the determination of the minimum number of processing units can limit the candidate deployment solution nodes, thereby improving the deployment efficiency.

[0200] The process of generating the deployment plan node jump relationship diagram

[0201] In one embodiment, referring to Fig.15 ,The generation process of the deployment scheme node jump relationship diagram includes:

[0202] Step 1510: Generate an anchor deployment solution node, where the anchor deployment solution node has an anchor pipeline parallel number and an anchor tensor parallel number;

[0203] Step 1520: increase or decrease only one of the anchor pipeline parallelism number and the anchor tensor parallelism number of the anchor deployment solution node, obtain the neighbor deployment solution node of the anchor deployment solution node, and connect the neighbor deployment solution node and the anchor deployment solution node;

[0204] Step 1530, update the anchor deployment solution node with the neighbor deployment solution node, and return to the step of increasing or decreasing only one of the anchor pipeline parallelism and the anchor tensor parallelism of the anchor deployment solution node, until the number of returns reaches a predetermined number.

[0205] Steps 1510 to 1530 are described in detail below.

[0206] In step 1510, an anchor deployment solution node is generated, and the anchor deployment solution node has an anchor pipeline parallel number and an anchor tensor parallel number.

[0207] The anchor deployment solution node is an anchor node for generating a deployment solution node jump relationship graph, and the node can be any node in the deployment solution node jump relationship graph. In addition, the anchor deployment solution node also includes corresponding pipeline parallelism and tensor parallelism, namely, anchor pipeline parallelism and anchor tensor parallelism.

[0208] In step 1520, only one of the anchor pipeline parallelism number and the anchor tensor parallelism number of the anchor deployment solution node is increased or decreased to obtain the neighbor deployment solution node of the anchor deployment solution node, and the neighbor deployment solution node and the anchor deployment solution node are connected.

[0209] A neighbor deployment solution node is a node adjacent to an anchor deployment solution node, and the neighbor deployment solution node is connected to the anchor deployment solution node through a single path. In the deployment solution node jump relationship graph, adjacent nodes differ only in pipeline parallelism or tensor parallelism. Therefore, in the generation process of the neighbor deployment solution node, only one of the anchor pipeline parallelism and the anchor tensor parallelism needs to be increased or decreased.

[0210] It should be noted that the increase or decrease of the anchor pipeline parallelism number or the anchor tensor parallelism number can be any value, as long as the anchor pipeline parallelism number and the anchor tensor parallelism number are both positive integers.

[0211] In step 1530, the anchor deployment solution node is updated with the neighbor deployment solution node, and the step of increasing or decreasing only one of the anchor pipeline parallelism and the anchor tensor parallelism of the anchor deployment solution node is returned until the number of returns reaches a predetermined number of times.

[0212] The predetermined number of times can be regarded as the number of paths traversed by the neighbor deployment solution nodes corresponding to the first anchor deployment solution node to the last returned anchor deployment solution node.

[0213] Reference Figure 5A, assuming that the anchor pipeline parallelism of the anchor deployment solution node is 4 and the anchor tensor parallelism is 1. Only the anchor tensor parallelism is increased by 1 to obtain a neighbor deployment solution node with a pipeline parallelism of 4 and a tensor parallelism of 2, and connect the neighbor deployment solution node and the anchor deployment solution node. After that, the neighbor deployment solution node with a pipeline parallelism of 4 and a tensor parallelism of 2 is used as the new anchor deployment solution node, and the steps of increasing or decreasing only one of the anchor pipeline parallelism and the anchor tensor parallelism of the anchor deployment solution node are returned until the number of returns reaches the predetermined number. When the multiple neighbor deployment solution nodes corresponding to the node with a pipeline parallelism of 4 and a tensor parallelism of 2 are set, return to the node with an anchor pipeline parallelism of 4 and an anchor tensor parallelism of 1, and increase the pipeline parallelism by 1 to start the setting of the new neighbor deployment solution node.

[0214] In one embodiment, a deployment solution node jump relationship graph is generated based on the minimum number of processing units and the maximum number of processing units, and then the nodes in the deployment solution node jump relationship graph all meet the restrictions of the minimum number of processing units and the maximum number of processing units. In this case, the nodes adjacent to the initial deployment solution node are all candidate deployment solution nodes.

[0215] The above-mentioned embodiments of steps 1510 to 1530 start from the deployment scheme node, and only increase or decrease one of the anchor pipeline parallel number and the anchor tensor parallel number of the anchor deployment scheme node, obtain the neighbor deployment scheme node of the anchor deployment scheme node, and connect the neighbor deployment scheme node and the anchor deployment scheme node. After that, the anchor deployment scheme node is updated with the neighbor deployment scheme node, and the neighbor deployment scheme node of the updated anchor deployment scheme node is determined. This method makes the information of each node in the generated deployment scheme node jump relationship graph more comprehensive, reduces deployment errors caused by errors in the jump relationship graph, improves the accuracy of the hardware deployment of the learning model, and improves the deployment efficiency.

[0216] The training process of reinforcement learning model

[0217] The reinforcement learning model is trained based on multiple sample solution node jump relationship graphs and the label deployment solution nodes in each sample solution node jump relationship graph. The training goal of the reinforcement learning model is to accurately jump from the sample initial deployment solution node to the label deployment solution node in the sample solution node jump relationship graph.

[0218] In one embodiment, referring to Fig.16 , the training process of the reinforcement learning model includes:

[0219] Step 1610: for each sample solution node jump relationship graph in a plurality of sample solution node jump relationship graphs, determine a label deployment solution node with the best reasoning performance;

[0220] Step 1620: Select a candidate sample deployment solution node that meets the minimum number of processing units and the maximum number of processing units from among the nodes adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram;

[0221] Step 1630: Based on the direct strategy function and the indirect strategy function, a sample jump strategy is selected between jumping to any candidate sample deployment solution node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate sample deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the sample deployment solution node after the candidate sample deployment solution node;

[0222] Step 1640: when the selected sample jump strategy is to jump to a candidate sample deployment solution node, the candidate sample deployment solution node to which the sample is jumped is used as the sample initial deployment solution node, and the process returns to the step of selecting a sample deployment solution node that meets the minimum number of processing units and the maximum number of processing units restrictions from the nodes adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram, until the selected sample jump strategy is not to jump, and the sample initial deployment solution node when not to jump is selected is used as the predicted deployment solution node;

[0223] Step 1650: If the predicted deployment solution node is inconsistent with the label deployment solution node, adjust the direct policy function and the indirect policy function until the predicted deployment solution node is consistent with the label deployment solution node.

[0224] Steps 1610 to 1650 are described in detail below.

[0225] In step 1610, for each sample solution node jump relationship graph in a plurality of sample solution node jump relationship graphs, a label deployment solution node with the best inference performance is determined.

[0226] The sample solution node jump relationship diagram corresponds to the deployment solution node jump relationship diagram. Each node in the sample solution node jump relationship diagram includes the pipeline parallel number and the tensor parallel number. The sample solution node jump relationship diagram corresponds to the sample learning model. The sample learning model is the training sample of the reinforcement learning model. Multiple sample solution node jump relationship diagrams can be the same diagram, but different sample learning models have different corresponding label deployment solution nodes and sample initial deployment solution nodes in the sample solution node jump relationship diagram.

[0227] The label deployment solution node is the node with the best reasoning performance in the sample learning model in the sample solution node jump relationship diagram. Fig.17, the pipeline parallelism of the label deployment scheme node is 5, and the tensor parallelism is 3. Under this pipeline parallelism and tensor parallelism, the inference performance of the sample learning model corresponding to the label deployment scheme node is optimized.

[0228] In step 1620, a candidate sample deployment solution node that meets the minimum number of processing units and the maximum number of processing units is selected from the nodes adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram.

[0229] The candidate sample deployment solution node is a node adjacent to the sample initial deployment solution node, and is a node to which the current sample initial deployment solution node may jump next.

[0230] The minimum number of processing units and the maximum number of processing units limit the number of processing units, thereby reducing the number of sample candidate deployment solution nodes and improving jump efficiency.

[0231] It should be noted that the number of pipeline parallelism and tensor parallelism of the sample initial deployment solution node is determined based on the sample learning model and the parameters of the processing unit. First, the greatest common divisor of the storage capacity occupied by the single-layer processing of the sample learning model and the maximum storage capacity of the processing unit is determined, and then the number of pipeline parallelism and tensor parallelism of the sample initial deployment solution node is determined based on the greatest common divisor.

[0232] In step 1630, based on the direct strategy function and the indirect strategy function, a sample jump strategy is selected between jumping to any candidate sample deployment scheme node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate sample deployment scheme node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the sample deployment scheme node after the candidate sample deployment scheme node.

[0233] The direct policy function reflects the reward value that can be obtained by jumping to the candidate sample deployment solution node itself, which can also be understood as the probability of choosing to jump to the candidate sample deployment solution node in the current state. The direct policy function can be expressed as π θ (a t |s t ), where π θ represents the direct policy function, θ represents the sample learning model, s t is the current state, which includes all nodes, path information, and information about the (t-1) actions taken from the initial sample deployment solution node to the current time step t. t Indicates the action of selecting and jumping to a candidate sample deployment solution node from the current state.

[0234] It should be noted that, assuming Fig.17The nodes in all meet the restrictions of the minimum number of processing units and the maximum number of processing units. For the sample initial deployment plan node with a pipeline parallelism of 5 and a tensor parallelism of 1, since it has not jumped to other nodes, the current state only includes the sample initial deployment plan node. In addition, the initial deployment plan node has three candidate deployment plan nodes, so there are three actions, which are jumping to the candidate deployment plan nodes. The number of values ​​of the indirect policy function corresponds to the number of candidate sample deployment plan nodes, and the three indirect policy functions respectively represent the probabilities of jumping from the sample initial deployment plan node to the three candidate sample deployment plan nodes.

[0235] The indirect strategy function reflects the reward value that can be obtained by jumping to the sample deployment solution node after the candidate sample deployment solution node. It can also be understood as the probability of jumping to the sample deployment solution node after the candidate sample deployment solution node. The direct strategy function focuses on the short-term jump probability, while the indirect strategy function focuses on the long-term jump probability. The indirect strategy function can be expressed as Q θ (a t ,s t ), where Q represents the indirect policy function. Fig.17 ,For the candidate sample deployment scheme node with pipeline parallelism of 5 and tensor parallelism of 2, the indirect policy function of the sample initial deployment scheme node also reflects the probability of jumping from the node to other nodes adjacent to it.

[0236] The sample jump strategy includes jumping to any candidate sample deployment solution node or not jumping. Fig.11 The sample initial deployment solution node shown includes four sample jump strategies, namely, a strategy of jumping to three candidate sample deployment solution nodes and a strategy of not jumping.

[0237] In step 1640, when the selected sample jump strategy is to jump to a candidate sample deployment plan node, the candidate sample deployment plan node to which it jumps is used as the sample initial deployment plan node, and the process returns to the nodes adjacent to the sample initial deployment plan node in the sample plan node jump relationship diagram, and the step of selecting the sample deployment plan node that meets the minimum number of processing units and the maximum number of processing units is performed, until the selected sample jump strategy is not to jump, and the sample initial deployment plan node when not to jump is selected is used as the predicted deployment plan node.

[0238] When the selected sample jump strategy is to jump to a candidate sample deployment solution node, the candidate sample deployment solution node jumped to is used as the sample initial deployment solution node, and the process returns to the step of selecting the candidate sample deployment solution node to perform a new jump.

[0239] When the selected sample jump strategy is not to jump, the current sample initial deployment solution node can be considered as the point with the best reasoning performance, and the node is used as the prediction deployment solution node. The prediction deployment solution node is the node with the best reasoning performance predicted by the reinforcement learning model according to the current parameters of the sample learning model corresponding to the sample initial deployment solution node.

[0240] Reference Fig.17 In the sample solution node jump relationship diagram, first jump from the sample initial deployment solution node to the candidate sample deployment solution node with a pipeline parallelism of 5 and a tensor parallelism of 3. Then, use the node as the new sample initial deployment solution node, and based on the direct strategy function and indirect strategy function of the corresponding candidate sample deployment solution node, choose not to jump. Then use the node with a pipeline parallelism of 5 and a tensor parallelism of 3 as the prediction deployment solution node.

[0241] In step 1650, if the predicted deployment solution node is inconsistent with the label deployment solution node, adjust the direct policy function and the indirect policy function until the predicted deployment solution node is consistent with the label deployment solution node.

[0242] The predicted deployment plan node is the node with the best predicted reasoning performance. However, the predicted deployment plan node cannot be determined to be the node with the best reasoning performance of the sample learning model. If the predicted deployment plan node is inconsistent with the label deployment plan node, it means that the reinforcement learning model prediction is wrong, and the direct policy function and indirect policy function need to be adjusted until the predicted deployment plan node is consistent with the label deployment plan node.

[0243] It can be said that the training process of the reinforcement learning model is the process of repeatedly adjusting the direct strategy function and the indirect strategy function to optimize the output reasoning performance node.

[0244] Reference Fig.17 , assuming that the reinforcement learning model jumps to the sample initial deployment solution node based on the sample solution node jump relationship diagram, the pipeline parallelism of the predicted deployment solution node is 5 and the tensor parallelism is 2, and this node is inconsistent with the label deployment solution node. Then adjust the direct strategy function and the indirect strategy function until the pipeline parallelism of the predicted deployment solution node is 5 and the tensor parallelism is 3.

[0245] Reference Fig.18, the disclosed embodiment first determines the sample initial deployment solution node of the sample learning model in the sample solution node jump relationship diagram based on the model parameters of the sample learning model and the hardware parameters of the processing unit. Afterwards, the objective function is defined to determine the label deployment solution node with the best reasoning performance. Based on the direct strategy function and the indirect strategy function, a sample jump strategy is selected between jumping to any candidate sample deployment solution node and not jumping. If the predicted deployment solution node finally determined by the sample jump strategy is inconsistent with the label deployment solution node, the direct strategy function and the indirect strategy function are optimized and adjusted until the predicted deployment solution node is consistent with the label deployment solution node.

[0246] The embodiment of steps 1610 to 1650 trains the reinforcement learning model. When the predicted deployment solution node output by the reinforcement learning model is inconsistent with the label deployment solution node, the direct strategy function and the indirect strategy function are optimized and adjusted until the predicted deployment solution node is consistent with the label deployment solution node. This scheme trains the reinforcement learning model so that the reinforcement learning model can output a node with the most optimized reasoning performance, thereby realizing automatic parallelization of deployment and improving deployment efficiency.

[0247] The above is a general description of step 1610 to step 1650. Since step 1620, step 1640 and step 1650 have been described in detail above, only the specific implementation process of step 1610 and step 1630 will be described in detail below.

[0248] In step 1610, for each sample solution node jump relationship graph in a plurality of sample solution node jump relationship graphs, a label deployment solution node with the best inference performance is determined.

[0249] In one embodiment, the inference performance is the inference time, referring to Fig.19 , step 1610 includes:

[0250] Step 1910: for a single sample solution node jump relationship graph, obtain the pipeline parallelism number and tensor parallelism number in each sample deployment solution node in the single sample solution node jump relationship graph;

[0251] Step 1920: Determine the inference duration of the sample deployment solution node based on the pipeline parallelism and tensor parallelism in the sample deployment solution node;

[0252] Step 1930: determine the sample deployment solution node with the shortest inference time as the label deployment solution node.

[0253] Steps 1910 to 1930 are described in detail below.

[0254] In step 1910, for a single sample solution node jump relationship graph, the pipeline parallelism number and tensor parallelism number in each sample deployment solution node in the single sample solution node jump relationship graph are obtained.

[0255] For each sample solution node jump relationship graph, obtain the pipeline parallelism and tensor parallelism corresponding to each sample deployment solution node in the graph.

[0256] Assumptions Fig.17 The sample solution node jump relationship diagram shown is a complete sample solution node jump relationship diagram, so get Fig.17 The number of pipeline parallelism and tensor parallelism in the nodes of all sample deployment schemes in .

[0257] In step 1920, the inference duration of the sample deployment solution node is determined based on the pipeline parallelism and tensor parallelism in the sample deployment solution node.

[0258] The inference time is the time required for the sample learning model to perform inference under the pipeline parallelism and tensor parallelism corresponding to the sample deployment solution node. The inference time of the sample deployment solution node is determined based on the pipeline parallelism and tensor parallelism in the node.

[0259] In step 1930, the sample deployment solution node with the shortest inference time is determined as the label deployment solution node.

[0260] After determining the inference times of multiple sample deployment solution nodes in the sample solution node jump relationship graph, the inference times of the multiple sample deployment solution nodes are compared, and the sample deployment solution node with the smallest inference time is determined as the label deployment solution node.

[0261] Reference Fig.17 First, the pipeline parallelism and tensor parallelism in each sample deployment solution node in the sample solution node jump relationship diagram are calculated, and the inference time of the sample deployment solution node is determined based on the pipeline parallelism and tensor parallelism. After that, the inference time of multiple sample deployment solution nodes is compared, and the sample deployment solution node with the smallest inference time, that is, the node with pipeline parallelism of 5 and tensor parallelism of 3, is determined as the label deployment solution node.

[0262] In the embodiment of the above steps 1910 to 1930, the sample deployment solution node with the shortest inference time in the sample solution node jump relationship diagram is used as the label deployment solution node, thereby ensuring the correctness of the label deployment solution node, facilitating the training of the reinforcement learning model, and improving the performance of the trained reinforcement learning model.

[0263] The above is a general description of step 1910 to step 1930. The specific implementation process of step 1920 is described in detail below.

[0264] In step 1920, the inference duration of the sample deployment solution node is determined based on the pipeline parallelism and tensor parallelism in the sample deployment solution node.

[0265] In one embodiment, referring to Fig. 20 , step 1920 includes:

[0266] Step 2010: Obtain the sample model calculation amount, sample parameter amount, sample maximum key value cache size and sample model configuration parameters of the sample learning model, where the sample learning model is the model applied by the sample deployment solution node;

[0267] Step 220: Obtain the peak computing power, memory bandwidth, and communication bandwidth of the processing unit;

[0268] Step 2030: Determine the sample model calculation duration based on the sample model calculation amount, peak computing power, and the tensor parallel number in the sample deployment solution node;

[0269] Step 2040: Determine the sample model memory access duration based on the sample parameter quantity, the sample maximum key value cache size, the video memory bandwidth, and the tensor parallel number in the sample deployment solution node;

[0270] Step 2050: Determine the communication duration of the sample model based on the sample model configuration parameters, the communication bandwidth, and the tensor parallelism and pipeline parallelism in the sample deployment solution node;

[0271] Step 2060: Determine the inference duration of the sample deployment solution node based on the maximum value of the sample model calculation duration and the sample model memory access duration, and the sample model communication duration.

[0272] Steps 2010 to 2060 are described in detail below.

[0273] In step 2010, the sample model calculation amount, sample parameter amount, sample maximum key value cache size and sample model configuration parameters of the sample learning model are obtained, and the sample learning model is the model applied by the sample deployment solution node.

[0274] The sample learning model is a sample used for training the reinforcement learning model. The sample model calculation amount refers to the number of floating-point operations, which refers to the number of floating-point operations required to run a sample learning model. It is a measure of how many floating-point operations the sample learning model needs to perform during the inference process. The sample parameter amount refers to the number of parameters that need to be learned in the sample learning model, which is another important indicator for measuring the complexity of the model. The sample model configuration parameters refer to the parameters related to the sample model configuration, which are related to the size, input, output, etc. of the sample model.

[0275] In the process of sample learning model, such as Transformer model inference, keys and values ​​are generated, and keys and values ​​need to occupy a certain amount of storage capacity. At this time, the maximum key-value cache size is the maximum value of the storage capacity occupied by the key or value.

[0276] In step 2020, the peak computing power, memory bandwidth, and communication bandwidth of the processing unit are obtained.

[0277] Peak computing power refers to the maximum number of floating-point operations that a processing unit can perform per second under ideal conditions, and is usually used to measure the processing power of a processing unit. Video memory bandwidth refers to the data transfer rate between a processing unit and video memory. Communication bandwidth refers to the data transfer rate between multiple processing units.

[0278] In step 2030, the sample model calculation duration is determined based on the sample model calculation amount, peak computing power, and the number of tensor parallelism in the sample deployment solution node.

[0279] The sample model calculation time refers to the time required to run a sample learning model, which is determined based on the sample model calculation amount, peak computing power, and the number of tensor parallelism in the sample deployment solution node. The sample model calculation time can be expressed as:

[0280] math time =MACs / (TOPS*TP),

[0281] Among them, math time is the calculation time of the sample model, MACs is the calculation amount of the sample model, TOPS is the peak computing power, and TP is the number of tensor parallelism in the sample deployment solution node.

[0282] In step 2040, the sample model memory access duration is determined based on the sample parameter quantity, the sample maximum key-value cache size, the video memory bandwidth, and the number of tensor parallelism in the sample deployment solution node.

[0283] The sample model memory access time refers to the time required to access the video memory during the inference process of the sample learning model. It is determined based on the sample parameter quantity, the sample maximum key value cache size, the video memory bandwidth, and the tensor parallelism number in the sample deployment solution node. The sample model memory access time can be expressed as:

[0284] memory time =Size(params+kv cache ) / (BW memory *TP),

[0285] Among them, memory time is the sample model access time, params is the sample parameter quantity, kv cache The maximum key value cache size of the sample, BW memoryis the video memory bandwidth, TP is the number of tensor parallelism in the sample deployment scheme node, and Size is the unit conversion of the sample parameter quantity and the sample maximum key value cache size for calculation.

[0286] In step 2050, the communication duration of the sample model is determined based on the sample model configuration parameters, the communication bandwidth, and the tensor parallelism and pipeline parallelism in the sample deployment solution node.

[0287] The sample model communication time refers to the time required for communication between processing units during the sample learning model inference process.

[0288] In step 2060, the inference duration of the sample deployment solution node is determined based on the maximum value of the sample model calculation duration and the sample model memory access duration, and the sample model communication duration.

[0289] The inference time of the sample deployment solution node is the sum of the maximum value of the sample model calculation time and the sample model memory access time, and the communication time with the sample model. The inference time of the sample deployment solution node can be expressed as:

[0290] latency(TP,PP)=Max(math time ,memory time )+commu time ,

[0291] Among them, latency is the inference time, TP is the tensor parallel number, PP is the pipeline parallel number, Max represents the maximum value calculation, and commu time is the communication time of the sample model. Assuming that the calculation time of the sample model is 20 milliseconds, the memory access time of the sample model is 25 milliseconds, and the communication time of the sample model is 30 milliseconds, the inference time of the sample deployment solution node is 55 milliseconds.

[0292] The embodiments of steps 2010 to 2060 above determine the sample model calculation time, sample model memory access time and sample model communication time in turn based on the parameters of the sample learning model, the parameters of the processing unit, and the tensor parallelism and pipeline parallelism of the sample deployment scheme node, and then determine the inference time of the sample deployment scheme node, so that the obtained inference time is strongly correlated with the sample learning model, the processing unit, the tensor parallelism and the pipeline parallelism, thereby improving the accuracy of the inference time, facilitating the determination of the label deployment scheme node that minimizes the inference time, and improving the performance of the reinforcement learning model.

[0293] The above is a general description of step 2010 to step 2060. The specific implementation process of step 2030 to step 2060 is described in detail below.

[0294] In step 2030, the sample model calculation duration is determined based on the sample model calculation amount, peak computing power, and the number of tensor parallelism in the sample deployment solution node.

[0295] In one embodiment, referring to Fig.21 , step 2030 includes:

[0296] Step 2110: determine the product of the peak computing power and the tensor parallel number as the tensor parallel computing power;

[0297] Step 2120: Determine the sample model calculation duration based on the ratio of the sample model calculation amount and the tensor parallel computing power.

[0298] Step 2110 and step 2120 are described in detail below.

[0299] In step 2110, the product of the peak computing power and the tensor parallel number is determined as the tensor parallel computing power.

[0300] Tensor parallel computing power refers to the maximum number of floating-point operations that a processing unit can perform per second under tensor parallelism. If the peak computing power is TOPS and the tensor parallel number is TP, then the tensor parallel computing power is TOPS*TP.

[0301] In step 2120, the sample model calculation duration is determined based on the ratio of the sample model calculation amount and the tensor parallel computing power.

[0302] The calculation time of the sample model is the ratio of the sample model calculation amount to the tensor parallel computing power, which can be specifically expressed as MACs / (TOPS*TP), where MACs is the sample model calculation amount.

[0303] The embodiments of step 2110 and step 2120 above first calculate the product of the peak computing power and the tensor parallel number, and use the product as the tensor parallel computing power, and then determine the sample model calculation time based on the sample model calculation amount and the tensor parallel computing power. This method accurately obtains the time required to run a sample learning model, thereby improving the accuracy of the inference time.

[0304] In step 2040, the sample model memory access duration is determined based on the sample parameter quantity, the sample maximum key-value cache size, the video memory bandwidth, and the number of tensor parallelism in the sample deployment solution node.

[0305] In one embodiment, referring to Fig. 22 , step 2040 includes:

[0306] Step 2210: Determine the product of the video memory bandwidth and the tensor parallel number as the tensor parallel bandwidth;

[0307] Step 2220: Determine the sample model memory access duration based on the sample first sum of the sample parameter quantity and the sample maximum key value cache size and the tensor parallel bandwidth.

[0308] Step 2210 and step 2220 are described in detail below.

[0309] In step 2210, the product of the video memory bandwidth and the tensor parallel number is determined as the tensor parallel bandwidth.

[0310] Tensor parallel bandwidth refers to the data transfer rate between the processing unit and the video memory under tensor parallelism. If the video memory bandwidth is BW memory , the tensor parallel number is TP, then the tensor parallel bandwidth is BW memory *TP.

[0311] In step 2220, the sample model memory access duration is determined based on the sample first sum of the sample parameter quantity and the sample maximum key value cache size and the tensor parallel bandwidth.

[0312] The sample model access time can be expressed as Size(params+kv cache ) / (BW memory *TP), where the sample parameter amount and the sample maximum key value cache size are the first sum of params+kv cache , B.W. memory *TP is tensor parallel bandwidth.

[0313] The embodiments of step 2210 and step 2220 above first convert the product of video memory bandwidth and tensor parallel number into tensor parallel bandwidth, then calculate the sample first sum of sample parameter quantity and sample maximum key value cache size, and determine the ratio of sample first sum to tensor parallel bandwidth as sample model memory access time. This method accurately obtains the time required to access video memory during sample learning model inference, thereby improving the accuracy of inference time.

[0314] In step 2050, the communication duration of the sample model is determined based on the sample model configuration parameters, the communication bandwidth, and the tensor parallelism and pipeline parallelism in the sample deployment solution node.

[0315] In one embodiment, the sample model configuration parameter is the first product of the sample model dimension, the sample batch size, the number of sample model layers, and the sample sentence length of the sample learning model. Fig.23 , step 2050 includes:

[0316] Step 2310: Determine the pipeline parallel communication volume based on the sample model parameters;

[0317] Step 2320, determining the tensor parallel communication volume based on the sample model parameters and the tensor parallel number in the sample deployment solution node;

[0318] Step 2330: Determine the duration of the pipeline parallel communication based on the pipeline parallel communication volume and the communication bandwidth;

[0319] Step 2340: Determine the tensor parallel communication duration based on the tensor parallel communication volume and the second product of the communication bandwidth and the tensor parallel number;

[0320] Step 2350: determine the second sum of the pipeline parallel communication duration and the tensor parallel communication duration as the sample model communication duration.

[0321] It should be noted that the sample model configuration parameter is the first product of the sample model dimension, sample batch size, number of sample model layers, and sample sentence length of the sample learning model. The sample model dimension, sample batch size, and number of sample model layers are determined based on the sample learning model, and the sample sentence length is the length of the sentence that the sample learning model needs to process. The sample model configuration parameter can be expressed as b*s*h*l, where b is the batch size, s is the sample sentence length, h is the sample model dimension, and l is the number of sample model layers.

[0322] Step 2310 and step 2350 are described in detail below.

[0323] In step 2310, the amount of pipeline parallel communication is determined based on the sample model parameters.

[0324] The pipeline parallel communication volume is the amount of data that needs to be transmitted between processing units under pipeline parallelism. The pipeline parallel communication volume is determined based on the sample model parameters and can be expressed as Traffic pp =b*s*h*l.

[0325] In step 2320, the amount of tensor parallel communication is determined based on the sample model parameters and the number of tensor parallels in the sample deployment solution nodes.

[0326] The tensor parallel communication volume is the amount of data that needs to be transmitted between processing units under tensor parallelism. The tensor parallel communication volume is determined based on the sample model parameters and the tensor parallel number, which can be expressed as:

[0327]

[0328] Among them, Traffic tp is the amount of tensor parallel communication.

[0329] In step 2330, the duration of the pipeline parallel communication is determined based on the pipeline parallel communication volume and the communication bandwidth.

[0330] The pipeline parallel communication duration is the time required for communication between processing units under pipeline parallelism. The pipeline parallel communication duration is the ratio of the pipeline parallel communication volume to the communication bandwidth. The pipeline parallel communication duration can be expressed as Traffic pp / BW commu , where Traffic pp is the pipeline parallel communication volume, BW commu is the communication bandwidth.

[0331] In step 2340 , the tensor parallel communication duration is determined based on the tensor parallel communication volume and the second product of the communication bandwidth and the tensor parallel number.

[0332] The tensor parallel communication duration is the duration required for communication between processing units under tensor parallelism. The tensor parallel communication duration is determined based on the tensor parallel communication volume, communication bandwidth, and tensor parallel number. The tensor parallel communication duration can be expressed as Traffic tp / (BW commu *TP).

[0333] In step 2350, the second sum of the pipeline parallel communication duration and the tensor parallel communication duration is determined as the sample model communication duration.

[0334] The communication duration of the sample model is the second sum of the pipeline parallel communication duration and the tensor parallel communication duration. tp / (BW commu *TP), the duration of pipeline parallel communication is Traffic pp / BW commu , then the communication duration of the sample model can be expressed as:

[0335] commu time =Traffic tp / (BW commu *TP)+Traffic pp / BW commu .

[0336] The embodiment of the above steps 2310 to 2350 first calculates the pipeline parallel communication volume and the tensor parallel communication volume, then calculates the pipeline parallel communication duration and the tensor parallel communication duration, and determines the second sum of the pipeline parallel communication duration and the tensor parallel communication duration as the sample model communication duration. This method accurately obtains the time required for communication between processing units during the sample learning model inference process, thereby improving the accuracy of the inference duration.

[0337] In step 1630, based on the direct strategy function and the indirect strategy function, a sample jump strategy is selected between jumping to any candidate sample deployment solution node and not jumping.

[0338] In one embodiment, referring to Fig.24 , step 1630 includes:

[0339] Step 2410: Calculate the reward value based on the consistency between the candidate sample deployment solution node and the label deployment solution node to which the sample jump strategy reaches;

[0340] Step 2420: Determine a direct strategy function and an indirect strategy function based on the reward value;

[0341] Step 2430, calculating the third sum of the sample jump strategy of jumping to any candidate sample deployment solution node and the sample jump strategy of not jumping, the third sum being the sum of the direct strategy function and the indirect strategy function;

[0342] Step 2440: Determine the third and largest sample jump strategy as the selected sample jump strategy.

[0343] Steps 2410 to 2440 are described in detail below.

[0344] In step 2410, the reward value is calculated based on the consistency between the candidate sample deployment solution node and the label deployment solution node to which the sample jump strategy reaches.

[0345] The reward value can also be called a reward function, which is used to measure the rewards and penalties obtained after executing the sample jump strategy. Based on the current state and reward value, the optimal action, that is, the sample jump strategy, can be selected to maximize the cumulative reward value.

[0346] When the sample jump strategy reaches the candidate sample deployment solution node and the label deployment solution node, a reward value is given, otherwise it is 0, so that the sample jump strategy can reach the node with the largest reward value, that is, the label deployment solution node.

[0347] In step 2420, based on the reward value, a direct policy function and an indirect policy function are determined.

[0348] After determining the reward value, the direct strategy function and the indirect strategy function can be further determined based on the current state, the actions required to reach each candidate sample deployment solution node, and the path passed.

[0349] In step 2430, the third sum of the sample jump strategy of jumping to any candidate sample deployment solution node and the sample jump strategy of not jumping is calculated, and the third sum is the sum of the direct strategy function and the indirect strategy function.

[0350] The third sum is the sum of the direct policy function and the indirect policy function. Assuming that the direct policy function of jumping to the candidate sample deployment solution node 1 is 0.4 and the indirect policy function is 0.2, then the third sum is 0.6.

[0351] In step 2440, the third and largest sample skipping strategy is determined as the selected sample skipping strategy.

[0352] The larger the value of the third sum is, the greater the probability of selecting the sample jump strategy corresponding to the third sum is. Therefore, the sample jump strategy with the largest third sum is determined as the selected jump strategy.

[0353] The embodiments of the above steps 2410 and 2440 calculate the reward value based on the consistency between the candidate sample deployment solution node and the label deployment solution node to which the sample jump strategy reaches, so as to determine the direct strategy function and the indirect strategy function of each candidate sample deployment solution node. Afterwards, the third and largest sample jump strategy of the direct strategy function and the indirect strategy function is determined as the selected sample jump strategy. The purpose of setting the reward value and the direct strategy function and the indirect strategy function is to make the predicted sample deployment solution node to which the sample jump strategy reaches the label deployment solution node. When there is inconsistency, the direct strategy function and the indirect strategy function are adjusted to make the predicted deployment solution node consistent with the label deployment solution node, thereby improving the processing performance of the reinforcement learning model and improving the deployment efficiency.

[0354] In step 2410, the reward value is calculated based on the consistency between the candidate sample deployment solution node and the label deployment solution node to which the sample jump strategy reaches.

[0355] In one embodiment, referring to Fig.25 , step 2410 includes:

[0356] Step 2510: If the candidate sample deployment solution node reached by the sample jump strategy is consistent with the label deployment solution node, the reward value is determined based on the inverse of the number of paths from the sample initial deployment solution node to the candidate sample deployment solution node in the sample jump strategy, and the two adjacent sample solution nodes are connected by a single path;

[0357] Step 2520: If the candidate sample deployment solution node to which the sample jump strategy reaches is inconsistent with the label deployment solution node, the reward value is set to 0.

[0358] Step 2510 and step 2520 are described in detail below.

[0359] In step 2510, if the candidate sample deployment solution node reached by the sample jump strategy is consistent with the label deployment solution node, the reward value is determined based on the inverse of the number of paths from the sample initial deployment solution node to the candidate sample deployment solution node in the sample jump strategy, and the two adjacent sample solution nodes are connected by a single path.

[0360] When the candidate sample deployment solution node reached by the sample jump strategy is consistent with the label deployment solution node, the reward function is not 0, ensuring that the sample jump strategy can reach the label deployment solution node.

[0361] In addition, the reward value is determined based on the inverse of the number of paths from the sample initial deployment solution node to the candidate sample deployment solution node in the sample jump strategy, so that the reward value is inversely proportional to the number of paths. Then the fewer the number of paths to reach the label deployment solution node, the shorter the time, and the greater the reward value.

[0362] In step 2520, if the candidate sample deployment solution node to which the sample jump strategy reaches is inconsistent with the label deployment solution node, the reward value is set to 0.

[0363] When the sample deployment solution node to which the sample jump strategy reaches is inconsistent with the label deployment solution node, the reward value is set to 0 so that there is a label deployment solution node among multiple candidate sample deployment solution nodes, which can reduce the possibility of jumping to nodes other than the label deployment solution node.

[0364] It should be noted that the reward value of the sample deployment solution node can be expressed as

[0365]

[0366] Among them, P is the reward value, and length(path) is the number of paths from the sample initial deployment plan node to the candidate sample deployment plan node in the sample jump strategy. is the candidate sample deployment solution node to which the sample jump strategy is directed, n T Deploy the solution node for the label.

[0367] Reference Fig.17 , when the candidate sample deployment solution node reached by the sample jump strategy is not a node with a pipeline parallelism of 5 and a tensor parallelism of 3, the reward value is 0. Assuming that the candidate sample deployment solution node reached by the sample jump strategy is a node with a pipeline parallelism of 5 and a tensor parallelism of 3, there are three jump strategies for jumping from the sample initial deployment solution node to the label deployment solution node. The number of paths passed by the three jump strategies are 1, 3, and 4, respectively, and the corresponding reward values ​​are 2, 4 / 3, and 5 / 4, respectively. Then the jump strategy corresponding to the path number of 1 is taken as the final jump strategy.

[0368] In the embodiments of the above steps 2510 and 2520, when the candidate sample deployment solution node reached by the sample jump strategy is inconsistent with the label deployment solution node, the reward value is set to 0. When the candidate sample deployment solution node reached by the sample jump strategy is consistent with the label deployment solution node, the reward value is determined based on the inverse of the number of paths passed from the sample initial deployment solution node to the candidate sample deployment solution node in the sample jump strategy. Therefore, the fewer the number of paths to reach the label deployment solution node, the shorter the time, and the greater the reward value, so that the trained reinforcement learning model can output the target deployment solution node with the minimized inference time in the shortest possible time, thereby improving the deployment efficiency.

[0369] It is to be understood that, although the steps in the above-mentioned flowcharts are sequentially displayed according to the characterization of arrows, these steps are not necessarily executed in sequence according to the order of arrow characterization. Unless there is a clear description in the present embodiment, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the above-mentioned flowcharts can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0370] In one embodiment, the inference performance is a throughput performance indicator, referring to Fig.26 , step 1610 includes:

[0371] Step 2610: for each sample solution node jump relationship graph, determine a first score based on the throughput duration of the sample deployment solution node;

[0372] Step 2620: Determine a second score based on the unit time throughput of the deployment solution node;

[0373] Step 2630: Perform a weighted sum of the first score and the second score to obtain a throughput performance indicator of the sample deployment solution node;

[0374] Step 2640: determine the sample deployment solution node with the largest throughput performance index as the label deployment solution node.

[0375] Steps 2610 to 2640 are described in detail below.

[0376] In step 2610, for each sample solution node jump relationship graph, a first score is determined based on the throughput duration of the sample deployment solution node.

[0377] Throughput time refers to the time required to complete a task through a sample model. The first score is inversely proportional to the throughput time. The smaller the throughput time, the larger the first score.

[0378] In step 2620, a second score is determined based on the throughput per unit time of the deployment solution nodes.

[0379] The throughput per unit time refers to the number of inferences performed by the sample model per unit time. The second score is proportional to the throughput per unit time. The larger the throughput per unit time, the larger the second score.

[0380] In step 2630, a weighted sum is performed on the first score and the second score to obtain a throughput performance indicator of the sample deployment solution node.

[0381] The throughput performance index of the sample deployment solution node is the weighted sum of the first score and the second score. First, the weight values ​​corresponding to the first score and the second score are obtained. The weight values ​​can be set as needed, and the sum of the weight values ​​of the first score and the second score is 1. Then, the first score and the second score are weighted and summed according to the obtained weight values ​​to obtain the throughput performance index of the sample deployment solution node.

[0382] In step 2640, the sample deployment solution node with the largest throughput performance index is determined as the label deployment solution node.

[0383] After determining the throughput performance indicators of multiple sample deployment solution nodes in the sample solution node jump relationship diagram, the sample deployment solution node with the largest throughput performance indicator is determined as the label deployment solution node.

[0384] It should be noted that the reasoning performance can also be only the throughput duration or the throughput per unit time. When the reasoning performance is determined only based on the throughput duration, the label deployment solution node is the node with the smallest throughput duration in the sample solution node jump relationship graph. When the reasoning performance is determined only based on the throughput per unit time, the label deployment solution node is the node with the largest throughput per unit time in the sample solution node jump relationship graph.

[0385] The above-mentioned embodiment of step 2610 to step 2640 determines the throughput performance index based on the first score corresponding to the throughput duration and the second score corresponding to the throughput per unit time, and then determines the sample deployment solution node with the largest throughput performance index as the label deployment solution node so that the inference performance of the obtained label deployment solution node pair is optimal. In addition, the inference performance is determined by multiple methods such as inference duration and throughput performance index, which improves the applicability of the hardware deployment method.

[0386] Device description of the present disclosure

[0387] Fig. 272 is a schematic diagram of a hardware deployment device 2700 for a learning model provided in an embodiment of the present disclosure. The hardware deployment device 2700 for a learning model includes:

[0388] A first acquisition unit 2710 is used to acquire an initial pipeline parallelism number and an initial tensor parallelism number for processing unit deployment of a target learning model;

[0389] A positioning unit 2720 is used to locate the initial deployment scheme node where the initial pipeline parallelism number and the initial tensor parallelism number are located on the deployment scheme node jump relationship graph, wherein each deployment scheme node in the deployment scheme node jump relationship graph includes the pipeline parallelism number and the tensor parallelism number;

[0390] An input unit 2730 is used to input the deployment solution node jump relationship graph and the initial deployment solution node into the reinforcement learning model to obtain a target deployment solution node that optimizes the reasoning performance of the target learning model, wherein the target deployment solution node includes a target pipeline parallel number and a target tensor parallel number, wherein the reinforcement learning model is trained by specifying a label deployment solution node with the best reasoning performance by jumping a relationship graph for each sample solution node in a plurality of sample solution node jump relationship graphs, and starting from the sample initial deployment solution node of the sample solution node jump relationship graph, jumping according to the principle of maximizing the reward value until jumping to the label deployment solution node;

[0391] The first deployment unit 2740 is used to deploy processing units for the target learning model based on the target pipeline parallelism and the target tensor parallelism.

[0392] Optionally, the first acquisition unit 2710 is specifically used to: determine the storage capacity occupied by a single-layer processing of the target learning model; obtain the maximum storage capacity of the processing unit; determine the greatest common divisor of the storage capacity occupied by the single-layer processing and the maximum storage capacity; based on the greatest common divisor, determine the initial pipeline parallel number and the initial tensor parallel number.

[0393] Optionally, the first acquisition unit 2710 is also specifically used to: obtain the parameter quantity of the target learning model; obtain the maximum key-value cache size used in the inference of the target learning model; and determine the storage capacity occupied by a single-layer processing based on the first sum of the parameter quantity and the maximum key-value cache size and the number of layers of the target learning model.

[0394] Optionally, the first acquisition unit 2710 is further specifically used to: determine the initial tensor parallel number based on the storage capacity occupied by single-layer processing and the greatest common divisor; determine the initial pipeline parallel number based on the number of layers, maximum storage capacity, and greatest common divisor of the target learning model.

[0395] Optionally, the hardware deployment device 2700 of the learning model also includes: a generation unit, which is used to generate an anchor deployment scheme node, and the anchor deployment scheme node has an anchor pipeline parallel number and an anchor tensor parallel number; a connection unit, which is used to increase or decrease only one of the anchor pipeline parallel number and the anchor tensor parallel number of the anchor deployment scheme node, obtain a neighbor deployment scheme node of the anchor deployment scheme node, and connect the neighbor deployment scheme node and the anchor deployment scheme node; an update unit, which is used to update the anchor deployment scheme node with the neighbor deployment scheme node, and return to the step of increasing or decreasing only one of the anchor pipeline parallel number and the anchor tensor parallel number of the anchor deployment scheme node, until the number of returns reaches a predetermined number of times.

[0396] Optionally, the input unit 2720 is specifically used to: input the deployment solution node jump relationship graph and the initial deployment solution node into the reinforcement learning model, so that the reinforcement learning model selects a candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units in the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph; based on the direct strategy function and the indirect strategy function, select a jump strategy between jumping to any candidate deployment solution node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the deployment solution node after the candidate deployment solution node; when the selected jump strategy is to jump to a candidate deployment solution node, the candidate deployment solution node jumped to is used as the initial deployment solution node, and the step of returning to the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph to select a candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units, until the selected jump strategy is not to jump, and the initial deployment solution node when not jumping is selected is used as the target deployment solution node.

[0397] Optionally, the input unit 2720 is also specifically used to: calculate the third sum of the jump strategy of jumping to any candidate deployment solution node and the jump strategy of not jumping, the third sum being the sum of the direct strategy function and the indirect strategy function; and determine the jump strategy with the largest third sum as the selected jump strategy.

[0398] Optionally, the input unit 2720 is also specifically used to: for a single adjacent node adjacent to the initial deployment plan node, determine the number of processing unit occupancy of the single adjacent node based on the pipeline parallelism and tensor parallelism in the single adjacent node; if the number of processing unit occupancy is between the minimum number of processing units and the maximum number of processing units, use the single adjacent node as a candidate deployment plan node.

[0399] Optionally, the hardware deployment device 2700 for the learning model also includes: a second acquisition unit, used to obtain the parameter quantity of the target learning model; a third acquisition unit, used to obtain the maximum storage capacity of the processing unit; and a first determination unit, used to determine the minimum number of processing units based on the parameter quantity and the maximum storage capacity.

[0400] Optionally, the hardware deployment device 2700 of the learning model also includes: a second determination unit, which determines the label deployment solution node with the best reasoning performance for each sample solution node jump relationship diagram in multiple sample solution node jump relationship diagrams; a screening unit, which is used to select candidate sample deployment solution nodes that meet the minimum number of processing units and the maximum number of processing units in the nodes adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram; a selection unit, which is used to select a sample jump strategy between jumping to any candidate sample deployment solution node and not jumping based on a direct strategy function and an indirect strategy function, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate sample deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the candidate sample deployment solution node. a return unit, which is used to, when the selected sample jump strategy is to jump to a candidate sample deployment solution node, use the candidate sample deployment solution node to which it jumps as the sample initial deployment solution node, and return to the nodes adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram, to select the sample deployment solution node that meets the minimum number of processing units and the maximum number of processing units, until the selected jump strategy is not to jump, and use the sample initial deployment solution node when not to jump is selected as the predicted deployment solution node; an adjustment unit, which is used to adjust the direct strategy function and the indirect strategy function if the predicted deployment solution node is inconsistent with the label deployment solution node, until the predicted deployment solution node is consistent with the label deployment solution node.

[0401] Optionally, the second determination unit is specifically used to: obtain the pipeline parallelism and tensor parallelism in each sample deployment scheme node in the single sample scheme node jump relationship diagram for a single sample scheme node; determine the inference time of the sample deployment scheme node based on the pipeline parallelism and tensor parallelism in the sample deployment scheme node; and determine the sample deployment scheme node with the shortest inference time as the label deployment scheme node.

[0402] Optionally, the second determination unit is also specifically used to: obtain the sample model calculation amount, sample parameter amount, sample maximum key-value cache size and sample model configuration parameters of the sample learning model, the sample learning model is the model applied by the sample deployment scheme node; obtain the peak computing power, memory bandwidth and communication bandwidth of the processing unit; determine the sample model calculation time based on the sample model calculation amount, peak computing power and the number of tensor parallelism in the sample deployment scheme node; determine the sample model memory access time based on the sample parameter amount, sample maximum key-value cache size, memory bandwidth and the number of tensor parallelism in the sample deployment scheme node; determine the sample model communication time based on the sample model configuration parameters, communication bandwidth, and the number of tensor parallelism and pipeline parallelism in the sample deployment scheme node; determine the inference time of the sample deployment scheme node based on the maximum value of the sample model calculation time and the sample model memory access time, and the sample model communication time.

[0403] Optionally, the second determination unit is also specifically used to: determine the sample model calculation time based on the sample model calculation amount, peak computing power and the number of tensor parallelism in the sample deployment plan node, including: determining the product of the peak computing power and the tensor parallelism as the tensor parallel computing power; determine the sample model calculation time based on the ratio of the sample model calculation amount and the tensor parallel computing power.

[0404] Optionally, the second determination unit is further specifically used to: determine the product of the video memory bandwidth and the tensor parallel number as the tensor parallel bandwidth; determine the sample model memory access duration based on the sample parameter quantity and the sample first sum of the sample maximum key value cache size and the tensor parallel bandwidth.

[0405] Optionally, the sample model configuration parameters are the first product of the sample model dimension, sample batch size, number of sample model layers, and sample sentence length of the sample learning model, and the second determination unit is specifically used to: determine the pipeline parallel communication volume based on the sample model parameters; determine the tensor parallel communication volume based on the sample model parameters and the number of tensor parallels in the sample deployment scheme node; determine the pipeline parallel communication duration based on the pipeline parallel communication volume and the communication bandwidth; determine the tensor parallel communication duration based on the tensor parallel communication volume and the second product of the communication bandwidth and the number of tensor parallels; and determine the second sum of the pipeline parallel communication duration and the tensor parallel communication duration as the sample model communication duration.

[0406] Optionally, the selection unit is specifically used to: calculate a reward value based on the consistency between the candidate sample deployment scheme node and the label deployment scheme node to which the sample jump strategy reaches; determine a direct strategy function and an indirect strategy function based on the reward value; calculate the third sum of the jump strategy of jumping to any candidate sample deployment scheme node and the jump strategy of not jumping, the third sum being the sum of the direct strategy function and the indirect strategy function; and determine the jump strategy with the largest third sum as the selected jump strategy.

[0407] Optionally, the selection unit is also specifically used to: if the candidate sample deployment plan node reached by the sample jump strategy is consistent with the label deployment plan node, determine the reward value based on the inverse of the number of paths passed from the sample initial deployment plan node to the candidate sample deployment plan node in the sample jump strategy, and connect two adjacent sample plan nodes through a single path; if the candidate sample deployment plan node reached by the sample jump strategy is inconsistent with the label deployment plan node, set the reward value to 0.

[0408] Optionally, the second determination unit is also specifically used to: determine a first score based on the throughput duration of the sample deployment scheme node for each sample scheme node jump relationship diagram; determine a second score based on the unit time throughput of the deployment scheme node; perform weighted summation of the first score and the second score to obtain the throughput performance index of the sample deployment scheme node; and determine the sample deployment scheme node with the largest throughput performance index as the label deployment scheme node.

[0409] Reference Fig.28 , Fig.28 The structural block diagram of the terminal part of the hardware deployment method for the learning model of the embodiment of the present disclosure is as follows: the terminal includes: a radio frequency (RF) circuit 2810, a memory 2815, an input unit 2830, a display unit 2840, a sensor 2850, an audio circuit 2860, a wireless fidelity (WiFi) module 2870, a processor 2880, and a power supply 2890. Those skilled in the art can understand that Fig.28 The terminal structure shown does not constitute a limitation on the mobile phone or computer, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0410] The RF circuit 2810 may be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information from the base station, it is sent to the processor 2880 for processing; in addition, the designed uplink data is sent to the base station.

[0411] The memory 2815 may be used to store software programs and modules. The processor 2880 executes various functional applications and data processing of the content terminal by running the software programs and modules stored in the memory 2815 .

[0412] The input unit 2830 may be used to receive input digital or character information and generate key signal input related to the setting and function control of the content terminal. Specifically, the input unit 2830 may include a touch panel 2831 and other input devices 2832.

[0413] The display unit 2840 may be used to display input information or provided information and various menus of the content terminal. The display unit 2840 may include a display panel 2841.

[0414] The audio circuit 2860, the speaker 2861, and the microphone 2862 may provide an audio interface.

[0415] In this embodiment, the processor 2880 included in the terminal can execute the hardware deployment method of the learning model in the previous embodiment.

[0416] The terminals of the embodiments of the present disclosure include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. The embodiments of the present invention can be applied to various scenarios, including but not limited to data collection, data export, data analysis, etc.

[0417] Fig.29 A block diagram of the structure of a portion of a server 140 for implementing the hardware deployment method of the learning model of the embodiment of the present disclosure. The server 140 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 2922 (e.g., one or more processors) and a memory 2932, and one or more storage media 2930 (e.g., one or more mass storage devices) storing application programs 2942 or data 2944. Among them, the memory 2932 and the storage medium 2930 may be temporary storage or permanent storage. The program stored in the storage medium 2930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server 140. Furthermore, the central processing unit 2922 may be configured to communicate with the storage medium 2930 and execute a series of instruction operations in the storage medium 2930 on the server 140.

[0418] The server 140 may also include one or more power supplies 2926, one or more wired or wireless network interfaces 2950, ​​one or more input and output interfaces 2958, and / or one or more operating systems 2941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0419] The central processor 2922 in the server 140 can be used to execute the hardware deployment method of the learning model of the embodiment of the present disclosure.

[0420] The embodiments of the present disclosure also provide a computer-readable storage medium, which is used to store program code, and the program code is used to execute the hardware deployment method of the learning model of each of the aforementioned embodiments.

[0421] The embodiment of the present disclosure also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, so that the computer device executes the hardware deployment method for implementing the above-mentioned learning model.

[0422] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar contents, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0423] It should be understood that in the present disclosure, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated content, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated content before and after is in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0424] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0425] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0426] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0427] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0428] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server 140, or network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0429] It should also be understood that the various implementations provided in the embodiments of the present disclosure can be combined arbitrarily to achieve different technical effects.

[0430] The above is a specific description of the implementation methods of the present disclosure, but the present disclosure is not limited to the above implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A hardware deployment method for a learning model, characterized in that: include: Obtain the initial pipeline parallelism and initial tensor parallelism of the processing unit deployment for the target learning model; On the deployment scheme node jump relationship graph, locate the initial deployment scheme node where the initial pipeline parallel number and the initial tensor parallel number are located, wherein each deployment scheme node in the deployment scheme node jump relationship graph includes a pipeline parallel number and a tensor parallel number; The deployment solution node jump relationship graph and the initial deployment solution node are input into a reinforcement learning model to obtain a target deployment solution node that optimizes the reasoning performance of the target learning model, wherein the target deployment solution node includes a target pipeline parallel number and a target tensor parallel number, wherein the reinforcement learning model specifies a label deployment solution node with the best reasoning performance by jumping a relationship graph for each sample solution node in a plurality of sample solution node jump relationship graphs, and starts from the sample initial deployment solution node of the sample solution node jump relationship graph, jumping according to the principle of maximizing the reward value until jumping to the label deployment solution node for training; Deploy a processing unit for the target learning model based on the target pipeline parallelism number and the target tensor parallelism number; The deployment scheme node jump relationship diagram is generated in the following way: Generate an anchor deployment solution node, wherein the anchor deployment solution node has an anchor pipeline parallel number and an anchor tensor parallel number; Only one of the anchor pipeline parallelism number and the anchor tensor parallelism number of the anchor deployment solution node is increased or decreased to obtain a neighbor deployment solution node of the anchor deployment solution node, and the neighbor deployment solution node and the anchor deployment solution node are connected; The anchor deployment scheme node is updated with the neighbor deployment scheme node, and the step of increasing or decreasing only one of the anchor pipeline parallelism number and the anchor tensor parallelism number for the anchor deployment scheme node is returned until the number of returns reaches a predetermined number of times.

2. The hardware deployment method according to claim 1, characterized in that: The obtaining of the initial pipeline parallelism and initial tensor parallelism for the processing unit deployment of the target learning model includes: Determining the storage capacity occupied by a single layer of processing of the target learning model; Obtaining a maximum storage capacity of the processing unit; Determine the greatest common divisor of the storage capacity occupied by the single-layer processing and the maximum storage capacity; Based on the greatest common divisor, the initial pipeline parallelism number and the initial tensor parallelism number are determined.

3. The hardware deployment method according to claim 2, characterized in that: The determining of the storage capacity occupied by a single-layer processing of the target learning model includes: Obtaining the parameter quantity of the target learning model; Obtaining a maximum key-value cache size used in inference of the target learning model; The storage capacity occupied by the single-layer processing is determined based on the first sum of the parameter quantity and the maximum key-value cache size and the number of layers of the target learning model.

4. The hardware deployment method according to claim 2, characterized in that: The determining the initial pipeline parallel number and the initial tensor parallel number based on the greatest common divisor includes: Determining the initial tensor parallel number based on the storage capacity occupied by the single-layer processing and the greatest common divisor; The initial pipeline parallel number is determined based on the number of layers of the target learning model, the maximum storage capacity, and the greatest common divisor.

5. The hardware deployment method according to claim 1, characterized in that: The step of inputting the deployment solution node jump relationship diagram and the initial deployment solution node into a reinforcement learning model to obtain a target deployment solution node that optimizes the reasoning performance of the target learning model includes: Inputting the deployment solution node jump relationship graph and the initial deployment solution node into a reinforcement learning model, so that the reinforcement learning model selects a candidate deployment solution node that meets the minimum number of processing units and the maximum number of processing units restrictions from the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph; Based on a direct strategy function and an indirect strategy function, a jump strategy is selected between jumping to any candidate deployment solution node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the deployment solution node after the candidate deployment solution node; When the selected jump strategy is to jump to a candidate deployment solution node, the candidate deployment solution node jumped to is used as the initial deployment solution node, and the process returns to the step of selecting candidate deployment solution nodes that meet the minimum number of processing units and the maximum number of processing units restrictions from the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship diagram, until the selected jump strategy is not to jump, and the initial deployment solution node when not to jump is selected is used as the target deployment solution node.

6. The hardware deployment method according to claim 5, characterized in that: The selecting a jump strategy between jumping to any candidate deployment solution node and not jumping based on the direct strategy function and the indirect strategy function includes: Calculate the third sum of the jump strategy of jumping to any candidate deployment solution node and the jump strategy of not jumping, respectively, where the third sum is the sum of the direct strategy function and the indirect strategy function; The third and largest jump strategy is determined as the selected jump strategy.

7. The hardware deployment method according to claim 5, characterized in that: The selecting, from the nodes adjacent to the initial deployment solution node in the deployment solution node jump relationship graph, candidate deployment solution nodes that meet the minimum number of processing units and the maximum number of processing units restrictions, includes: For a single adjacent node adjacent to the initial deployment solution node, based on the pipeline parallelism and the tensor parallelism in the single adjacent node, determine the number of occupied processing units of the single adjacent node; If the number of occupied processing units is between the minimum number of processing units and the maximum number of processing units, the single adjacent node is used as the candidate deployment solution node.

8. The hardware deployment method according to claim 5, characterized in that: The minimum number of processing units is determined by: Obtaining the parameter quantity of the target learning model; Obtaining a maximum storage capacity of the processing unit; Based on the parameter quantity and the maximum storage capacity, the minimum number of processing units is determined.

9. The hardware deployment method according to claim 1, characterized in that: The reinforcement learning model is trained through the following process: For each sample solution node jump relationship graph in a plurality of sample solution node jump relationship graphs, determining the label deployment solution node with the best reasoning performance; Selecting a candidate sample deployment solution node that meets the minimum number of processing units and the maximum number of processing units restrictions from among the nodes adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram; Based on the direct strategy function and the indirect strategy function, a sample jump strategy is selected between jumping to any candidate sample deployment solution node and not jumping, wherein the direct strategy function reflects the reward value that can be obtained by jumping to the candidate sample deployment solution node itself, and the indirect strategy function reflects the reward value that can be obtained by jumping to the sample deployment solution node after the candidate sample deployment solution node; When the selected sample jump strategy is to jump to a candidate sample deployment solution node, the candidate sample deployment solution node to which the sample is jumped is used as the sample initial deployment solution node, and the step of returning to the node adjacent to the sample initial deployment solution node in the sample solution node jump relationship diagram to select a sample deployment solution node that meets the minimum number of processing units and the maximum number of processing units, until the selected jump strategy is not to jump, and the sample initial deployment solution node when not to jump is selected is used as the predicted deployment solution node; If the predicted deployment solution node is inconsistent with the label deployment solution node, adjust the direct strategy function and the indirect strategy function until the predicted deployment solution node is consistent with the label deployment solution node.

10. The hardware deployment method according to claim 9, characterized in that: The reasoning performance is the reasoning time, and the jump relationship diagram of each sample solution node in the multiple sample solution node jump relationship diagrams is used to determine the label deployment solution node with the best reasoning performance, including: For a single sample solution node jump relationship graph, obtain the pipeline parallel number and the tensor parallel number in each sample deployment solution node in the single sample solution node jump relationship graph; Determine the inference duration of the sample deployment solution node based on the pipeline parallelism number and the tensor parallelism number in the sample deployment solution node; The sample deployment solution node with the shortest inference time is determined as the label deployment solution node.

11. The hardware deployment method according to claim 10, characterized in that: The determining the inference duration of the sample deployment solution node based on the pipeline parallel number and the tensor parallel number in the sample deployment solution node includes: Obtaining a sample model calculation amount, a sample parameter amount, a sample maximum key value cache size, and a sample model configuration parameter of a sample learning model, wherein the sample learning model is a model applied by the sample deployment solution node; Obtaining the peak computing power, memory bandwidth, and communication bandwidth of the processing unit; Determine the sample model calculation duration based on the sample model calculation amount, the peak computing power, and the tensor parallel number in the sample deployment solution node; Determine the sample model memory access duration based on the sample parameter quantity, the sample maximum key value cache size, the video memory bandwidth, and the tensor parallel number in the sample deployment solution node; Determine the communication duration of the sample model based on the sample model configuration parameters, the communication bandwidth, and the tensor parallelism and the pipeline parallelism in the sample deployment solution node; The inference duration of the sample deployment solution node is determined based on the maximum value of the sample model calculation duration and the sample model memory access duration, and the sample model communication duration.

12. The hardware deployment method according to claim 11, characterized in that: The sample model configuration parameter is a first product of a sample model dimension, a sample batch size, a sample model layer number, and a sample sentence length of the sample learning model, and determining the sample model communication duration based on the sample model configuration parameter, the communication bandwidth, and the tensor parallel number and the pipeline parallel number in the sample deployment solution node includes: Based on the sample model parameters, determining the pipeline parallel communication volume; Determining tensor parallel communication volume based on the sample model parameters and the tensor parallel number in the sample deployment solution node; Determining the duration of the pipeline parallel communication based on the pipeline parallel communication volume and the communication bandwidth; Determine a tensor parallel communication duration based on the tensor parallel communication volume and a second product of the communication bandwidth and the tensor parallel number; The second sum of the pipeline parallel communication duration and the tensor parallel communication duration is determined as the sample model communication duration.

13. The hardware deployment method according to claim 9, characterized in that: The selecting of the sample jump strategy between jumping to any candidate sample deployment solution node and not jumping based on the direct strategy function and the indirect strategy function includes: Calculating the reward value based on the consistency between the candidate sample deployment solution node and the label deployment solution node to which the sample jump strategy reaches; Based on the reward value, determining the direct strategy function and the indirect strategy function; Calculate the third sum of the sample jump strategy of jumping to any candidate sample deployment solution node and the sample jump strategy of not jumping, respectively, where the third sum is the sum of the direct strategy function and the indirect strategy function; The third and largest sample skipping strategy is determined as the selected sample skipping strategy.

14. The hardware deployment method according to claim 13, characterized in that: The calculating the reward value based on the consistency between the candidate sample deployment solution node and the label deployment solution node to which the sample jump strategy is directed comprises: If the candidate sample deployment solution node reached by the sample jump strategy is consistent with the label deployment solution node, the reward value is determined based on the inverse of the number of paths from the sample initial deployment solution node to the candidate sample deployment solution node in the sample jump strategy, and two adjacent sample solution nodes are connected by a single path; If the candidate sample deployment solution node to which the sample jump strategy reaches is inconsistent with the label deployment solution node, the reward value is set to 0.

15. The hardware deployment method according to claim 10, characterized in that: The reasoning performance is a throughput performance indicator, and the reasoning performance is each sample solution node jump relationship diagram in a plurality of sample solution node jump relationship diagrams, and determining the label deployment solution node with the best reasoning performance includes: For each sample solution node jump relationship graph, determine a first score based on the throughput duration of the sample deployment solution node; Determining a second score based on the unit time throughput of the deployment solution node; Taking a weighted sum of the first score and the second score to obtain a throughput performance indicator of the node of the sample deployment solution; The sample deployment solution node with the largest throughput performance indicator is determined as the label deployment solution node.

16. A hardware deployment device for a learning model, characterized in that: include: A first acquisition unit, used to acquire an initial pipeline parallelism number and an initial tensor parallelism number for processing unit deployment of a target learning model; A positioning unit, used to locate the initial deployment scheme node where the initial pipeline parallel number and the initial tensor parallel number are located on the deployment scheme node jump relationship graph, wherein each deployment scheme node in the deployment scheme node jump relationship graph includes a pipeline parallel number and a tensor parallel number; An input unit, used for inputting the deployment solution node jump relationship graph and the initial deployment solution node into a reinforcement learning model, to obtain a target deployment solution node that optimizes the reasoning performance of the target learning model, wherein the target deployment solution node includes a target pipeline parallel number and a target tensor parallel number, wherein the reinforcement learning model is trained by specifying a label deployment solution node with the best reasoning performance for each sample solution node jump relationship graph in a plurality of sample solution node jump relationship graphs, and starting from the sample initial deployment solution node of the sample solution node jump relationship graph, jumping according to the principle of maximizing the reward value until jumping to the label deployment solution node; A first deployment unit, configured to deploy a processing unit for the target learning model based on the target pipeline parallelism number and the target tensor parallelism number; The deployment scheme node jump relationship diagram is generated in the following way: Generate an anchor deployment solution node, wherein the anchor deployment solution node has an anchor pipeline parallel number and an anchor tensor parallel number; Only one of the anchor pipeline parallelism number and the anchor tensor parallelism number of the anchor deployment solution node is increased or decreased to obtain a neighbor deployment solution node of the anchor deployment solution node, and the neighbor deployment solution node and the anchor deployment solution node are connected; The anchor deployment scheme node is updated with the neighbor deployment scheme node, and the step of increasing or decreasing only one of the anchor pipeline parallelism number and the anchor tensor parallelism number for the anchor deployment scheme node is returned until the number of returns reaches a predetermined number of times.

17. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the hardware deployment method of the learning model according to any one of claims 1 to 15 is implemented.

18. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the hardware deployment method of the learning model according to any one of claims 1 to 15 is implemented.

19. A computer program product, comprising a computer program, wherein the computer program is read and executed by a processor of a computer device, so that the computer device executes the hardware deployment method of the learning model according to any one of claims 1 to 15.

Images