Method and apparatus for pairing large language model based on hardware and model characteristics, terminal, medium and product

Abstract

The application provides a large language model pairing method and device based on hardware and model characteristics, a terminal, a medium and a product. The method comprises the following steps: obtaining at least one hardware platform and at least one large language model according to an obtained target pairing requirement; obtaining hardware characteristic parameters of each hardware platform, and enabling each large language model to execute a corresponding preset target task to obtain characteristic parameters of each large language model; based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each large language model, using a prediction function to predict key index parameters when each large language model is deployed on each hardware platform; and using an optimization objective function to determine a pairing result of the hardware platform and the large language model that meets the target pairing requirement based on the key index parameters when each large language model is deployed on each hardware platform. The application fully considers the characteristics of the hardware platform and the large language model, and ensures that the model can better match the hardware platform.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and in particular to a method, device, terminal, medium and product for pairing large language models based on hardware and model features. Background Technology

[0002] Large Language Models (LLMs) are a core technology in current artificial intelligence research, typically involving a massive number of parameters. LLMs come in various architectures, such as Autoregressive (AR) and Diffusion (DF) architectures. Even with similar parameter sizes, different LLM architectures can exhibit significant differences in computational and data access characteristics when performing specific tasks (such as text generation).

[0003] When deploying large language models on a specific hardware platform, selecting the appropriate model is crucial. Existing selection methods often rely on general benchmark results, parameter magnitude, or empirical trial and error, lacking quantitative analysis and systematic guidance on the deep adaptation relationship between specific hardware characteristics and LLMs with similar parameter sizes but different architectures. Furthermore, they fail to fully consider the matching relationship between the hardware's micro-characteristics and the computational and data handling volumes of different LLM architectures under specific tasks. At the same time, trial-and-error selection is time-consuming and labor-intensive, and cannot guarantee that the selected model will fully utilize the hardware's computational and storage resources. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, terminal, medium and product for large language model pairing based on hardware and model features, so as to solve the problem that the existing selection methods do not comprehensively consider the characteristics of hardware platform and the features of large language models.

[0005] To achieve the above and other related objectives, a first aspect of this application provides a method for pairing large language models based on hardware and model features, comprising: obtaining target pairing requirements, and obtaining at least one hardware platform and at least one large language model according to the target pairing requirements; obtaining hardware feature parameters of each hardware platform, and having each large language model execute a corresponding preset target task to obtain feature parameters of each large language model; predicting key indicator parameters of each large language model when deployed on each hardware platform using a prediction function based on the hardware feature parameters of each hardware platform and the feature parameters of each large language model; and determining the pairing result of the hardware platform and the large language model that meet the target pairing requirements using an optimization objective function based on the key indicator parameters of each large language model when deployed on each hardware platform.

[0006] In some embodiments of the first aspect of this application, the hardware characteristic parameters include: the effective computing power of the hardware platform, the effective memory bandwidth, the energy consumption per unit of computing operation, and the energy consumption per unit of memory access.

[0007] In some embodiments of the first aspect of this application, the characteristic parameters include: total number of floating-point operations and total data transfer volume.

[0008] In some embodiments of the first aspect of this application, the prediction function includes: a delay prediction function and an energy consumption prediction function.

[0009] In some embodiments of the first aspect of this application, based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each language model, a prediction function is used to predict the key indicator parameters of each language model deployed on each hardware platform, including: based on the effective computing power and effective memory bandwidth of each hardware platform, and based on the total number of floating-point operations and the total amount of data transported by each large language model, a latency prediction function is used to predict the execution time of each large language model deployed on each hardware platform; based on the unit energy consumption of computing operations and unit energy consumption of memory accesses of each hardware platform, and based on the total number of floating-point operations and the total amount of data transported by each large language model, an energy consumption prediction function is used to predict the total energy consumption of each large language model deployed on each hardware platform; and the execution time and total energy consumption of each large language model deployed on each hardware platform are output as key indicator parameters of each large language model deployed on each hardware platform.

[0010] In some embodiments of the first aspect of this application, an optimization objective function is used to determine the pairing results of hardware platforms and large language models that meet the target pairing requirements based on key indicator parameters when each language model is deployed on each hardware platform. This includes: calculating the optimization objective function value of each language model deployed on each hardware platform based on the key indicator parameters when each language model is deployed on each hardware platform; and determining the pairing results of hardware platforms and large language models that meet the target pairing requirements among each language model and each hardware platform based on the optimization objective function value of each language model deployed on each hardware platform and according to the pairing rules between language models and hardware platforms.

[0011] To achieve the above and other related objectives, a second aspect of this application provides a large language model pairing device based on hardware and model features, comprising: an acquisition module, configured to acquire target pairing requirements, and acquire at least one hardware platform and at least one large language model according to the target pairing requirements; a parameter acquisition module, configured to acquire hardware feature parameters of each hardware platform, and instruct each large language model to execute a corresponding preset target task to obtain feature parameters of each large language model; a prediction module, configured to predict key indicator parameters of each large language model when deployed on each hardware platform using a prediction function based on the hardware feature parameters of each hardware platform and the feature parameters of each large language model; and a selection module, configured to determine the pairing result of the hardware platform and the large language model that meets the target pairing requirements using an optimization objective function based on the key indicator parameters of each large language model when deployed on each hardware platform.

[0012] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large language model pairing method based on hardware and model features.

[0013] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the large language model pairing method based on hardware and model features.

[0014] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the large language model pairing method based on hardware and model features.

[0015] As described above, the large language model pairing method, apparatus, terminal, medium, and product based on hardware and model features of this application have the following beneficial effects:

[0016] This invention quantifies the characteristics of large language models and hardware platforms, and pairs them based on predicted key indicator parameters. This ensures that large language models can better match the computing and storage characteristics of hardware platforms, thereby maximizing hardware utilization and achieving a better performance-to-power ratio. Attached Figure Description

[0017] Figure 1 The diagram shown is a flowchart illustrating a large language model pairing method based on hardware and model features in one embodiment of this application.

[0018] Figure 2The diagram shown is a schematic block diagram of a large language model matching device based on hardware and model features according to an embodiment of this application.

[0019] Figure 3 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0020] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0021] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0022] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0023] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0024] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1This document illustrates a flowchart of a large language model pairing method based on hardware and model features, as described in an embodiment of the present invention. The large language model pairing method based on hardware and model features in this embodiment mainly includes the following steps:

[0025] Step S11: Obtain the target pairing requirements, and based on the target pairing requirements, obtain at least one hardware platform and at least one large language model.

[0026] Specifically, there are three types of target pairing requirements. The first type involves selecting the most suitable large language model for a target hardware platform from multiple candidate large language models. Under this type, a target hardware platform and multiple candidate large language models are obtained. The second type involves selecting the most suitable hardware platform for a large language model to perform its preset task from multiple hardware platforms. Under this type, the target large language model to perform the preset task is obtained, and multiple candidate hardware platforms are obtained. The third type involves optimal pairing of multiple hardware platforms and multiple large language models. Under this type, multiple target hardware platforms and multiple target large language models are obtained.

[0027] Step S12: Obtain the hardware feature parameters of each hardware platform, and have each language model execute the corresponding preset target task to obtain the feature parameters of each language model.

[0028] In one embodiment, the hardware characteristic parameters include: the effective computing power of the hardware platform, the effective memory bandwidth, the energy consumption per unit of computing operation, and the energy consumption per unit of memory access.

[0029] It should be understood that the effective computing power of a hardware platform refers to the real computing performance that the hardware can continuously provide and that can be effectively utilized by software in actual application scenarios. Effective memory bandwidth refers to the actual memory data transfer rate that the hardware platform can utilize during program execution. The energy consumption per unit of computational operation of a hardware platform refers to the energy consumed to complete a computational operation (such as a basic operation like addition or multiplication). The energy consumption per unit of memory access of a hardware platform refers to the energy required to complete a memory access operation (such as a read or write). The above hardware characteristic parameters are directly obtainable hardware platform parameters.

[0030] In one embodiment, the characteristic parameters include: the total number of floating-point operations and the total amount of data moved. It should be understood that the total number of floating-point operations refers to the total number of floating-point operations (i.e., operations involving addition, subtraction, multiplication, division, etc.) performed by a computing task or program during execution. The total amount of data moved refers to the total amount of data moved between different storage media (such as hard disk, memory, video memory, etc.) during model training, inference, or operation. It should be noted that the total number of floating-point operations and the total amount of data moved can be obtained using existing methods, which will not be elaborated upon here.

[0031] In one embodiment, each language model is instructed to perform a corresponding preset target task to obtain the characteristic parameters of each language model. The specific implementation process includes: each large language model has a preset target task based on its own parameter count and model architecture type. The target task includes specific task parameters. For example, for an autoregressive model, the preset target task includes task parameters such as: prompt pre-padding length and generated sequence length. For a diffusion model, the preset target task includes task parameters such as: total sequence length processed and number of diffusion steps. After each large language model completes its preset target task, the total number of floating-point operations and the total data transfer volume required for each large language model to complete its preset target task are obtained. It should be noted that the target task of each large language model can be set according to requirements, and this invention does not limit this.

[0032] Step S13: Based on the hardware feature parameters of each hardware platform and the characteristic parameters of each language model, use the prediction function to predict the key indicator parameters of each language model when deployed on each hardware platform.

[0033] In one embodiment, the prediction function includes a latency prediction function and an energy consumption prediction function. Based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each language model, the prediction function is used to predict the key indicator parameters of each language model deployed on each hardware platform. These include: based on the effective computing power and effective memory bandwidth of each hardware platform, and based on the total number of floating-point operations and the total data transfer volume of each large language model, the latency prediction function is used to predict the execution time of each large language model deployed on each hardware platform; based on the unit energy consumption of computing operations and unit energy consumption of memory accesses of each hardware platform, and based on the total number of floating-point operations and the total data transfer volume of each large language model, the energy consumption prediction function is used to predict the total energy consumption of each large language model deployed on each hardware platform; and the execution time and total energy consumption of each large language model deployed on each hardware platform are output as the key indicator parameters of each large language model deployed on each hardware platform.

[0034] Specifically, for M large language models and N hardware platforms (M and N are positive integers), using the delay prediction function, based on the total number of floating-point operations and the total amount of data transported by the i-th large language model, and based on the effective computing power and effective memory bandwidth of the j-th hardware platform, the execution time required for the i-th large language model to execute the preset target task on the j-th hardware platform is calculated (that is, the execution time of the i-th large language model when deployed on the j-th hardware platform).

[0035] The delayed prediction function is shown in Function 1 below:

[0036]

[0037] Among them, pred_Lat ij The execution time required to deploy the i-th large language model on the j-th hardware platform and execute the preset target task of the i-th large language model, max(*) is the maximum value, M_FLOPs i M_Data represents the total number of floating-point operations for the i-th large language model. i H_E_FLOPS represents the total data transfer volume of the i-th large language model. j H_E_bw represents the effective computing power of the j-th hardware platform. j This represents the effective memory bandwidth of the j-th hardware platform.

[0038] Furthermore, using the energy consumption prediction function, based on the total number of floating-point operations and the total amount of data transported by the i-th large language model, and based on the unit energy consumption of computing operations and the unit energy consumption of memory access on the j-th hardware platform, the total energy consumption required for the i-th large language model to execute the preset target task on the j-th hardware platform is calculated (that is, the total energy consumption when the i-th large language model is deployed on the j-th hardware platform).

[0039] The energy consumption prediction function is shown in Function 2 below:

[0040] Pred_Energy ij =M_FLOPs i ×H_η_comp j +M_Data i ×H_η_mem j (Function 2)

[0041] Among them, Pred_Energy ij The total energy consumption required to deploy the i-th large language model on the j-th hardware platform to execute the preset target task of the i-th large language model is M_FLOPs. i M_Data represents the total number of floating-point operations for the i-th large language model. iH_η_comp represents the total data transfer volume of the i-th large language model. j H_η_mem represents the unit energy consumption for computational operations on the j-th hardware platform. j Let be the energy consumption per unit of memory access for the j-th hardware platform.

[0042] Step S14: Using the optimization objective function, based on the key indicator parameters of each language model when deployed on different hardware platforms, determine the pairing results of the hardware platform and the large language model that meet the target pairing requirements.

[0043] In one embodiment, an optimization objective function is used to determine the pairing results of hardware platforms and large language models that meet the target pairing requirements based on key indicator parameters of each language model deployed on each hardware platform. This includes: calculating the optimization objective function value of each language model deployed on each hardware platform based on the key indicator parameters of each language model deployed on each hardware platform; and determining the pairing results of hardware platforms and large language models that meet the target pairing requirements among each language model and each hardware platform based on the optimization objective function value of each language model deployed on each hardware platform and according to the pairing rules between language models and hardware platforms.

[0044] Specifically, for M large language models and N hardware platforms (M and N are positive integers), based on the key performance indicators of the i-th large language model deployed on the j-th hardware platform, the optimization objective function value of the i-th large language model deployed on the j-th hardware platform is calculated using the optimization objective function.

[0045] The objective function is optimized to function 3 as follows:

[0046] O ij =w lat ×Norm(pred_Lat ij )+w energy ×Norm(Pred_Energy ij (Function 3)

[0047] Among them, O ij Pred_Energy is the objective function value for optimizing the i-th large language model when it is deployed on the j-th hardware platform. Norm(*) is the normalization function set according to requirements. ij The total energy consumption of the i-th large language model when deployed on the j-th hardware platform is given by pred_Lat. ij w represents the execution time when the i-th large language model is deployed on the j-th hardware platform. lat and w energy The weighting parameter can be set according to requirements, and this invention does not limit it. In one specific embodiment, wlat +w energy =1.

[0048] Furthermore, the pairing rules between language models and hardware platforms include:

[0049] If the target pairing requirement is of the first type mentioned above, then M is greater than 1 and N equals 1. Since there is only one hardware platform, we only need to determine the smallest optimization objective function value among the optimization objective function values ​​of each language model deployed on that hardware platform. The output of the large language model corresponding to the determined smallest optimization objective function value and the hardware platform is the pairing result of the hardware platform and the large language model that satisfies the target pairing requirement.

[0050] If the target pairing requirement is of the second type mentioned above, then M equals 1 and N is greater than 1. Since there is only one large language model, we only need to determine the smallest optimization objective function value among the optimization objective function values ​​when the large language model is deployed on each hardware platform. The hardware platform corresponding to the determined smallest optimization objective function value is then output as the pairing result between the hardware platform and the large language model that satisfies the target pairing requirement.

[0051] If the target pairing requirement is of the third type, then M is greater than 1 and N is greater than 1. The major language models can be split, and a suitable hardware platform can be selected for each major language model using the second type of method described above. Alternatively, the hardware platforms can be split, and a suitable major language model can be selected for each hardware platform using the first type of method described above, thus obtaining the final pairing result of the hardware platform and major language model that satisfies the target pairing requirement.

[0052] It should be noted that this invention, by quantifying the characteristics of large language models and hardware platforms and pairing them based on predicted performance / energy efficiency indicators, ensures that the large language model can better match the computing and storage characteristics of the hardware platform, thereby maximizing hardware utilization and achieving a better performance-to-power ratio. Simultaneously, the automated and systematic pairing process shortens the model selection cycle and reduces deployment costs. Furthermore, it can provide a useful reference for the architectural design of large language models facing specific hardware constraints.

[0053] To improve the method of the present invention, a specific embodiment is now provided.

[0054] Example 1: A method for selecting a large language model

[0055] The hardware characteristics of the hardware platform H1 are as follows: effective computing power H_E_flops_H1 = 70e12 FLOPs / s (FP16), effective memory bandwidth H_E_bw_H1 = 400e9 Bytes / s, energy consumption per unit of computing operation H_η_comp_H1 = 10e-12 J / FLOP (10pJ / FLOP), and energy consumption per unit of memory access H_η_mem_H1 = 100e-12 J / Byte (100pJ / Byte).

[0056] Candidate large language models include: large language model M_1, M_1 = M_AR (autoregressive architecture, parameter count ≈ 7B); large language model M_2, M_2 = M_DF (diffusion architecture, parameter count ≈ 7B).

[0057] For the large language model M_1, the preset target tasks are: prompt pre-padding length L_prompt = 2048 tokens, and generated sequence length L_gen = 1024 tokens. Weights are stored in 4-bit format, the key-value cache in 8-bit format, and activation in 16-bit (FP16) format. The model structure of the large language model M_1 is Llama2-7B, with 32 layers and a hidden layer dimension of 4096.

[0058] For the large language model M_2, the preset target task is: to process a total sequence length of 3072 tokens and to perform 16 diffusion steps.

[0059] The characteristic parameters of the large language model M_1 are obtained as follows: The number of floating-point operations in the Prefill stage (L_prompt = 2048) is approximately 29.3e12 FLOPs. The number of floating-point operations in the Generation stage (L_gen = 1024) is FLOPs_AR_gen = 1024 * 14.5e9 = 14.848e12 FLOPs; where the average computation cost per token is approximately 14.5GFLOPs (14.5e9 FLOPs). Therefore, the total number of floating-point operations for the large language model is 29.3e12 + 14.848e12 = 44.148e12 FLOPs.

[0060] Furthermore, the total data transfer volume of the large language model M_1 is obtained as follows: Weight (4 bits): Data_weight_prefill = 7e9 params * 0.5 Bytes / param = 3.5e9 Bytes. Activation (16 bits): Assuming the total activation volume written to main memory during the prefilling stage is roughly estimated as Data_act_prefill ≈ 2048 * 32 * ((12 + 1 + 8 / 3 * 2) * 4096 * 2) ≈ 10e9 Bytes (10GB). KV Cache write (8 bits): Data_KVCache_write_prefill = 2048 * 32 * (2 * 4096 * 1 Byte) ≈ 0.537e9 Bytes. Data_AR_prefill = 3.5e9 + 10e9 + 0.537e9 = 14.037e9 Bytes.

[0061] Total data transported per token generation: Data_per_token_gen_total: Weight (Weight, 4 bits): Data_weight_per_token_gen = 3.5e9 Bytes.

[0062] Activation (16-bit): Data_act_per_token_gen_all_layers≈1*32*((12+1+8 / 3*2)*4096*2)≈4.8e6 Bytes.

[0063] KV Cache read (8bit): Data_KVCache_read_per_token_gen≈(2048+1024 / 2)*32*(2*4096*1Byte)≈0.671e9 Bytes.

[0064] KV Cache write (8bit): Data_KVCache_write_per_token_gen≈1*32*(2*4096*1Byte)≈0.00026e9 Bytes.

[0065] Data_per_token_gen_total≈4.174e9 Bytes.

[0066] Data_AR_gen=1024*4.174e9 Bytes≈4.274e12 Bytes.

[0067] The total data transfer volume M_Data_AR of the large language model M_1 is: M_Data_AR=Data_AR_prefill+Data_AR_gen≈4.288e12 Bytes.

[0068] For the large language model M_2: the number of floating-point operations for the prefill equivalent length (3072 tokens) is: FLOPs_AR_prefill_equiv(3072)≈43.95e12 FLOPs.

[0069] The number of floating-point operations per step in the large language model M_2 is calculated as 100% of the AR prefill equivalent length: FLOPs_per_step_DF = 43.95e12 FLOPs. The total number of floating-point operations in the large language model M_2 is M_FLOPs_DF = 16 * 43.95e12 = 703.2e12 FLOPs.

[0070] Furthermore, obtain the total data transfer volume of the large language model M_2:

[0071] Single-step weight data (4 bits): Data_weights_DF = 3.5e9 Bytes. Single-step activation data (16 bits): Roughly estimated as Data_act ≈ 3072 * 32 * ((12 + 1 + 8 / 3 * 2) * 4096 * 2) ≈ 14.76e9 Bytes (14.76 GB). Data_per_step_DF = 3.5e9 + 14.76e9 ≈ 18e9 Bytes. Total data transfer amount for the large language model M_2: M_Data_DF = 16 * 18e9 = 288e9 Bytes = 0.288e12 Bytes.

[0072] Furthermore, the key performance parameters of the large language models M_1 and M_2 when deployed on hardware platform H1 were predicted:

[0073] Key performance indicators (M_1) for predicting the large language model M_1 deployed on hardware platform H1 (total floating-point operations M_FLOPs_AR = 44.148e12, total data transfer M_Data_AR = 4.288e12):

[0074] Theoretical computation time: Time_compute_AR = M_FLOPs_AR / H_E_flops_H1 = 44.148e12 FLOPs / 70e12 FLOPs / s ≈ 0.6307s. Theoretical memory access time: Time_memory_AR = M_Data_AR / H_E_bw_H1 = 4.288e12 Bytes / 400e9 Bytes / s ≈ 10.72s.

[0075] The execution time of the predicted large language model M_1 when deployed on hardware platform H1: Pred_Lat_AR = max(Time_compute_AR, Time_memory_AR) = max(0.6307s, 10.72s) = 10.72s.

[0076] Energy consumption of the computed component: Energy_compute_AR=M_FLOPs_AR*H_η_comp_H1=44.148e12FLOPs*10e-12J / FLOP=441.48J.

[0077] Calculate the energy consumption of memory access: Energy_memory_AR=M_Data_AR*H_η_mem_H1=4.288e12Bytes*100e-12J / Byte=428.8J.

[0078] The total energy consumption of the predicted large language model M_1 when deployed on hardware platform H1 is: Pred_Energy_AR = Energy_compute_AR + Energy_memory_AR = 441.48J + 428.8J = 870.28J.

[0079] Key performance indicators (M_2) for predicting the large language model M_2 deployed on hardware platform H1 (total floating-point operations M_FLOPs_DF = 703.2e12, total data transfer M_Data_DF = 0.288e12):

[0080] The theoretical computation time is calculated as follows: Time_compute_DF = M_FLOPs_DF / H_E_flops_H1 = 703.2e12FLOPs / 70e12 FLOPs / s ≈ 10.046s.

[0081] The theoretical memory access time is calculated as follows: Time_memory_DF = M_Data_DF / H_E_bw_H1 = 0.288e12 Bytes / 400e9 Bytes / s ≈ 0.72s.

[0082] The execution time of the predicted large language model M_2 when deployed on hardware platform H1 is Pred_Lat_DF = max(Time_compute_DF, Time_memory_DF) = max(10.046s, 0.72s) = 10.046s.

[0083] The energy consumption of the calculation section is Energy_compute_DF=M_FLOPs_DF*H_η_comp_H1=703.2e12FLOPs*10e-12J / FLOP=7032J.

[0084] Energy consumption for memory access: Energy_memory_DF = M_Data_DF * H_η_mem_H1 = 0.288e12 Bytes * 100e-12 J / Byte = 28.8 J.

[0085] The total energy consumption of the large language model M_2 when deployed on hardware platform H1 is Pred_Energy_DF=Energy_compute_DF+Energy_memory_DF=7032J+28.8J=7060.8J.

[0086] Furthermore, the objective function O_i = 0.6*Norm(Pred_Lat_i) + 0.4*Norm(Pred_Energy_i) is optimized. Min-Max normalization is used.

[0087] Calculate the normalized value: Min_Lat = min(10.72, 10.046) = 10.046

[0088] Max_Lat=10.72. Min_Energy=min(870.28,7060.8)=870.28

[0089] Max_Energy = 7060.8

[0090] Norm_Lat_AR=(10.72-10.046) / (10.72-10.046)=1

[0091] Norm_Energy_AR=(870.28-870.28) / (7060.8-870.28)=0

[0092] Norm_Lat_DF=(10.046-10.046) / (10.72-10.046)=0

[0093] Norm_Energy_DF=(7060.8-870.28) / (7060.8-870.28)=1

[0094] Calculate the optimization objective function value O_i:

[0095] O_AR = 0.6 * 1 + 0.4 * 0 = 0.6

[0096] O_DF = 0.6 * 0 + 0.4 * 1 = 0.4

[0097] Because O_AR(0.6)>O_DF(0.4), the large language model M_2 is selected for deployment on hardware platform H1.

[0098] Figure 2 This is a schematic block diagram of a large language model matching device based on hardware and model features provided in an embodiment of this application. Figure 2 As shown, the large language model matching device 200 based on hardware and model features includes:

[0099] The acquisition module 201 is used to acquire the target pairing requirements and, based on the target pairing requirements, acquire at least one hardware platform and at least one large language model.

[0100] The parameter acquisition module 202 is used to acquire the hardware feature parameters of each hardware platform and to make each language model execute the corresponding preset target task to obtain the feature parameters of each language model.

[0101] The prediction module 203 is used to predict the key indicator parameters of each language model when it is deployed on each hardware platform based on the hardware feature parameters of each hardware platform and the characteristic parameters of each language model.

[0102] Selection module 204 is used to determine the pairing results of hardware platforms and large language models that meet the target pairing requirements by using an optimization objective function and based on the key indicator parameters of each language model when deployed on different hardware platforms.

[0103] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0104] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0105] In one embodiment, the hardware characteristic parameters include: the effective computing power of the hardware platform, the effective memory bandwidth, the energy consumption per unit of computing operation, and the energy consumption per unit of memory access.

[0106] In one embodiment, the characteristic parameters include: total number of floating-point operations and total data transfer volume.

[0107] In one embodiment, the prediction function includes a delay prediction function and an energy consumption prediction function.

[0108] In one embodiment, based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each language model, a prediction function is used to predict the key indicator parameters of each language model deployed on each hardware platform. These parameters include: based on the effective computing power and effective memory bandwidth of each hardware platform, and based on the total number of floating-point operations and the total data transfer volume of each large language model, a latency prediction function is used to predict the execution time of each large language model deployed on each hardware platform; based on the energy consumption per unit of computational operation and the energy consumption per unit of memory access of each hardware platform, and based on the total number of floating-point operations and the total data transfer volume of each large language model, an energy consumption prediction function is used to predict the total energy consumption of each large language model deployed on each hardware platform; and the execution time and total energy consumption of each large language model deployed on each hardware platform are output as key indicator parameters of each large language model deployed on each hardware platform.

[0109] In one embodiment, an optimization objective function is used to determine the pairing results of hardware platforms and large language models that meet the target pairing requirements based on key indicator parameters of each language model deployed on each hardware platform. This includes: calculating the optimization objective function value of each language model deployed on each hardware platform based on the key indicator parameters of each language model deployed on each hardware platform; and determining the pairing results of hardware platforms and large language models that meet the target pairing requirements among each language model and each hardware platform based on the optimization objective function value of each language model deployed on each hardware platform and according to the pairing rules between language models and hardware platforms.

[0110] Figure 3 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 3 As shown, the electronic terminal includes at least one processor 301, a memory 302, at least one network interface 303, and a user interface 305. The various components in the device are coupled together via a bus system 304. It is understood that the bus system 304 is used to implement communication between these components. In addition to a data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 3 The general will label all buses as bus systems.

[0111] The user interface 305 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0112] It is understood that memory 302 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0113] In this embodiment of the invention, the memory 302 is used to store various types of data to support the operation of the electronic terminal 300. Examples of this data include: any executable program for operation on the electronic terminal 300, such as the operating system 3021 and application programs 3022; the operating system 3021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 3022 may contain various applications, such as media players, browsers, etc., for implementing various application services. The implementation of the large language model matching method based on hardware and model features provided in this embodiment of the invention can be included in the application program 3022.

[0114] The methods disclosed in the above embodiments of the present invention can be applied to processor 301, or implemented by processor 301. Processor 301 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 301 or by instructions in the form of software. The processor 301 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 301 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 301 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0115] In an exemplary embodiment, the electronic terminal 300 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0116] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 1 The large language model pairing method based on hardware and model features in the illustrated embodiment.

[0117] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 1 The large language model pairing method based on hardware and model features in the illustrated embodiment.

[0118] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0119] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0120] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0124] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0125] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0127] In summary, this application provides a method, apparatus, terminal, medium, and product for pairing large language models based on hardware and model features. The method includes: acquiring at least one hardware platform and at least one large language model based on the acquired target pairing requirements; acquiring the hardware feature parameters of each hardware platform and having each large language model execute a corresponding preset target task to obtain the feature parameters of each large language model; based on the hardware feature parameters of each hardware platform and the feature parameters of each large language model, using a prediction function to predict the key indicator parameters of each large language model when deployed on each hardware platform; and using an optimization objective function, based on the key indicator parameters of each large language model when deployed on each hardware platform, determining the pairing result of the hardware platform and the large language model that meets the target pairing requirements. This application fully considers the features of the hardware platform and the large language model, ensuring that the model can better match the hardware platform. Therefore, this application effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0128] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A hardware and model feature based large language model pairing method, characterized in that, The method comprises the following steps: obtaining a target pairing requirement, and obtaining at least one hardware platform and at least one large language model according to the target pairing requirement; obtaining hardware characteristic parameters of each hardware platform, and causing each large language model to execute a corresponding preset target task to obtain characteristic parameters of each large language model; wherein the hardware characteristic parameters comprise effective computing power, effective memory bandwidth, computing operation unit energy consumption and memory access unit energy consumption of the hardware platform; and the characteristic parameters comprise total floating point operation times and total data carrying capacity; based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each large language model, using a prediction function to predict key indicator parameters of each large language model when each large language model is deployed on each hardware platform; wherein the prediction function comprises a delay prediction function and an energy consumption prediction function; based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each large language model, using the prediction function to predict the key indicator parameters of each large language model when each large language model is deployed on each hardware platform, comprising: based on the effective computing power and the effective memory bandwidth of each hardware platform, and based on the total floating point operation times and the total data carrying capacity of each large language model, using the delay prediction function to predict the execution time of each large language model when each large language model is deployed on each hardware platform; based on the computing operation unit energy consumption and the memory access unit energy consumption of each hardware platform, and based on the total floating point operation times and the total data carrying capacity of each large language model, using the energy consumption prediction function to predict the total energy consumption of each large language model when each large language model is deployed on each hardware platform; outputting the execution time and the total energy consumption of each large language model when each large language model is deployed on each hardware platform as the key indicator parameters of each large language model when each large language model is deployed on each hardware platform; using an optimization objective function, based on the key indicator parameters of each large language model when each large language model is deployed on each hardware platform, determining a pairing result of the hardware platform and the large language model that meets the target pairing requirement.

2. The hardware and model feature-based large language model pairing method according to claim 1, characterized in that, using an optimization objective function, based on the key indicator parameters of each large language model when each large language model is deployed on each hardware platform, determining a pairing result of the hardware platform and the large language model that meets the target pairing requirement, comprising: based on the key indicator parameters of each large language model when each large language model is deployed on each hardware platform, using the optimization objective function to calculate an optimization objective function value of each large language model when each large language model is deployed on each hardware platform; based on the optimization objective function value of each large language model when each large language model is deployed on each hardware platform, according to a pairing rule of the language model and the hardware platform, determining the pairing result of the hardware platform and the large language model that meets the target pairing requirement in each large language model and each hardware platform.

3. A hardware and model feature based large language model pairing apparatus, characterized in that, The method comprises the following steps: an obtaining module is configured to obtain a target pairing requirement, and obtain at least one hardware platform and at least one large language model according to the target pairing requirement; The parameter acquisition module is configured to acquire hardware characteristic parameters of each hardware platform, and cause each large language model to perform a corresponding preset target task to obtain characteristic parameters of each large language model. The hardware characteristic parameters include effective computing power, effective memory bandwidth, computing operation unit energy consumption, and memory access unit energy consumption of the hardware platform. The characteristic parameters include total floating point operation times and total data carrying capacity. The prediction module is configured to predict key indicator parameters of each large language model when each large language model is deployed on each hardware platform based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each large language model by using a prediction function. The prediction function includes a delay prediction function and an energy consumption prediction function. The key indicator parameters of each large language model when each large language model is deployed on each hardware platform are predicted based on the hardware characteristic parameters of each hardware platform and the characteristic parameters of each large language model by using the prediction function, including: The execution time of each large language model when each large language model is deployed on each hardware platform is predicted based on the effective computing power and the effective memory bandwidth of each hardware platform and based on the total floating point operation times and the total data carrying capacity of each large language model by using the delay prediction function. The total energy consumption of each large language model when each large language model is deployed on each hardware platform is predicted based on the computing operation unit energy consumption and the memory access unit energy consumption of each hardware platform and based on the total floating point operation times and the total data carrying capacity of each large language model by using the energy consumption prediction function. The execution time and the total energy consumption of each large language model when each large language model is deployed on each hardware platform are output as the key indicator parameters of each large language model when each large language model is deployed on each hardware platform. The selection module is configured to determine a pairing result of the hardware platform and the large language model that meets the target pairing requirement based on the key indicator parameters of each large language model when each large language model is deployed on each hardware platform by using an optimization objective function.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 2.

5. A computer program product, characterised in that, The computer program product includes computer program code, which, when executed on a computer, causes the computer to implement the method of any one of claims 1 to 2.

6. An electronic terminal comprising a memory, a processor and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 2.

Images