Expert popularity prediction method and device in MoE, storage medium and equipment
By statistically analyzing the resource usage parameters of experts in MoE and using the EMA model to predict popularity levels, the problem of low resource utilization is solved, and more efficient resource allocation and expert popularity prediction are achieved.
Patent Information
- Application Number
- CN202510998796.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-21
- Publication Date
- 2025-11-21
AI Technical Summary
In existing technologies, the resource utilization of each expert in the hybrid expert model (MoE) is low, the inference latency fluctuates greatly, and the concurrency capability is limited. It is also unable to effectively predict the popularity of experts to optimize resource allocation.
By using a sliding window to statistically analyze the resource usage parameters of each expert in MoE, determining the weight parameters according to the model type, and using the exponential moving average (EMA) model to process the resource usage information, the popularity level of experts is predicted, and resource allocation is adjusted accordingly.
It improved resource utilization, enabled more accurate expert popularity prediction and resource allocation, and optimized MoE resource scheduling.
Smart Images
Figure CN120994330A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, in particular to a method and device for predicting the heat of experts in a MoE, a storage medium and equipment. BACKGROUND
[0002] A mixture of experts (MoE) is composed of a gating network and multiple specialized experts (i.e., sub-models). After receiving a token, the gating network will distribute the token to an expert that is good at processing problems in the field corresponding to the token, and the expert will process the token.
[0003] In related technologies, a computer device allocates the same GPU resources to all experts, and the experts use the GPU resources to process tokens. However, some experts are frequently called, and the remaining majority of experts are in a long cold start or low load state, resulting in low resource utilization, large reasoning delay fluctuations, and limited concurrency capabilities. Therefore, how to predict the heat of each expert and allocate resources to the experts according to the heat has become a problem to be solved. SUMMARY
[0004] The present application provides a method and device for predicting the heat of experts in a MoE, a storage medium and equipment, to solve the problem of being unable to predict the heat of each expert in a MoE. The technical solution is as follows:
[0005] According to a first aspect of the present application, a method for predicting the heat of experts in a MoE is provided, the method comprising:
[0006] using a sliding window to count resource occupation parameters of each expert in the MoE;
[0007] determining a weight parameter corresponding to each resource occupation parameter according to a model type of the MoE;
[0008] calculating resource occupation information of each expert according to the resource occupation parameters and the weight parameters;
[0009] processing the resource occupation information using an exponential moving average (EMA) model to obtain a heat parameter of each expert;
[0010] predicting the heat level of each expert according to the heat parameter.
[0011] In one possible implementation, the processing of the resource occupation information using the EMA model to obtain the heat parameter of each expert comprises:
[0012] using the formula Et = α × x t + (1 - α) × E t-1 calculating the resource occupation information to obtain a heat parameter of each expert;
[0013] wherein, E t denotes the heat parameter at time t, E t-1 denotes the heat parameter at time t-1, a denotes a smoothing coefficient, and x t denotes the resource occupation information at time t.
[0014] In a possible implementation, the method further includes:
[0015] obtaining a heat level change frequency of each expert in the MoE;
[0016] determining the smoothing coefficient according to the heat level change frequency, wherein the size of the smoothing coefficient is in positive correlation with the level of the heat level change frequency.
[0017] In a possible implementation, the resource occupation parameter includes at least one of a number of invocations per unit time, a number of tokens processed per unit time, a total service time per unit time, a queue depth, and a video graphics processing unit (vGPU) instance memory occupation ratio of the expert.
[0018] In a possible implementation, the resource occupation parameter includes at least one of a number of invocations per unit time, a number of tokens processed per unit time, a total service time per unit time, a queue depth, and a video graphics processing unit (vGPU) instance memory occupation ratio of the expert.
[0019] configuring a first sliding window and a second sliding window, the first sliding window representing a window of a fixed time length, and the second sliding window representing a window during a period of binding with a vGPU instance;
[0020] statistically obtaining, in the first sliding window, at least one of a number of invocations, a number of tokens processed, a total service time, and a queue depth of each expert in the MoE;
[0021] statistically obtaining, in the second sliding window, a vGPU instance memory occupation ratio of the vGPU instance.
[0022] In a possible implementation, the resource occupation information of each expert is calculated according to the resource occupation parameter and the weight parameter, including:
[0023] according to a model type of the MoE, selecting a target resource occupation parameter from the resource occupation parameter, and performing weighted calculation on the target resource occupation parameter and a corresponding weight parameter to obtain the resource occupation information of each expert; or
[0024] The resource occupation information of each expert is obtained by weighting calculation of all resource occupation parameters and corresponding weight parameters.
[0025] According to a second aspect of the present application, a device for predicting the heat of an expert in a MoE is provided, and the device comprises:
[0026] a statistical module configured to statistically calculate resource occupation parameters of each expert in the MoE by using a sliding window;
[0027] a determination module configured to determine weight parameters corresponding to each resource occupation parameter according to a model type of the MoE;
[0028] a calculation module configured to calculate resource occupation information of each expert according to the resource occupation parameters and the weight parameters;
[0029] a prediction module configured to process the resource occupation information by using an exponential moving average (EMA) model to obtain a heat parameter of each expert;
[0030] The prediction module is configured to predict a heat level of each expert according to the heat parameter.
[0031] In a possible implementation, the prediction module is further configured to:
[0032] the resource occupation information is calculated by using a formula E t = alpha * x t + (1-alpha) * E t-1 to obtain the heat parameter of each expert;
[0033] wherein E t represents the heat parameter at time t, E t-1 represents the heat parameter at time t-1, alpha represents a smoothing coefficient, and x t represents the resource occupation information at time t.
[0034] According to a third aspect of the present application, a computer readable storage medium is provided, and the storage medium stores at least one instruction, which is loaded and executed by a processor to implement the method for predicting the heat of an expert in a MoE as described above.
[0035] According to a fourth aspect of the present application, a computer device is provided, and the computer device comprises the device for predicting the heat of an expert in a MoE.
[0036] The technical scheme provided by the present application has at least the following beneficial effects:
[0037] By using the sliding window to count the resource occupation parameters of each expert in the MoE, then determining the weight parameters corresponding to each resource occupation parameter according to the model type of the MoE, then calculating the resource occupation information of each expert according to the resource occupation parameters and the weight parameters, finally processing the resource occupation information by using the EMA model to obtain the heat parameters of each expert, and predicting the heat level of each expert according to the heat parameters, the heat level of each expert can be predicted according to the resource scheduling of each expert, so as to adjust the resource allocation of the expert according to the heat level, thereby improving the utilization rate of resources.
[0038] By selecting the target resource occupation parameters according to the model type of the MoE, and the weight parameters corresponding to each target resource occupation parameter, the corresponding parameters can be selected with emphasis in different tasks, and the heat of the expert can be more accurately predicted.
[0039] The smoothing coefficient can control the degree of response to new data. The greater the smoothing coefficient, the more sensitive the EMA model, and the faster the response to new data. The smaller the smoothing coefficient, the smoother the EMA model, and the more conservative the response to new data. Therefore, the smoothing coefficient can be adjusted according to the heat level change frequency, that is, the higher the heat level change frequency, the greater the smoothing coefficient; the lower the heat level change frequency, the smaller the smoothing coefficient, thereby improving the accuracy of heat prediction. BRIEF DESCRIPTION OF DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0041] Figure 1 is a flow chart of the heat prediction method of the expert in the MoE provided by an embodiment of the present application;
[0042] Figure 2 is a flow chart of the heat prediction method of the expert in the MoE provided by another embodiment of the present application;
[0043] Figure 3 is a structural block diagram of the heat prediction device of the expert in the MoE provided by another embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.
[0045] The system of the present application comprises multiple GPU nodes, each of which is configured with at least one graphics card. When deploying a MoE model, we can deploy each expert on a graphics card on a GPU node. However, for some experts with low popularity, allocating a graphics card to each expert will result in waste of graphics card resources. Therefore, we can virtualize a graphics card into multiple vGPU instances, allocate a certain resource to each vGPU instance, and then bind each expert to a vGPU instance. In this way, when multiple vGPU instances are created on a graphics card, the experts bound to these vGPU instances can share the resources of a graphics card to improve the utilization of the graphics card. The resources mentioned here can be any resources of the graphics card, including but not limited to the size of the video memory and the computing power quota.
[0046] As shown in Figure 1 the method flowchart of the expert popularity prediction method in the MoE provided by an embodiment of the present application, which can be applied in a computer device. The expert popularity prediction method in the MoE can comprise:
[0047] Step 101, using a sliding window to count the resource occupation parameters of each expert in the MoE.
[0048] After the gating network in the MoE receives a token, it allocates the token to an expert who is good at processing problems in the field corresponding to the token, and the expert calls the resources of the corresponding vGPU instance to process the token.
[0049] In the process of processing the token by the expert, the computer device can use a sliding window to count the resource occupation parameters of the expert. The resource occupation parameters can be selected according to business requirements, and the sliding window can be set according to the resource occupation parameters to be counted, which is not limited here.
[0050] Step 102, determining the weight parameters corresponding to each resource occupation parameter according to the model type of the MoE.
[0051] The computer device can set the corresponding parameters for each resource occupation parameter according to the importance of the resource occupation parameters. The computer device can set the same set of weight parameters for different MoEs, or can also set different weight parameters for different MoEs.
[0052] In this embodiment, the computer device can set different weight parameters for different MoEs according to the model types of the MoEs. The MoEs can have different model types. The model types mentioned here include but are not limited to: Large Language Model (LLM), Diffusion Model, and Retrieval-Augmented Generation (RAG) model.
[0053] In step 103, the resource occupation information of each expert is calculated according to the resource occupation parameters and the weight parameters.
[0054] The computer device can select all the resource occupation parameters and the corresponding weight parameters to calculate the resource occupation information, or select part of the resource occupation parameters and the corresponding weight parameters to calculate the resource occupation information. The specific calculation method is described below.
[0055] In step 104, the resource occupation information is processed by using the EMA model to obtain the heat parameter of each expert.
[0056] The Exponential Moving Average (EMA) model is a model for smoothing time series data. It calculates the weighted average of the current value and the past exponential moving average value to reduce the noise and fluctuations in the data, thereby reflecting the trend of the data.
[0057] The computer device can input the resource occupation information into the formula of the EMA model for calculation, and the heat parameter can be obtained.
[0058] In step 105, the heat level of each expert is predicted according to the heat parameter.
[0059] In this embodiment, the computer device can divide the heat of the experts into multiple levels according to the business needs, such as low, medium, high, etc. The number of heat levels is not limited in this embodiment.
[0060] The computer device can pre-set the mapping relationship between the heat parameter and the heat level, and then for each expert, find the heat level corresponding to the heat parameter of the expert in the mapping relationship. Alternatively, the computer device can pre-set the calculation formula between the heat parameter and the heat level, and then for each expert, input the heat parameter into the calculation formula to obtain the corresponding heat level.
[0061] In summary, the method for predicting the heat of experts in the MoE provided by the embodiments of the present application can be used in a computer device. The method for predicting the heat of experts in the MoE can include the following steps.
[0062] As shown in Figure 2 the flowchart of the method for predicting the heat of experts in the MoE provided by an embodiment of the present application. The method for predicting the heat of experts in the MoE can be applied to a computer device. The method for predicting the heat of experts in the MoE can include the following steps.
[0063] Step 201, configure a first sliding window and a second sliding window. The first sliding window represents a window of a fixed time length, and the second sliding window represents a window during which a vGPU instance is bound.
[0064] After the gating network in the MoE receives a token, it assigns the token to an expert who is good at processing problems in the field corresponding to the token, and the expert calls the resource of the corresponding vGPU instance to process the token.
[0065] In the process of processing the token by the expert, the computer device can use a sliding window to count the resource occupation parameters of the expert. There are two ways to count the resource occupation parameters. The first way is to count in units of time, and the second way is to count in units of bound vGPU instances.
[0066] For the first counting method, the computer device configures a first sliding window. The first sliding window represents a window of a fixed time length, and the time length can be set according to business requirements, such as 10s, 60s, etc.
[0067] For the second counting method, the computer device configures a second sliding window. The second sliding window represents a window during which a vGPU instance is bound.
[0068] Step 202, count at least one of the number of calls, the number of tokens processed, the total service time length, and the queue depth of each expert in the MoE in the first sliding window.
[0069] In this embodiment, the resource occupation parameter can include at least one of the number of invocations per unit time, the number of tokens processed per unit time, the total service time per unit time, and the queue depth.
[0070] The number of invocations refers to the number of times that the expert is hit by the reasoning request per unit time.
[0071] The number of tokens processed refers to the total number of all tokens processed by the expert per unit time.
[0072] The total service time refers to the total time for the expert to process all tokens per unit time. For example, in the last 60 seconds, the expert processed 30 tokens, and each time the average time consumption was 50 ms, so the total service time was about 30 x 50 ms = 1.5 s.
[0073] The queue depth refers to the queue length of the tokens to be processed by the expert.
[0074] In step 203, the memory occupation ratio of the vGPU instance is counted in the second sliding window.
[0075] In this embodiment, the resource occupation parameter can also include the memory occupation ratio of the vGPU instance to which the expert is bound.
[0076] In step 204, the weight parameter corresponding to each resource occupation parameter is determined according to the model type of the MoE.
[0077] The computer device can set a corresponding parameter for each resource occupation parameter according to the importance of the resource occupation parameter. The computer device can set the same set of weight parameters for different MoEs, or can also set different weight parameters for different MoEs.
[0078] In this embodiment, the computer device can set different weight parameters for different MoEs according to the model type of the MoE. The MoE can have different model types. The model type mentioned here includes but is not limited to LLM, DiffusionModel, and RAG model.
[0079] In one example, the computer device sets the weight parameter for the number of invocations to 0.3, the weight parameter for the number of tokens to 0.2, the weight parameter for the total service time to 0.2, the weight parameter for the queue depth to 0.1, and the weight parameter for the memory occupation ratio to 0.3.
[0080] In step 205, the resource occupation information of each expert is calculated according to the resource occupation parameter and the weight parameter.
[0081] The computer device can select all resource occupation parameters and corresponding weight parameters to calculate the resource occupation information, or select part of the resource occupation parameters and corresponding weight parameters to calculate the resource occupation information.
[0082] Specifically, calculating the resource occupation information of each expert according to the resource occupation parameters and the weight parameters can include: selecting target resource occupation parameters from the resource occupation parameters according to the model type of the MoE, and calculating the target resource occupation parameters and the corresponding weight parameters to obtain the resource occupation information of each expert; or calculating all resource occupation parameters and the corresponding weight parameters to obtain the resource occupation information of each expert.
[0083] Suppose the resource occupation information is x t , the call count is CallCount, and the corresponding weight parameter is β1; the token count is TokenCount, and the corresponding weight parameter is β2; the total service time is ServiceTime, and the corresponding weight parameter is β3; the queue depth is QueueDepth, and the corresponding weight parameter is β4; the memory occupation ratio is MemUtil, and the corresponding weight parameter is β5; then x t = CallCount × β1 + TokenCount × β2 + ServiceTime × β3 + QueueDepth × β4 + MemUtil × β5.
[0084] In step 206, the resource occupation information is processed by using the EMA model to obtain the heat parameter of each expert.
[0085] The EMA model is a model for smoothing time series data, which reduces the noise and fluctuations in the data by weightedly averaging the current value and the past exponential moving average value, so as to reflect the trend of the data.
[0086] Specifically, processing the resource occupation information by using the exponential moving average value EMA model to obtain the heat parameter of each expert can include:
[0087] using the formula E t = α × x t + (1-α) × E t-1 to calculate the resource occupation information to obtain the heat parameter of each expert;
[0088] wherein, E t represents the heat parameter at time t, E t-1 represents the heat parameter at time t-1, α represents a smoothing coefficient, and x t represents the resource occupation information at time t.
[0089] In this embodiment, the smoothing coefficient a is in the range of (0, 1). The smoothing coefficient a can control the degree of response to new data. The greater the smoothing coefficient a, the more sensitive the EMA model, and the faster the response to new data. The smaller the smoothing coefficient a, the smoother the EMA model, and the more conservative the response to new data.
[0090] Based on the above characteristics of the smoothing coefficient a, the computer device can further obtain the frequency of change of the heat level of each expert in the MoE; and determine the smoothing coefficient according to the frequency of change of the heat level, wherein the size of the smoothing coefficient is in a positive correlation with the level of the frequency of change of the heat level. That is, the higher the frequency of change of the heat level, the greater the smoothing coefficient a; and the lower the frequency of change of the heat level, the smaller the smoothing coefficient a.
[0091] In one example, if the heat of the expert is stable for a long time, the smoothing coefficient a can be set to 0.1 to smooth the expert with a large change in the heat level; if the heat of the expert is relatively stable and occasionally spikes, the smoothing coefficient a can be set to 0.3-0.5 to balance the response and stability; and if the heat of the expert changes frequently, the smoothing coefficient a can be set to 0.8 to respond quickly.
[0092] In step 207, the heat level of each expert is predicted according to the heat parameter.
[0093] In this embodiment, the computer device can divide the heat of the expert into multiple levels according to the business needs, such as low, medium, high, and the like, and the number of heat levels is not limited in this embodiment.
[0094] The computer device can pre-set a mapping relationship between the heat parameter and the heat level, and then for each expert, find the heat level corresponding to the heat parameter of the expert in the mapping relationship. Alternatively, the computer device can pre-set a calculation formula between the heat parameter and the heat level, and then for each expert, input the heat parameter into the calculation formula to obtain the corresponding heat level.
[0095] In summary, the method for predicting the heat of the expert in the MoE provided in the embodiments of the present application, by using the sliding window to count the resource occupation parameter of each expert in the MoE, then determining the weight parameter corresponding to each resource occupation parameter according to the model type of the MoE, then calculating the resource occupation information of each expert according to the resource occupation parameter and the weight parameter, and finally processing the resource occupation information by using the EMA model to obtain the heat parameter of each expert, and predicting the heat level of each expert according to the heat parameter, so that the heat level of the expert can be predicted according to the resource scheduling of each expert, so as to adjust the resource allocation of the expert according to the heat level, thereby improving the utilization rate of the resource.
[0096] By selecting the target resource occupation parameters according to the model type of the MoE, and the weight parameters corresponding to each target resource occupation parameter, the corresponding parameters can be selected with emphasis in different tasks, and the heat of the experts can be more accurately predicted.
[0097] The smoothing coefficient can control the degree of response to new data. The greater the smoothing coefficient, the more sensitive the EMA model, and the faster the response to new data. The smaller the smoothing coefficient, the smoother the EMA model, and the more conservative the response to new data. Therefore, the smoothing coefficient can be adjusted according to the heat level change frequency, that is, the higher the heat level change frequency, the greater the smoothing coefficient; the lower the heat level change frequency, the smaller the smoothing coefficient, thereby improving the accuracy of heat prediction.
[0098] As shown in FIG. 1, Figure 3 FIG. 1 shows a structural block diagram of a heat prediction device for experts in a MoE according to an embodiment of the present application. The heat prediction device for experts in the MoE can be applied to a computer device, and the SOC includes:
[0099] The statistical module 310 is configured to use a sliding window to count resource occupation parameters of each expert in the MoE.
[0100] The determination module 320 is configured to determine weight parameters corresponding to each resource occupation parameter according to the model type of the MoE.
[0101] The calculation module 330 is configured to calculate resource occupation information of each expert according to the resource occupation parameters and the weight parameters.
[0102] The processing module 340 is configured to process the resource occupation information using an EMA model to obtain heat parameters of each expert.
[0103] The prediction module 350 is configured to predict a heat level of each expert according to the heat parameters.
[0104] In an optional embodiment, the processing module 340 is further configured to:
[0105] use the formula E t = α × x t + (1-α) × E t-1 to calculate the resource occupation information to obtain the heat parameters of each expert.
[0106] wherein E t represents the heat parameter at time t, E t-1 represents the heat parameter at time t-1, α represents a smoothing coefficient, and x t represents the resource occupation information at time t.
[0107] In an optional embodiment, the processing module 340 is further configured to:
[0108] obtain a frequency of change of the heat level of each expert in the MoE;
[0109] determine a smoothing coefficient according to the frequency of change of the heat level, wherein the size of the smoothing coefficient is in positive correlation with the high and low of the frequency of change of the heat level.
[0110] In an optional embodiment, the resource occupation parameter includes at least one of the number of invocations per unit time, the number of tokens processed per unit time, the total service time per unit time, the queue depth, and the memory occupation proportion of a virtual graphics processing unit (vGPU) instance bound to the expert.
[0111] In an optional embodiment, the statistical module 310 is further configured to:
[0112] configure a first sliding window and a second sliding window, the first sliding window representing a window of a fixed time length, and the second sliding window representing a window during the binding of the vGPU instance;
[0113] statistically obtain at least one of the number of invocations, the number of tokens processed, the total service time, and the queue depth of each expert in the MoE within the first sliding window;
[0114] statistically obtain the memory occupation proportion of the vGPU instance within the second sliding window.
[0115] In an optional embodiment, the calculation module 330 is further configured to:
[0116] select a target resource occupation parameter from the resource occupation parameters according to the model type of the MoE, and perform weighted calculation on the target resource occupation parameter and the corresponding weight parameter to obtain the resource occupation information of each expert; or
[0117] perform weighted calculation on all the resource occupation parameters and the corresponding weight parameters to obtain the resource occupation information of each expert.
[0118] In summary, the heat prediction device for experts in the MoE provided by the embodiments of the present application can statistically obtain the resource occupation parameters of each expert in the MoE by using the sliding window, then determine the weight parameter corresponding to each resource occupation parameter according to the model type of the MoE, then calculate the resource occupation information of each expert according to the resource occupation parameter and the weight parameter, finally process the resource occupation information by using the EMA model to obtain the heat parameter of each expert, and predict the heat level of each expert according to the heat parameter, so that the heat level of each expert can be predicted according to the resource scheduling of each expert, so as to adjust the resource allocation of the experts according to the heat level, thereby improving the utilization rate of resources.
[0119] By selecting the target resource occupation parameters according to the model type of the MoE, and the weight parameters corresponding to each target resource occupation parameter, the corresponding parameters can be selected with emphasis in different tasks, and the heat of the expert can be more accurately predicted.
[0120] The smoothing coefficient can control the degree of response to new data. The greater the smoothing coefficient, the more sensitive the EMA model, and the faster the response to new data. The smaller the smoothing coefficient, the smoother the EMA model, and the more conservative the response to new data. Therefore, the smoothing coefficient can be adjusted according to the heat level change frequency, that is, the higher the heat level change frequency, the greater the smoothing coefficient; the lower the heat level change frequency, the smaller the smoothing coefficient, thereby improving the accuracy of heat prediction.
[0121] An embodiment of the present application provides a computer readable storage medium, the storage medium stores at least one instruction, the at least one instruction is loaded and executed by a processor to realize the heat prediction method of the expert in the MoE as described above.
[0122] An embodiment of the present application provides a computer device, the computer device comprises the heat prediction device of the expert in the MoE.
[0123] It should be noted that: the heat prediction device of the expert in the MoE provided in the above embodiment is only used as an example to illustrate the prediction of the above-mentioned functional modules when the heat prediction of the expert in the MoE is performed. In actual application, the above-mentioned functions can be distributed to different functional modules to complete, that is, the internal structure of the heat prediction device of the expert in the MoE is predicted into different functional modules to complete all or part of the functions described above. In addition, the heat prediction device of the expert in the MoE provided in the above embodiment and the heat prediction method of the expert in the MoE embodiment belong to the same concept, and the specific implementation process is described in the method embodiment, which will not be repeated here.
[0124] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program to instruct related hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.
[0125] The above description does not limit the embodiments of the present application. Any adjustment, equivalent replacement, improvement, etc. made within the spirit and principles of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.
Claims
1. A method for predicting the hotness of an expert in a MoE, characterized in that, The method comprises: statistically obtaining resource occupation parameters of each expert in the MoE by using a sliding window; determining weight parameters corresponding to each resource occupation parameter according to a model type of the MoE; calculating resource occupation information of each expert according to the resource occupation parameters and the weight parameters; processing the resource occupation information by using an exponential moving average (EMA) model to obtain a heat parameter of each expert; predicting a heat level of each expert according to the heat parameter. 2.The method of claim 1, wherein, The processing of the resource occupation information by using the EMA model to obtain the heat parameter of each expert comprises: Using the formula E t = a x x t + (1 - a) x E t-1 The resource occupation information is calculated to obtain a heat parameter of each expert; Wherein, E t represents the hotness parameter at time t, E t-1 represents the hotness parameter at time t-1, and α represents a smoothing coefficient, x t represents the resource occupation information at time t. 3.The method of claim 2, wherein, The method further comprises: obtaining a heat level change frequency of each expert in the MoE; determining the smoothing coefficient according to the heat level change frequency, wherein the size of the smoothing coefficient is in a positive correlation with the level of the heat level change frequency. 4.The method of claim 1, wherein, The resource occupation parameters comprise at least one of a number of invocations per unit time, a number of tokens processed per unit time, a total service time per unit time, a queue depth, and a video graphics processing unit (vGPU) instance memory occupation ratio of the expert. 5.The method of claim 4, wherein, The statistically obtaining of the resource occupation parameters of each expert in the MoE by using the sliding window comprises: configuring a first sliding window and a second sliding window, the first sliding window representing a fixed time window, and the second sliding window representing a window during which a vGPU instance is bound; statistically obtaining at least one of a number of invocations, a number of tokens processed, a total service time, and a queue depth of each expert in the MoE in the first sliding window; and statistically obtaining a vGPU instance memory occupation ratio in the second sliding window.
6. The method of claim 1 to 5, wherein, The calculating of the resource occupation information of each expert according to the resource occupation parameters and the weight parameters comprises: selecting a target resource occupation parameter from the resource occupation parameters according to the model type of the MoE, and performing weighted calculation on the target resource occupation parameter and the corresponding weight parameter to obtain the resource occupation information of each expert; or performing weighted calculation on all resource occupation parameters and the corresponding weight parameters to obtain the resource occupation information of each expert.
7. A device for predicting the heat of an expert in a MoE, characterized by, The device comprises: a statistical module configured to statistically obtain resource occupation parameters of each expert in the MoE by using a sliding window; a determination module configured to determine weight parameters corresponding to each resource occupation parameter according to a model type of the MoE; a calculation module configured to calculate resource occupation information of each expert according to the resource occupation parameters and the weight parameters; a prediction module configured to process the resource occupation information by using an exponential moving average (EMA) model to obtain a heat parameter of each expert; a prediction module configured to predict a heat level of each expert according to the heat parameter. 8.The expert hotness prediction apparatus in the MoE of claim 7, wherein, The prediction module is further configured to: Using the formula E t = a x x t + (1 - a) x E t-1 The resource occupation information is calculated to obtain a heat parameter of each expert. Wherein, E t represents the heat parameter at time t, E t-1 represents the heat parameter at time t-1, and a represents a smoothing coefficient, x t represents the resource occupation information at time t.
9. A computer-readable storage medium, characterized in that, The storage medium has at least one instruction stored therein, and the at least one instruction is loaded and executed by the processor to implement the heat prediction method of the expert in the MoE according to any one of claims 1 to 6.
10. A computer device, comprising: The computer device comprises the heat prediction device of the expert in the MoE according to claim 7 or 8.