Method and apparatus for predicting elastic adjustment of computing power
By training a neural network model with multiple sub-networks and dynamically selecting sub-networks for computation, the problem of fixed computing power in neural network models is solved, achieving efficient elastic adjustment and resource optimization, and supporting green AI online inference.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
- Filing Date
- 2022-09-26
- Publication Date
- 2026-04-24
AI Technical Summary
In existing technologies, the computing power consumption of neural network models is fixed and lacks flexible adjustment, resulting in few scalable levels of computing power and limited room for flexible adjustment.
A neural network model comprising n subnetworks is adopted, which is trained through gradient boosting or hybrid expert algorithms. k subnetworks are dynamically selected for computation, and the computing power coefficient is determined based on the resource scarcity and effect contribution level of the predicted request, thereby achieving elastic adjustment.
It significantly increases the computational power flexibility of neural network models, offers multiple levels of flexible adjustment, and supports efficient resource utilization for green AI online inference.
Smart Images
Figure CN115564022B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to one or more embodiments in the field of computers, and more particularly to a predictive method and apparatus for flexibly adjusting computing power. Background Technology
[0002] A current trend in large-scale artificial intelligence (AI) online inference is the adoption of greener, more carbon-efficient online inference systems, termed green AI inference. Inference can also be referred to as prediction.
[0003] In typical online AI inference, the computational power consumption of the neural network models used is relatively fixed. However, in green AI inference, the computational power consumption of the neural network models is adjustable. For example, the computational power consumption is dynamically adjusted based on the desired effect of the online inference. More computational power is used for requests with ample resources and high impact, while less computational power is used for requests with limited resources and low impact. This allows the neural network model to support this flexible computational power model.
[0004] In existing technologies, there are very few levels of flexibly scalable computing power, and the space for flexibly adjusting computing power is small. Summary of the Invention
[0005] This specification describes a prediction method and apparatus for flexibly adjusting computing power through one or more embodiments. The computing power can be flexibly scaled up and down at multiple levels, and the space for flexibly adjusting computing power is large.
[0006] Firstly, a predictive method for flexibly adjusting computing power is provided, executed through a computing platform deployed with a trained neural network model, the neural network model comprising n sub-networks, where n>2, and the method includes:
[0007] Receive prediction requests, which include the sample to be tested;
[0008] Determine the computing power coefficient allocated to the prediction request, which indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform;
[0009] Based on the computing power coefficient, determine k subnetworks out of the n subnetworks to be used this time;
[0010] The sample to be tested is input into the k sub-networks to obtain the prediction results.
[0011] In one possible implementation, the neural network model is trained in the following manner:
[0012] The gradient boosting ensemble algorithm uses a sample set consisting of multiple labeled training samples to train n sub-networks one by one in a certain order.
[0013] Furthermore, determining k subnetworks out of the n subnetworks used in this instance includes:
[0014] From the n subnetworks, select the first k subnetworks in the order stated.
[0015] Furthermore, the step of training n sub-networks sequentially in a certain order includes:
[0016] With the goal of minimizing the total prediction loss, the first sub-network among the n sub-networks is trained using the sample set;
[0017] By using residual iteration, any second subnetwork after the first subnetwork in the n subnetworks is trained using the sample set.
[0018] Furthermore, any of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object;
[0019] The step of training the first sub-network among the n sub-networks using the sample set includes:
[0020] Input the feature values of any sample user corresponding to multiple dimensions into the first sub-network, and output the predicted click probability of that sample user for the target object through the first sub-network;
[0021] The prediction loss is determined based on the click probability label of the sample user for the target object, the predicted click probability of the sample user for the target object, and the preset loss function.
[0022] The parameters of the first sub-network are adjusted with the goal of minimizing the sum of the predicted losses of each user in the sample set.
[0023] Furthermore, any of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object;
[0024] The step of training any second subnetwork among the n subnetworks using the sample set includes:
[0025] Input the feature values of any sample user corresponding to multiple dimensions into each of the trained sub-networks, and output the probability of each first click of the sample user on the target object through each sub-network.
[0026] The residuals are determined based on the click probability labels of the sample users for the target object and the first click probabilities of the sample users for each target object.
[0027] The parameters of the second sub-network are adjusted using the residual as the fitting target.
[0028] Furthermore, the target object is any object in the candidate object set.
[0029] In one possible implementation, the neural network model is trained in the following manner:
[0030] Based on the Mixture of Experts (MoE) algorithm, n sub-networks are trained using a sample set consisting of multiple labeled training samples. Each sub-network corresponds to an expert network in the MoE algorithm.
[0031] Furthermore, determining k subnetworks out of the n subnetworks used in this instance includes:
[0032] From the n subnetworks, k subnetworks are randomly selected.
[0033] Secondly, a predictive device for flexibly adjusting computing power is provided, disposed on a computing platform, wherein the computing platform deploys a trained neural network model, the neural network model comprising n sub-networks, n>2, and the device comprising:
[0034] A receiving unit is used to receive prediction requests, including the sample to be tested.
[0035] A coefficient determination unit is used to determine the computing power coefficient allocated to the prediction request received by the receiving unit. The computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform.
[0036] The network determination unit is used to determine k sub-networks out of the n sub-networks to be used this time, based on the computing power coefficients determined by the coefficient determination unit.
[0037] The prediction unit is used to input the sample to be tested into the k sub-networks determined by the network determination unit to obtain the prediction result.
[0038] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of the first aspect.
[0039] Fourthly, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of the first aspect.
[0040] The method and apparatus provided in the embodiments of this specification employ a trained neural network model comprising n sub-networks, where n>2. First, a prediction request is received, including a sample to be tested. Then, a computing power coefficient is determined for the prediction request. This computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform. Next, based on the computing power coefficient, k sub-networks from the n sub-networks to be used are determined. Finally, the sample to be tested is input into the k sub-networks to obtain the prediction result. As can be seen from the above, in the embodiments of this specification, n sub-networks are generated during training (n can be customized), and during prediction, only k sub-networks can be dynamically activated as needed (k can be selected from 1 to n). This makes the computing power flexibility of the neural network model much higher than traditional solutions, resulting in more adjustable levels of computing power and a larger space for flexible adjustment of computing power. Attached Figure Description
[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;
[0043] Figure 2 A flowchart of a prediction method for flexibly adjusting computing power according to one embodiment is shown;
[0044] Figure 3 A schematic diagram illustrating the relationship between prediction results and subnetwork outputs according to one embodiment is shown.
[0045] Figure 4 This diagram illustrates the relationship between the prediction results and the subnetwork output results according to another embodiment.
[0046] Figure 5 A schematic block diagram of a prediction device that flexibly adjusts computing power according to one embodiment is shown. Detailed Implementation
[0047] The solution provided in this specification will now be described with reference to the accompanying drawings.
[0048] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification. This implementation scenario involves predicting the elastic adjustment of computing power. (Refer to...) Figure 1 The prediction task is executed through a computing platform, which is equipped with a trained neural network model. The neural network model includes n sub-networks, where n>2. For the current prediction task, all n sub-networks can be used, or some of the n sub-networks can be used. It is understood that when multiple sub-networks are used, the output results of each sub-network are superimposed to obtain the prediction result for the test sample. The superposition can be a simple summation or a weighted summation.
[0049] In the embodiments described in this specification, the number of subnetworks used when performing the current prediction task determines the computing power. The computing power is maximized when all n subnetworks are used, and is recorded as 1. The computing power is minimized when only 1 subnetwork is used, and is recorded as... When using k subnetworks, the computing power is denoted as follows: This allows for flexible computing power by selecting the number of subnetworks to use.
[0050] FLOPS stands for floating-point operations per second. The number of FLOPS can be used to measure the computing power required by a neural network model.
[0051] Elastic computing power: A neural network model requires floating-point calculations in hardware each time it performs inference. If the floating-point calculations required for a single prediction by the neural network model consume N FLOPS, and the value of N can be adjusted from 0 to a certain maximum value Max through parameter tuning, then the neural network model is considered to support elastic computing power.
[0052] Reference Figure 1 The neural network model comprises n subnetworks, thus supporting n levels of scalable computing power. n is customizable; the larger the value of n, the more scalable levels of computing power are available, and the greater the flexibility in adjusting the computing power. For example, when n is 4, the scalable computing power has 4 levels, namely... 1; When the value of n is 5, the scalability of computing power is 5 levels, namely... 1. It is understandable that when n is large enough, the scalability of the computing power of the neural network model can be considered to be basically continuous.
[0053] In the embodiments described in this specification, it is assumed that the computing power of each sub-network is equivalent. The structure of each sub-network may be, but is not limited to, deep neural networks (DNNs). In addition to containing n sub-networks, the neural network model may also contain other network structures, such as processing units that superimpose the output results of each sub-network, or gating units used to determine the weights of each sub-network.
[0054] The aforementioned prediction tasks can be prediction tasks in various scenarios. For example, in a risk control scenario, predicting the risk score of a target user, or in a recommendation scenario, predicting the relevance between the product to be recommended and the target user. These will not be listed one by one here.
[0055] In the embodiments described in this specification, the prediction task can be either offline prediction or online prediction. Online prediction is more typical and has a higher requirement for elastic computing power.
[0056] The embodiments in this specification enable green AI online inference. Green AI online inference refers to technologies where the resource consumption ROI of online AI inference is significantly higher than that of general inference systems. Wherein, ROI = Inference Performance / Inference Cost.
[0057] Figure 2 The flowchart illustrates a predictive method for elastically adjusting computing power according to one embodiment, executed via a computing platform deployed with a trained neural network model comprising n subnetworks, where n>2. This method can be based on... Figure 1 The implementation scenario is shown. For example... Figure 2 As shown, the prediction method for flexibly adjusting computing power in this embodiment includes the following steps: Step 21, receiving a prediction request, including a sample to be tested; Step 22, determining a computing power coefficient allocated to the prediction request, wherein the computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform; Step 23, determining k subnetworks out of n subnetworks to be used this time based on the computing power coefficient; Step 24, inputting the sample to be tested into the k subnetworks to obtain the prediction result. The specific execution method of each of the above steps is described below.
[0058] First, in step 21, a prediction request is received, which includes the sample to be tested. It is understood that the sample to be tested can correspond to an organization, individual, or item, and typically has feature values corresponding to multiple dimensions.
[0059] In the embodiments described in this specification, a prediction request can be considered as a specific prediction task. For example, predicting the risk score of a sample to be tested, predicting the category of a sample to be tested, etc.
[0060] Then, in step 22, a computing power coefficient is determined for the prediction request. This computing power coefficient indicates the ratio of the hardware computing resources allocated to the prediction request to the total hardware computing resources required for the entire neural network model to run on the computing platform. It is understood that this computing power coefficient can be 50% or 80%, etc.
[0061] In this embodiment of the specification, the computing power coefficient allocated to the prediction request can be determined based on at least one of the current resource scarcity level and the effectiveness contribution level of the prediction request. The effectiveness contribution level can correspond to the contribution to a business objective, and can be determined based on the feature values of preset features of the sample to be tested. For example, if the sample to be tested is a user, the preset feature could be the user's membership level or the number of times the user accesses the service.
[0062] In the embodiments described in this specification, hardware computing resources may include, but are not limited to, central processing unit (CPU) processing resources, memory resources, etc.
[0063] Next, in step 23, based on the computing power coefficient, k subnetworks out of the n subnetworks to be used this time are determined. It is understood that k is less than or equal to n.
[0064] The embodiments in this specification are adapted to the training method of the neural network model. K subnetworks can be arbitrarily selected from n subnetworks. Alternatively, not only is there a requirement for the number of subnetworks k used, but it is also necessary to determine which k subnetworks to select.
[0065] In one example, the neural network model is trained as follows:
[0066] The gradient boosting ensemble algorithm uses a sample set consisting of multiple labeled training samples to train n sub-networks one by one in a certain order.
[0067] Furthermore, determining k subnetworks out of the n subnetworks used in this instance includes:
[0068] From the n subnetworks, select the first k subnetworks in the order stated.
[0069] For example, by evaluating the online computing performance using green AI, the computing power elasticity coefficient x% is obtained for this computing task. x% represents the required computing power level. The first k sub-networks are dynamically activated to participate in the prediction, where k = [n*x%] + 1. When n is large enough, it can be considered that the elastic computing power level of the model is basically continuous.
[0070] Furthermore, the step of training n sub-networks sequentially in a certain order includes:
[0071] With the goal of minimizing the total prediction loss, the first sub-network among the n sub-networks is trained using the sample set;
[0072] By using residual iteration, any second subnetwork after the first subnetwork in the n subnetworks is trained using the sample set.
[0073] Furthermore, any of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object;
[0074] The step of training the first sub-network among the n sub-networks using the sample set includes:
[0075] Input the feature values of any sample user corresponding to multiple dimensions into the first sub-network, and output the predicted click probability of that sample user for the target object through the first sub-network;
[0076] The prediction loss is determined based on the click probability label of the sample user for the target object, the predicted click probability of the sample user for the target object, and the preset loss function.
[0077] The parameters of the first sub-network are adjusted with the goal of minimizing the sum of the predicted losses of each user in the sample set.
[0078] Furthermore, any of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object;
[0079] The step of training any second subnetwork among the n subnetworks using the sample set includes:
[0080] Input the feature values of any sample user corresponding to multiple dimensions into each of the trained sub-networks, and output the probability of each first click of the sample user on the target object through each sub-network.
[0081] The residuals are determined based on the click probability labels of the sample users for the target object and the first click probabilities of the sample users for each target object.
[0082] The parameters of the second sub-network are adjusted using the residual as the fitting target.
[0083] For example, the general training process includes: parameter initialization; iteratively training the neural network model M times, where M is a hyperparameter that can be adjusted for sufficient training; each training iteration of the neural network model includes training each of the n sub-networks, numbered from 0 to n-1. Assuming the current training is of sub-network k, training sub-network k involves: fixing the parameters of sub-networks numbered from 0 to k-1, and training sub-network k through backpropagation with the residuals of yf(Sub DNN0,...,Sub DNN k-1) as the target. It can be understood that y represents the sample label, and f(Sub DNN0,...,Sub DNN k-1) represents the prediction result obtained based on the first k-1 sub-networks.
[0084] Furthermore, the target object is any object in the candidate object set. It is understood that this example can correspond to a recommendation scenario.
[0085] In another example, the neural network model is trained as follows:
[0086] Based on the Mixture of Experts (MoE) algorithm, n sub-networks are trained using a sample set consisting of multiple labeled training samples. Each sub-network corresponds to an expert network in the MoE algorithm.
[0087] Furthermore, determining k subnetworks out of the n subnetworks used in this instance includes:
[0088] From the n subnetworks, k subnetworks are randomly selected.
[0089] In this example, the training of each sub-network is relatively uniform, and there is no polarization. In other words, the prediction effect of each sub-network is relatively uniform, and there is no situation where the prediction effects of different sub-networks differ greatly.
[0090] Finally, in step 24, the sample to be tested is input into the k sub-networks to obtain the prediction result. It can be understood that the prediction result can be obtained by superimposing the outputs of the k sub-networks.
[0091] Figure 3 This diagram illustrates the relationship between prediction results and subnetwork outputs according to one embodiment. (Refer to...) Figure 3The n subnetworks are trained using the aforementioned gradient boosting ensemble algorithm. The prediction result can be obtained by summing the outputs of the k subnetworks. For example, if k is 2, and the k subnetworks are subnetwork 0 and subnetwork 1, with subnetwork 0 having an output of 0.8 and subnetwork 1 having an output of 0.1, then the prediction result is 0.9 by summing 0.8 and 0.1.
[0092] Figure 4 A schematic diagram illustrating the relationship between prediction results and subnetwork outputs according to another embodiment is shown. (Refer to...) Figure 4 The n subnetworks are trained using the aforementioned MoE algorithm. The prediction result is obtained by weighted summation of the outputs of the k subnetworks. The weights of each subnetwork are obtained through a gating unit. If a subnetwork has a weight of 0, it is not used in this prediction. For example, if k is 2, and the k subnetworks are subnetwork 0 and subnetwork n-1, with subnetwork 0 having an output of 0.8 and a weight of a, and subnetwork n-1 having an output of 0.1 and a weight of b, then by weighted summation of the outputs of the two subnetworks (0.8*a + 0.1*b), the prediction result is 0.8a + 0.1b.
[0093] The method provided in the embodiments of this specification employs a trained neural network model comprising n sub-networks, where n>2. First, a prediction request is received, including a sample to be tested. Then, a computing power coefficient is determined for the prediction request. This computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform. Next, based on the computing power coefficient, k sub-networks from the n sub-networks to be used are determined. Finally, the sample to be tested is input into the k sub-networks to obtain the prediction result. As can be seen from the above, in the embodiments of this specification, n sub-networks are generated during training (n can be customized), and during prediction, only k sub-networks can be dynamically activated as needed (k can be selected from 1 to n). This makes the computing power flexibility of the neural network model much higher than traditional solutions, resulting in more adjustable levels of computing power and a larger space for flexible adjustment of computing power.
[0094] According to another embodiment, a predictive device for flexibly adjusting computing power is also provided, disposed on a computing platform, the computing platform having a trained neural network model comprising n sub-networks, n>2, the device being used to perform the methods provided in the embodiments of this specification. Figure 5 A schematic block diagram of a prediction apparatus that flexibly adjusts computing power according to one embodiment is shown. Figure 5 As shown, the device 500 includes:
[0095] Receiving unit 51 is used to receive prediction requests, including the sample to be tested;
[0096] The coefficient determination unit 52 is used to determine the computing power coefficient allocated to the prediction request received by the receiving unit 51. The computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform.
[0097] The network determination unit 53 is used to determine k sub-networks out of the n sub-networks used this time based on the computing power coefficients determined by the coefficient determination unit 52.
[0098] The prediction unit 54 is used to input the sample to be tested into the k sub-networks determined by the network determination unit 53 to obtain the prediction result.
[0099] Alternatively, as an embodiment, the neural network model is trained in the following manner:
[0100] The gradient boosting ensemble algorithm uses a sample set consisting of multiple labeled training samples to train n sub-networks one by one in a certain order.
[0101] Furthermore, the network determining unit 53 is specifically used to select the first k sub-networks from the n sub-networks in the order stated above.
[0102] Furthermore, the step of training n sub-networks sequentially in a certain order includes:
[0103] With the goal of minimizing the total prediction loss, the first sub-network among the n sub-networks is trained using the sample set;
[0104] By using residual iteration, any second subnetwork after the first subnetwork in the n subnetworks is trained using the sample set.
[0105] Furthermore, any of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object;
[0106] The step of training the first sub-network among the n sub-networks using the sample set includes:
[0107] Input the feature values of any sample user corresponding to multiple dimensions into the first sub-network, and output the predicted click probability of that sample user for the target object through the first sub-network;
[0108] The prediction loss is determined based on the click probability label of the sample user for the target object, the predicted click probability of the sample user for the target object, and the preset loss function.
[0109] The parameters of the first sub-network are adjusted with the goal of minimizing the sum of the predicted losses of each user in the sample set.
[0110] Furthermore, any of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object;
[0111] The step of training any second subnetwork among the n subnetworks using the sample set includes:
[0112] The feature values of any sample user corresponding to multiple dimensions are input into each of the trained sub-networks, and each sub-network outputs the probability of the sample user's first click on the target object.
[0113] The residuals are determined based on the click probability labels of the sample users for the target object and the first click probabilities of the sample users for each target object.
[0114] The parameters of the second sub-network are adjusted using the residual as the fitting target.
[0115] Furthermore, the target object is any object in the candidate object set.
[0116] Alternatively, as an embodiment, the neural network model is trained in the following manner:
[0117] Based on the hybrid expert MoE algorithm, n sub-networks are trained using a sample set consisting of multiple labeled training samples. Each sub-network corresponds to an expert network in the MoE algorithm.
[0118] Furthermore, determining k subnetworks out of the n subnetworks used in this instance includes:
[0119] From the n subnetworks, k subnetworks are randomly selected.
[0120] The apparatus provided in this embodiment employs a trained neural network model comprising n sub-networks, where n>2. First, the receiving unit 51 receives a prediction request, including a sample to be tested. Then, the coefficient determination unit 52 determines the computing power coefficient allocated to the prediction request. This computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform. Next, the network determination unit 53 determines k sub-networks from the n sub-networks to be used based on the computing power coefficient. Finally, the prediction unit 54 inputs the sample to be tested into the k sub-networks to obtain the prediction result. As can be seen from the above, in this embodiment, n sub-networks are generated during training (n can be customized), and during prediction, only k sub-networks can be dynamically activated as needed (k can be selected from 1 to n). This results in a much higher computing power flexibility than traditional solutions, allowing for more flexibly adjustable computing power levels and a larger space for adjusting computing power.
[0121] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 The method described.
[0122] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, it implements a combination... Figure 2 The method described.
[0123] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.
[0124] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.
Claims
1. A predictive method for flexibly adjusting computing power, executed through a computing platform, the computing platform being deployed with a trained neural network model, the neural network model comprising n sub-networks, n>2, each sub-network having equivalent computing power, the method comprising: Receive prediction requests, which include the sample to be tested; Determine the computing power coefficient allocated to the prediction request, which indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform; Based on the computing power coefficient, determine k subnetworks out of the n subnetworks to be used this time; The sample to be tested is input into the k sub-networks to obtain the prediction results.
2. The method as described in claim 1, wherein, The neural network model is trained in the following manner: The gradient boosting ensemble algorithm uses a sample set consisting of multiple labeled training samples to train n sub-networks one by one in a certain order.
3. The method as described in claim 2, wherein, The process of determining k subnetworks out of the n subnetworks used in this instance includes: From the n subnetworks, select the first k subnetworks in the order stated.
4. The method of claim 2, wherein, The process of training n sub-networks sequentially in a certain order includes: With the goal of minimizing the total prediction loss, the first sub-network among the n sub-networks is trained using the sample set; By using residual iteration, any second subnetwork after the first subnetwork in the n subnetworks is trained using the sample set.
5. The method of claim 4, wherein, Each of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object; The step of training the first sub-network among the n sub-networks using the sample set includes: Input the feature values of any sample user corresponding to multiple dimensions into the first sub-network, and output the predicted click probability of that sample user for the target object through the first sub-network; The prediction loss is determined based on the click probability label of the sample user for the target object, the predicted click probability of the sample user for the target object, and the preset loss function. The parameters of the first sub-network are adjusted with the goal of minimizing the sum of the predicted losses of each user in the sample set.
6. The method of claim 4, wherein, Each of the training samples includes the feature values of the sample user corresponding to multiple dimensional features, and the click probability label of the sample user for the target object; The step of training any second subnetwork among the n subnetworks using the sample set includes: Input the feature values of any sample user corresponding to multiple dimensions into each of the trained sub-networks, and output the probability of each first click of the sample user on the target object through each sub-network. The residuals are determined based on the click probability labels of the sample users for the target object and the first click probabilities of the sample users for each target object. The parameters of the second sub-network are adjusted using the residual as the fitting target.
7. The method of claim 5 or 6, wherein, The target object is any object in the set of candidate objects.
8. The method of claim 1, wherein, The neural network model is trained in the following manner: Based on the hybrid expert MoE algorithm, n sub-networks are trained using a sample set consisting of multiple labeled training samples. Each sub-network corresponds to an expert network in the MoE algorithm.
9. The method of claim 8, wherein, The process of determining k subnetworks out of the n subnetworks used in this instance includes: From the n subnetworks, k subnetworks are randomly selected.
10. A predictive device for flexibly adjusting computing power, disposed on a computing platform, the computing platform deploying a trained neural network model, the neural network model comprising n sub-networks, n>2, each sub-network having equivalent computing power, the device comprising: A receiving unit is used to receive prediction requests, including the sample to be tested. A coefficient determination unit is used to determine the computing power coefficient allocated to the prediction request received by the receiving unit. The computing power coefficient indicates the ratio of the hardware computing power resources allocated to the prediction request to the total hardware computing power resources required for the entire neural network model to run on the computing platform. The network determination unit is used to determine k sub-networks out of the n sub-networks to be used this time, based on the computing power coefficients determined by the coefficient determination unit. The prediction unit is used to input the sample to be tested into the k sub-networks determined by the network determination unit to obtain the prediction result.
11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-9.
12. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-9.
Citation Information
Patent Citations
Neural network scheduling method and system
CN109165729A
Neural network training and deployment method, text translation method and related products
CN111985624A