Multi-stage computing power recycling method and system

Abstract

A multi-stage computing power recycling method and system, according to the cascade recommendation system architecture, the computing power recycling strategy is constructed, and the effect loss estimation model is constructed and trained in the offline learning stage to estimate the loss amount of the recommendation effect caused by any request using M+1 computing power recycling strategies, and the request is processed in real time in the online service stage; the output of the recommendation model in different stages can be cached and reused to realize personalized allocation of computing power resources and recycling of computing power resources, so that the recommendation effect remains unchanged, and the improvement effect of improving the resource utilization rate of the recommendation system is achieved.

Description

Technical Field

[0001] This invention relates to a technology in the field of information processing, specifically a multi-stage computing power recovery method and system. Background Technology

[0002] As the diminishing marginal returns of computing power investment in improving performance become increasingly pronounced, existing cascaded recommender system architectures, while optimizing efficiency to the extreme for single traffic flows—responding to user requests and returning results within hundreds or even tens of milliseconds—allocate the same computing resources to multiple traffic flows. Therefore, there is still room for efficiency optimization in existing recommender systems across multiple user traffic flows. In some scenarios, a user may initiate multiple requests to the server within a single session. For each request, the online recommender system runs the entire cascaded recommender model, including recall, coarse ranking, fine ranking, and re-ranking models, to generate high-quality recommendations. However, this fair computing power allocation method ignores the redundancy between requests, meaning the system may return the same results for multiple user requests; it also ignores the different values ​​of different requests, meaning the return on investment (ROI) differs for different requests. Currently, some works are researching intelligent computing power. These works propose elastic queues, elastic channels, and elastic models as means to regulate computing power allocation. Specifically, these control measures include adjusting the candidate set size of the recommendation model, adjusting the number of recall channels, and adjusting the complexity of the ranking model to allocate different computing resources to different requests. However, these works only consider improving system resource utilization from the perspective of adjusting computing resource allocation, without considering new solutions for recovering wasted resources. Summary of the Invention

[0003] To address the aforementioned shortcomings of existing technologies, this invention proposes a multi-stage computing power recovery method and system. By caching the outputs of recommendation models at different stages and reusing them, this invention achieves personalized allocation of computing power resources for requests and recovery and utilization of computing power resources. This invention can achieve the improvement effect of maintaining the recommendation effect while increasing the resource utilization rate of the recommendation system.

[0004] The computing resources mentioned include hardware resources such as CPU, GPU, and storage used in recommendation model inference, as well as abstract resources such as system response latency.

[0005] The aforementioned computing power recovery refers to saving computing power resources consumed in the current model inference by reusing the output results of past models, which is similar in principle to recovering computing power resources used in the past.

[0006] This invention is achieved through the following technical solution:

[0007] This invention relates to a multi-stage computing power recovery method, comprising:

[0008] Step 1: Construct a computing power recovery strategy based on the cascaded recommendation system architecture, specifically including:

[0009] 1.1 Cache the output results of the model at each stage in the cascaded recommendation system architecture.

[0010] The cascaded recommendation system architecture includes: a recall model, a coarse ranking model, a fine ranking model, and a re-ranking model. The output of the previous stage model is the input candidate set of the next stage model.

[0011] The caching mentioned above refers to caching all output results from different stages of all models.

[0012] Preferably, the computational scale is reduced by caching partial output results of all models or caching the output results of all models with different individual values.

[0013] 1.2 After reading the cache of any stage and generating a set of candidate items, output it to the model of the next stage.

[0014] When the cascaded recommendation system architecture includes an M-stage model, an M-stage cache is established, and M+1 computing power recycling strategies are constructed accordingly:

[0015] a. Without reading the cache, a complete cascaded recommendation system architecture is used, and no computing power recycling strategy is used at this time;

[0016] b. Read the cache of the recommendation model at any stage, select the corresponding strategy from M fine-grained computing power recycling strategies, and input the cache as a candidate item set into the recommendation model for the next stage.

[0017] Different computing power recovery strategies have different computing power overhead. For example, in a recommendation system that includes recall, coarse ranking, and fine ranking, when a computing power recovery strategy is adopted that reads the recall cache and uses the cache as a set of candidate items to input into the coarse ranking model, the recall model is skipped and therefore has no computing power overhead. The computing power overhead of the entire system is the sum of the computing power overhead of the coarse ranking and fine ranking models.

[0018] 1.3 Pre-calculate the computing power overhead for different computing power recovery strategies using floating-point operations per second (FLOPS), response time (RT), or number of CPU cores. .

[0019] The present invention preferably uses response delay to represent computing power overhead.

[0020] Step 2, Offline Learning Phase: Construct and train an effect loss prediction model to estimate the potential loss in recommendation performance that might result from applying M+1 computational power recovery strategies to any request. Specifically, this includes:

[0021] 2.1 Use the collected data to train the effect loss prediction model.

[0022] The collected data represents the recommendation effect of different computing power recovery strategies for different user requests, including user characteristics describing static user information and request characteristics describing request redundancy.

[0023] The effect loss prediction model is a deep learning model that includes a gated recurrent unit (GRU) and a multi-expert hybrid model (MMoE), wherein the GRU models user behavior and the MMoE performs multi-task learning.

[0024] The recommendation effect includes, but is not limited to, posterior click count, prior click probability, and posterior user dwell time. This invention uses prior click probability as a metric for the recommendation effect.

[0025] 2.2 The decision-making problem of online computing power recovery strategy is modeled as a constrained optimization problem, and the dual problem of the constrained optimization problem is solved to obtain the performance evaluation of the strategy to improve resource utilization while achieving basically the same recommendation effect.

[0026] The constrained optimization problem mentioned above refers to: The model is constrained as follows: Where: N is the number of recommendation requests, i represents the sequence number of any of the N requests, and j represents the number of the M+1 different computing power recovery strategies, strategy allocation This indicates that algorithm decision request i adopts computing power recovery strategy j, and the effect loss when request i adopts computing power recovery strategy j is: The system requires that the total loss of recommendation system performance does not exceed a threshold B. The computational cost of the pre-calculated computational cost recovery strategy j is as follows: The coefficient of the regularization term; This indicates the regular expression processing for strategy assignment.

[0027] The aforementioned strategy performance evaluation is used to measure the effectiveness of different computing power recovery strategies, specifically: It is a Lagrange multiplier.

[0028] 2.3 After collecting request logs within a given time period, the optimal Lagrange multipliers are obtained by solving the constrained optimization problem using a bisection method. result.

[0029] Step 3, the online service phase, involves real-time request processing. Specifically, incoming user requests and related information are filtered and categorized into initial requests and duplicate requests. For initial requests, the complete recommendation chain is invoked while the output results of each model are cached and the service is terminated. For duplicate requests, the performance evaluation is performed using the effect loss prediction network obtained in 2.1 and the strategy derived in 2.2. The strategy with the highest performance evaluation is selected and the corresponding strategy in the strategy construction section of 1.2 is implemented in the recommendation system chain for cache reading and reuse.

[0030] The user request refers to a user's request for new items, including but not limited to goods, music, and videos, when operating application software or browsing web pages.

[0031] The aforementioned request-related information refers to objective information that can be used to determine the repetition of a request, including but not limited to the time interval since the previous request and the exposure rate of the previously requested item.

[0032] The aforementioned request filtering refers to classifying requests into initial requests or duplicate requests based on request attributes, using either rules or models.

[0033] The requested attributes include, but are not limited to, features such as the time interval between previous requests, the number of clicks in the previous request, and the number of impressions in the previous request.

[0034] The present invention preferably uses a rule-based approach to filter requests.

[0035] The complete recommendation chain refers to the process of generating recommendation results by going through all models in the hierarchical recommendation architecture of recall, coarse ranking, and fine ranking. Technical effect

[0036] This invention significantly improves the resource utilization of recommendation systems through a fine-grained computing power recycling strategy for cache re-recycling, an online optimal recycling decision algorithm derived from constraint optimization modeling, an effect loss prediction model, and a request filtering module. Attached Figure Description

[0037] Figure 1 This is a flowchart of the present invention;

[0038] In the diagram: Red dashed lines indicate that no computing power recovery strategy is used, and the complete recommendation process is performed; Yellow dashed lines indicate that the recall cache is reused, and the cached results are input into the computing power recovery strategy of the coarse ranking model, while the recall model step is skipped; Green dashed lines indicate that the coarse ranking cache is reused, and the cached results are input into the computing power recovery strategy of the fine ranking model, while the recall and coarse ranking model steps are skipped; Blue dashed lines indicate that the fine ranking cache is reused, and the cached results are input into the computing power recovery strategy of the rearrangement model, while the recall, coarse ranking, and fine ranking model steps are skipped.

[0039] Figure 2 This is a schematic diagram of the system structure;

[0040] Figure 3 This is a schematic diagram of the effect loss prediction model;

[0041] Figure 4 This is a schematic diagram illustrating the computing power overhead of an example. Detailed Implementation

[0042] like Figure 2 As shown, this embodiment relates to a computing power recovery framework system, including: a fine-grained computing power recovery strategy module, an online computing power recovery decision module, and a request filtering module. The fine-grained computing power recovery strategy module performs engineering modifications to the cascaded recommendation system architecture, making corresponding engineering link modifications for each constructed computing power recovery strategy. Specifically, this refers to implementing caching and reuse of multi-stage model results. The online computing power recovery decision module calculates the optimal computing power recovery strategy in real time based on requests and user characteristics. The request filtering module filters out requests that can be recovered based on request characteristics, reducing the additional computing power overhead introduced by the effect loss prediction model.

[0043] like Figure 1 As shown, this embodiment illustrates a multi-stage computing power recovery method based on the aforementioned system, including:

[0044] Step 1, the recycling strategy construction section, specifically includes:

[0045] 1.1 Four computing power reclamation strategies are constructed, including no computing power reclamation, reuse of recall cache results, reuse of coarse-ranked cache results, and reuse of fine-ranked cache results.

[0046] 1.2 Utilizing system-statistical response time as computing power overhead .

[0047] Step 2, the offline learning part, specifically includes:

[0048] 2.1 Construction as follows Figure 3 The effect loss prediction model shown includes: an embedding layer, a gated recurrent unit, a gated multi-expert hybrid unit, and a function activation unit. Specifically: the embedding layer performs low-dimensional embedding mapping based on the input user behavior sequence, user information, request information, etc., to obtain a low-dimensional embedding representation of the features; the gated recurrent unit performs sequence modeling based on the low-dimensional embedding sequence information of user behavior, to obtain the low-dimensional embedding representation result of the user behavior sequence; the gated multi-expert hybrid unit performs multi-task learning based on the low-dimensional embeddings of the behavior sequence, user information, and request information, to obtain logits for different computing power recovery strategies; and the function activation unit performs activation function processing based on the logits to obtain the predicted recommendation effect of the request using different computing power recovery strategies.

[0049] The effect loss prediction model employs user-side features such as gender, age, and nationality, request-side features such as the time interval since the last request, the number of impressions and clicks of the previous request, and user historical click behavior. A GRU is used to perform sequence modeling of user historical behavior. Then, the extracted features are concatenated with other features mapped to low-dimensional embeddings and input into a gated multi-expert hybrid module (MMoE) for multi-task learning. The MMoE outputs four values, including... . , , ,at last, The final output obtained after the activation function is the estimated effect loss value.

[0050] 2.2 Set the specific value of threshold B according to actual business needs.

[0051] 2.3 Due to traffic fluctuations in the online system, data is collected every 15 minutes, and the latest Lagrange multiplier is solved using the binary search method for requests arriving in the next 15 minutes.

[0052] Step 3, the online service section, specifically includes: based on the time interval between the request and the previous request, dividing the request into initial requests and repeated requests, calling the complete recommendation link for initial requests while caching the output results of each model and ending the service; for repeated requests, using the effect loss prediction network and policy performance evaluation parameters obtained in 2.1 and 2.2, calculating the performance evaluation of different policies, selecting the policy with the highest score, and implementing the corresponding policy in the policy recycling section of the recommendation link for cache reading and reuse.

[0053] In this embodiment, the request filtering module is specifically configured as follows: requests with a time interval of less than 2 minutes are treated as repeated requests, and requests with a time interval of more than 2 minutes are treated as initial requests.

[0054] Through specific experiments, data logs were collected from the display advertising systems of publicly traded websites. A random sample of users was selected, and their ad requests and system responses were recorded on April 2, 2024 (within one day). This dataset contains approximately 52,000 different users and 205,000 ad requests. Among them, 51% of users initiated two or more requests, and 74% of the requests were duplicate requests.

[0055] Dynamic Programming (DP) solves for a computational resource recovery strategy offline for all requests based on the estimated decline in recommendation performance. The computational resource recovery strategy (ComRecycle) uses binary search to solve for the Lagrange multipliers based on the estimated decline in recommendation performance and assigns a computational resource recovery strategy to each request according to the performance evaluation formula. A fixed-strategy method assigns the same computational resource recovery strategy to all requests. Optimal Dynamic Programming (DP*) solves for a computational resource recovery strategy offline for all requests based on the actual decline in recommendation performance, providing the optimal solution to the original problem. Optimal Computational Resource Recovery (ComRecycle*) uses binary search to solve for the Lagrange multipliers based on the actual decline in recommendation performance and assigns a computational resource recovery strategy to each request according to the performance evaluation formula.

[0056] By adjusting the threshold B, a relationship between performance and computational cost can be established, such as... Figure 4 As shown in Table 1, the recommendation performance of different methods under the same computing power cost, and the computing power cost required to achieve the same recommendation performance for different methods are summarized.

[0057] Table 1. Comparison of resource utilization rates in the embodiments;

[0058] In Table 1, Strategy 1 and Strategy 2 assign computing power recovery strategies of reuse recall or coarse ranking to all requests.

[0059] 1. The curves of DP* and ComRecycle* are almost identical, and the curves of DP and ComRecycle are also almost identical, indicating that ComRecycle can approach the optimal solution;

[0060] 2. ComRecycle* significantly outperforms ComRecycle, indicating that the accuracy of the estimation limits the effectiveness of the framework. On the other hand, this also shows that if the model's estimation is accurate enough, resource utilization can be greatly improved.

[0061] 3. ComRecycle outperforms the fixed strategy method (blue and orange dots in the figure), indicating that ComRecycle improves system resource utilization.

[0062] Based on the experimental results, compared with existing technologies, this invention can provide better recommendation results using the same computing resources, and saves computing resources while providing the same recommendation results. This demonstrates the effectiveness of this invention in improving the utilization rate of computing resources in recommendation systems.

[0063] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A multi-stage computing power recovery method, characterized in that, include: Step 1: Construct a computing power recovery strategy based on the cascaded recommendation system architecture, specifically including: 1.1 Cache the output results of the model at each stage in the cascaded recommendation system architecture; 1.2 After reading the cache of any stage and generating a set of candidate items, output it to the model of the next stage; 1.3 Pre-calculate the computing power overhead for different computing power recovery strategies using floating-point operations per second (FLOPS), response time (RT), or number of CPU cores. ; Step 2, Offline Learning Phase: Construct and train an effect loss prediction model to estimate the potential loss in recommendation performance that might result from applying M+1 computational power recovery strategies to any request. Specifically, this includes: 2.1 Train the performance loss prediction model using the collected data; 2.2 The decision problem of online computing power recovery strategy is modeled as a constrained optimization problem, and the dual problem of the constrained optimization problem is solved to obtain the performance evaluation of the strategy to improve resource utilization under the premise of achieving basically the same recommendation effect; 2.3 After collecting request logs within a given time period, the optimal Lagrange multipliers are obtained by solving the constrained optimization problem using a bisection method. result; Step 3, the online service phase, involves real-time request processing. Specifically, incoming user requests and related information are filtered and categorized into initial requests and duplicate requests. For initial requests, the complete recommendation chain is invoked while the output results of each model are cached and the service is terminated. For duplicate requests, the performance evaluation is performed using the effect loss prediction network obtained in 2.1 and the strategy derived in 2.

2. The strategy with the highest performance evaluation is selected and the corresponding strategy in the strategy construction section of 1.2 is implemented in the recommendation system chain for cache reading and reuse.

2. The multi-stage computing power recovery method according to claim 1, characterized in that, The cascaded recommendation system architecture includes: a recall model, a coarse ranking model, a fine ranking model, and a re-ranking model. The output of the previous stage model is the input candidate set of the next stage model. The caching mentioned refers to: caching all output results of all models at different stages; and reducing the computational scale by caching a portion of the output results of all models or caching a personalized number of output results of all models.

3. The multi-stage computing power recovery method according to claim 2, characterized in that, When the cascaded recommendation system architecture includes an M-stage model, an M-stage cache is established, and M+1 computing power recycling strategies are constructed accordingly: a. Without reading the cache, a complete cascaded recommendation system architecture is used, and no computing power recycling strategy is used at this time; b. Read the cache of the recommendation model at any stage, select the corresponding strategy from M fine-grained computing power recycling strategies, and input the cache as a candidate item set into the recommendation model for the next stage.

4. The multi-stage computing power recovery method according to claim 1, characterized in that, The effect loss prediction model includes an embedding layer, a gated recurrent unit, a gated multi-expert hybrid unit, and a function activation unit. Specifically: the embedding layer performs low-dimensional embedding mapping processing based on the input user behavior sequence, user information, request information, etc., to obtain a low-dimensional embedding representation of the features; the gated recurrent unit performs sequence modeling processing based on the low-dimensional embedding sequence information of user behavior, to obtain the low-dimensional embedding representation result of the user behavior sequence; the gated multi-expert hybrid unit performs multi-task learning processing based on the low-dimensional embedding of the behavior sequence, the low-dimensional embedding of user information, and the low-dimensional embedding of request information, to obtain logits for different computing power recovery strategies; and the function activation unit performs activation function processing based on the logits to obtain the prediction result of the recommendation effect of the request using different computing power recovery strategies.

5. The multi-stage computing power recovery method according to claim 1, characterized in that, The constrained optimization problem mentioned above refers to: The model is constrained as follows: Where: N is the number of recommendation requests, i represents the sequence number of any of the N requests, and j represents the number of the M+1 different computing power recovery strategies, strategy allocation This indicates that algorithm decision request i adopts computing power recovery strategy j, and the effect loss when request i adopts computing power recovery strategy j is: The system requires that the total loss of recommendation system performance does not exceed a threshold B. The computational cost of the pre-calculated computational cost recovery strategy j is as follows: The coefficient of the regularization term; This indicates the regular expression processing for strategy assignment.

6. The multi-stage computing power recovery method according to claim 1, characterized in that, The aforementioned strategy performance evaluation is used to measure the effectiveness of different computing power recovery strategies, specifically: It is a Lagrange multiplier.

7. The multi-stage computing power recovery method according to claim 1, characterized in that, The aforementioned request filtering refers to classifying requests into initial requests or duplicate requests based on request attributes, using either rules or models. The requested attributes include features such as the time interval between previous requests, the number of clicks in the previous request, and the number of impressions in the previous request.

8. The multi-stage computing power recovery method according to claim 1, characterized in that, The complete recommendation chain refers to the process of generating recommendation results by going through all models in the hierarchical recommendation architecture of recall, coarse ranking, and fine ranking.

9. A computing power recovery framework system for implementing the method of any one of claims 1-8, characterized in that, include: The system includes a fine-grained computing power recovery strategy module, an online computing power recovery decision module, and a request filtering module. The fine-grained computing power recovery strategy module performs engineering modifications on the cascaded recommendation system architecture. For each computing power recovery strategy constructed, corresponding engineering link modifications are made. Specifically, this refers to the implementation of caching and reuse of multi-stage model results. The online computing power recovery decision module calculates the optimal computing power recovery strategy in real time based on requests and user characteristics; the request filtering module filters out requests that can be recovered based on request characteristics, reducing the additional computing power overhead introduced by the effect loss prediction model.

Images