Parameter adjustment methods, devices, recommendation systems, electronic equipment, and storage media

Abstract

This application provides a parameter adjustment method, apparatus, recommendation system, electronic device, and storage medium, which solves the problems that manual parameter adjustment in recommendation systems relies too heavily on rule design and human experience, resulting in low adjustment efficiency, high maintenance costs, and difficulty in achieving global optimization when the number of parameters in a parameter combination increases. The parameter adjustment method includes: responding to a parameter adjustment request for a sub-module in the recommendation system; obtaining feedback indicators of N sets of parameter combinations from the previous round in the previous test cycle, wherein the feedback indicator of each set of parameter combinations is the execution behavior indicator of each test object using the parameter combination, and the N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm; and generating a new set of N sets of parameter combinations based on the feedback indicators of the N sets of parameter combinations from the previous round and the preset evolutionary strategy algorithm.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to parameter adjustment methods, apparatus, recommendation systems, electronic devices and storage media. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] The emergence and widespread adoption of the internet have brought users a wealth of information, satisfying their information needs in the information age. Recommendation systems are personalized information recommendation systems that recommend information (such as news, videos, and advertisements) and items that users are interested in, based on their information needs and online behavior. Unlike traditional search engines, recommendation systems study user interests and preferences, performing personalized calculations to discover user interests and guide users to find their own information needs. The recommendation process of a recommendation system generally includes content retrieval, filtering, and ranking. In the retrieval and ranking stages, machine learning algorithms from artificial intelligence are typically used to score the users and content to be recommended, thus filtering the recommended content.

[0004] In recommender systems, rule-based control or algorithmic models are typically used in sub-modules. Each sub-module involves multiple parameter combinations. For example, a ranking module usually uses multiple ranking sub-models, and the ranking is based on a weighted fusion of the scores given by each ranking sub-model. The weights of each ranking sub-model constitute a set of parameters for the ranking module. Similarly, a filtering module might use multiple threshold combinations, and a re-ranking module might use combinations of diversity control parameters. During the use of a recommender system, it is usually necessary to adjust the parameter combinations in each sub-module to find optimal parameters and achieve better recommendation results. In related technologies, the adjustment of parameter combinations in the sub-modules of a recommender system is usually done manually, setting each parameter based on human experience. However, this manual adjustment method relies heavily on rule design and human experience, resulting in low adjustment efficiency, high maintenance costs, and difficulty in achieving global optimization as the number of parameters in the combination increases. Summary of the Invention

[0005] To address the problems in the background art, embodiments of this application provide a parameter adjustment method, apparatus, recommendation system, electronic device, and storage medium.

[0006] On one hand, embodiments of this application provide a parameter adjustment method, including:

[0007] Responding to parameter adjustment requests for submodules in the recommendation system;

[0008] Obtain feedback metrics for N sets of parameter combinations from the previous round within the previous test cycle. The feedback metrics for each set of parameter combinations are the execution behavior metrics of each test object using the corresponding parameter combination. The N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm.

[0009] The new round of N parameter combinations is generated based on the feedback indicators of the previous round's N parameter combinations and the preset evolutionary strategy algorithm.

[0010] In one possible implementation, a new set of N parameter combinations is generated based on the feedback metrics of the previous round's N parameter combinations and the preset evolutionary strategy algorithm, specifically including:

[0011] Based on the feedback indicators and preset return functions of the N parameter combinations from the previous round, determine the return values ​​corresponding to each of the N parameter combinations from the previous round.

[0012] Arrange the N sets of parameters from the previous round in descending order of return value, and select a preset proportion of parameter combinations from front to back.

[0013] Calculate the mean and variance of each parameter in each selected parameter combination;

[0014] Based on the preset evolutionary strategy algorithm, the algorithm iterates according to the preset number of iterations to generate a new round of N sets of parameter combinations that satisfy the Gaussian distribution of the mean and variance of each parameter.

[0015] In one possible implementation, the return value corresponding to each of the N parameter combinations in the previous round is determined based on the feedback indicators and preset return functions of the N parameter combinations in the previous round, specifically including:

[0016] Each feedback index of each parameter combination from the previous round is sorted.

[0017] Based on the number of positions of each feedback indicator in each set of parameters and the weight of each feedback indicator, the return value corresponding to each set of parameters in the previous round is determined.

[0018] In one possible implementation, the preset reward function is:

[0019]

[0020] Among them, reward j This represents the reward for the j-th parameter combination in the previous round, where j = 1 to N;

[0021] ω iThis represents the weight corresponding to the i-th feedback indicator in the j-th parameter combination, where i = 1 to m, and m is the number of feedback indicators.

[0022] l i This indicates the number of positions of the i-th feedback index in the j-th parameter combination.

[0023] In one possible implementation, after calculating the variance of each parameter in each selected parameter combination, the method further includes:

[0024] The offset variance corresponding to the set perturbation variance, the set maximum perturbation round, and the current iteration round are used to determine the variance.

[0025] The offset variance is added to the variance of each parameter respectively.

[0026] In one possible implementation, the offset variance corresponding to the variance is determined based on a set disturbance variance and a set maximum disturbance round, specifically including:

[0027] The offset variance corresponding to the variance is determined by the following formula:

[0028]

[0029] Wherein, σ′ represents the offset variance corresponding to the variance;

[0030] p represents the current iteration round;

[0031] Δt represents the set maximum disturbance round;

[0032] Δσ represents the set disturbance variance.

[0033] On one hand, embodiments of this application provide a parameter adjustment device, including:

[0034] A response unit is used to respond to parameter adjustment requests for submodules in the recommendation system;

[0035] The acquisition unit is used to acquire the feedback indicators of the N sets of parameter combinations in the previous round within the previous test cycle. The feedback indicator of each set of parameter combinations is the execution behavior indicator of each test object using the corresponding parameter combination. The N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm.

[0036] The generation unit is used to generate a new set of N parameter combinations based on the feedback indicators of the N parameter combinations from the previous round and the preset evolutionary strategy algorithm.

[0037] In one possible implementation, the generating unit is specifically used for:

[0038] Based on the feedback indicators and preset return functions of the N parameter combinations from the previous round, determine the return values ​​corresponding to each of the N parameter combinations from the previous round.

[0039] Arrange the N sets of parameters from the previous round in descending order of return value, and select a preset proportion of parameter combinations from front to back.

[0040] Calculate the mean and variance of each parameter in each selected parameter combination;

[0041] Based on the preset evolutionary strategy algorithm, the algorithm iterates according to the preset number of iterations to generate a new round of N sets of parameter combinations that satisfy the Gaussian distribution of the mean and variance of each parameter.

[0042] In one possible implementation, the generating unit is specifically used for:

[0043] Each feedback index of each parameter combination from the previous round is sorted.

[0044] Based on the number of positions of each feedback indicator in each set of parameters and the weight of each feedback indicator, the return value corresponding to each set of parameters in the previous round is determined.

[0045] In one possible implementation, the preset reward function is:

[0046]

[0047] Among them, reward j This represents the reward for the j-th parameter combination in the previous round, where j = 1 to N;

[0048] ω i This represents the weight corresponding to the i-th feedback indicator in the j-th parameter combination, where i = 1 to m, and m is the number of feedback indicators.

[0049] l i This indicates the number of positions of the i-th feedback index in the j-th parameter combination.

[0050] In one possible implementation, the device further includes:

[0051] The determining unit is used to determine the offset variance corresponding to the variance after calculating the variance of each parameter of each selected parameter combination, based on the set perturbation variance, the set maximum perturbation round, and the current iteration round.

[0052] The processing unit is used to add the offset variance to the variance of each parameter respectively.

[0053] In one possible implementation, the determining unit is specifically used for:

[0054] The offset variance corresponding to the variance is determined by the following formula:

[0055]

[0056] Wherein, σ′ represents the offset variance corresponding to the variance;

[0057] p represents the current iteration round;

[0058] Δt represents the set maximum disturbance round;

[0059] Δσ represents the set disturbance variance.

[0060] On the one hand, embodiments of this application provide a recommendation system, which includes several sub-modules. When any sub-module needs to adjust its parameters, it uses N sets of parameter combinations generated by the parameter adjustment method described in this application to recommend information.

[0061] On one hand, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the parameter adjustment method described in this application.

[0062] On one hand, embodiments of this application provide a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the steps in the parameter adjustment method described in this application.

[0063] The beneficial effects of the embodiments of this application are as follows:

[0064] The parameter adjustment method, apparatus, recommendation system, electronic device, and storage medium provided in this application embodiment respond to parameter adjustment requests for sub-modules in the recommendation system by obtaining feedback indicators of N sets of parameter combinations from the previous round in the previous test cycle. Each set of parameter combinations represents the execution behavior indicator of each test object using the corresponding parameter combination. These N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm. The parameter adjustment device generates a new set of N sets of parameter combinations based on the feedback indicators of the previous round's N sets of parameter combinations and the preset evolutionary strategy algorithm. Compared to existing technologies, this application automatically generates parameter combinations for sub-modules in the recommendation system by combining the preset evolutionary strategy algorithm, achieving automatic parameter adjustment in the recommendation system without relying on human experience, thus improving parameter adjustment efficiency. Furthermore, since each set of parameter combinations generated in each round is assigned to different groups of test objects for testing, and a new set of parameter combinations is generated based on the execution behavior indicators corresponding to different parameter combinations used by each group of test objects and the preset evolutionary strategy algorithm, the generated new set of parameter combinations can be further optimized, thereby enabling the recommendation system to achieve better recommendation results. Attached Figure Description

[0065] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which:

[0066] Figure 1 This is a schematic diagram illustrating an application scenario of the parameter adjustment method provided in the embodiments of this application;

[0067] Figure 2 A schematic diagram illustrating the implementation process of the parameter adjustment method provided in this application embodiment;

[0068] Figure 3 A schematic diagram illustrating the implementation process for generating N sets of parameter combinations for a new round, provided in an embodiment of this application;

[0069] Figure 4 A schematic diagram of the implementation process for determining the return value corresponding to each of the N sets of parameter combinations in the previous round, provided for an embodiment of this application;

[0070] Figure 5 This is a schematic diagram of the parameter adjustment device provided in the embodiments of this application;

[0071] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0072] The principles and spirit of this application will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this application, and are not intended to limit the scope of this application in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0073] Those skilled in the art will recognize that embodiments of this application can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0074] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0075] For ease of understanding, the technical terms used in the embodiments of this application are explained as follows:

[0076] 1. CEM (Cross Entropy Method) Algorithm: Evolutionary Strategies (ES) is an algorithm based on evolutionary theory. As a method for solving parameter optimization problems, ES imitates the principle of biological evolution, assuming that no matter what changes occur in genes, the resulting traits always follow a Gaussian distribution with zero mean and a certain variance.

[0077] The CEM algorithm is a Monte Carlo and evolutionary strategy-based algorithm. Its core is a parameter optimization process, a search algorithm based on parameter perturbation. It introduces reasonable perturbations into the parameter space, searches for and selects better parameter combinations within these perturbations, and then uses cross-entropy to guide parameter updates, making the perturbation directions increasingly closer to the desired optimization direction. Like many supervised learning algorithms, CEM uses cross-entropy to guide the update direction, but it is a gradient-free method, solving the optimization problem from an evolutionary perspective.

[0078] 2. Reward function: A function used to specify and quantify the task objective to be optimized.

[0079] The principles and spirit of this application will be explained in detail below with reference to several representative embodiments. Invention Overview

[0081] The emergence and widespread adoption of the internet have brought users a wealth of information, satisfying their information needs in the information age. Recommendation systems are personalized information recommendation systems that recommend information (such as news, videos, and advertisements) and items that users are interested in, based on their information needs and online behavior. Unlike traditional search engines, recommendation systems study user interests and preferences, performing personalized calculations to discover user interests and guide users to find their own information needs. The recommendation process of a recommendation system generally includes content retrieval, filtering, and ranking. In the retrieval and ranking stages, machine learning algorithms from artificial intelligence are typically used to score the users and content to be recommended, thus filtering the recommended content.

[0082] In recommender systems, rule-based control or algorithmic models are typically used in sub-modules. Each sub-module involves multiple parameter combinations. For example, a ranking module usually uses multiple ranking sub-models, and the ranking is based on a weighted fusion of the scores given by each ranking sub-model. The weights of each ranking sub-model constitute a set of parameters for the ranking module. Similarly, a filtering module might use multiple threshold combinations, and a re-ranking module might use combinations of diversity control parameters. During the use of a recommender system, it is usually necessary to adjust the parameter combinations in each sub-module to find optimal parameters and achieve better recommendation results. In related technologies, the adjustment of parameter combinations in the sub-modules of a recommender system is usually done manually, setting each parameter based on human experience. However, this manual adjustment method relies heavily on rule design and human experience, resulting in low adjustment efficiency, high maintenance costs, and difficulty in achieving global optimization as the number of parameters in the combination increases.

[0083] To address the aforementioned issues, this application proposes a parameter adjustment method, apparatus, electronic device, and storage medium. The parameter adjustment device, in response to a parameter adjustment request for a sub-module in a recommendation system, obtains feedback indicators for N parameter combinations from the previous round within the previous testing period. Each parameter combination's feedback indicator represents the execution behavior indicator for each test subject using that parameter combination. These N parameter combinations are generated based on a preset evolutionary strategy algorithm. The parameter adjustment device generates a new set of N parameter combinations based on the feedback indicators from the previous round's N parameter combinations and the preset evolutionary strategy algorithm. Compared to existing technologies, this application automatically generates parameter combinations for sub-modules in the recommendation system using a preset evolutionary strategy algorithm, achieving automatic parameter adjustment within the recommendation system. This eliminates reliance on human experience, improving parameter adjustment efficiency. Furthermore, since each generated parameter combination is assigned to different test subjects for testing, and a new set of parameter combinations is generated based on the execution behavior indicators for each test subject using different parameter combinations and the preset evolutionary strategy algorithm, the generated new set of parameter combinations can be further optimized, leading to better recommendation performance for the recommendation system.

[0084] After introducing the basic principles of this application, the various non-limiting embodiments of this application will be described in detail below.

[0085] Application Scenarios Overview

[0086] refer to Figure 1This is a schematic diagram illustrating an application scenario of the parameter adjustment method provided in this application embodiment. The application scenario may include a recommendation system 101, a parameter tuning device 102, and a parameter server 103. The recommendation system 101 may include multiple sub-modules, such as a ranking module, a filtering module, and a re-ranking module. The ranking module typically uses multiple ranking sub-models, and the ranking is based on a weighted fusion of the scores estimated by each ranking sub-model. The weights corresponding to each ranking sub-model constitute a set of parameters for the ranking module. Similarly, multiple threshold combinations in the filtering module constitute a set of parameters for the filtering module, and the diversity control parameter combinations in the re-ranking module constitute a set of parameters for the re-ranking module. The parameter adjustment method provided in this application embodiment involves the parameter tuning device 102 adjusting the parameter combinations in any sub-module of the recommendation system 101. The parameter tuning device 102 can be a server or a terminal device; this application embodiment does not limit this. The parameter server 103 stores the parameter combinations generated by the parameter tuning device 102 for each round in the sub-modules of the recommendation system 101. If any submodule in the recommendation system 101 sends a parameter adjustment request to the parameter tuning device 102, the parameter tuning device 102 responds to the request by obtaining the feedback indicators of the N sets of parameter combinations corresponding to the submodule in the previous test cycle. Based on the feedback indicators of the N sets of parameter combinations in the previous cycle and the preset evolutionary strategy algorithm, it automatically generates a new set of N sets of parameter combinations for the next cycle and sends the generated new set of N sets of parameter combinations to the parameter server 103 for storage. The recommendation system 101 loads the new set of N sets of parameter combinations from the parameter server 103 and sets the N sets of parameter combinations... Parameter combinations are assigned to N groups of test subjects for testing in the current testing period. Recommendation system 101 records the execution behavior metrics corresponding to the parameter combinations used by each group of test subjects in the current testing period. These execution behavior metrics may include, but are not limited to, the following metrics: reading time, user click-through rate, user like rate, user favorite rate, and whether a user commented. Recommendation system 101 feeds back the execution behavior metrics corresponding to the parameter combinations used by each group of test subjects to the parameter tuning device 102 (or the parameter tuning device 102 actively obtains them from recommendation system 101 when it needs to generate the parameter combinations for the next round). In this embodiment, the preset evolutionary strategy algorithm may be, but is not limited to, the CEM algorithm; this embodiment does not limit this.

[0087] Exemplary methods

[0088] The parameter adjustment method according to the exemplary embodiments of this application will be described below in conjunction with the above application scenarios. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. On the contrary, the embodiments of this application can be applied to any applicable scenario.

[0089] refer to Figure 2 This application provides a parameter adjustment method that can be applied to... Figure 1 The parameter adjustment device 102 shown may specifically include the following steps:

[0090] S21. Responding to a parameter adjustment request for a submodule in the recommendation system.

[0091] In practice, the recommendation system sends parameter adjustment requests for any sub-module of the recommendation system to the parameter tuning device as needed, and the parameter tuning device responds to the parameter adjustment request. For example, the recommendation system sends a parameter adjustment request to the parameter tuning device for the ranking module. Assume that the ranking module contains the following three ranking sub-models: a prediction model for whether a click is made, a prediction model for the duration of a click, and a prediction model for whether a like is given. The weights of these three ranking sub-models are α1, α2, and α3, respectively. The final ranking score of the ranking module is obtained by linearly weighting these three ranking sub-models and their corresponding weights. The weight combination of these three ranking sub-models, α1, α2, and α3, is the set of parameter combinations requested for adjustment.

[0092] S22. Obtain feedback metrics for the N sets of parameter combinations from the previous round within the previous test cycle.

[0093] In practice, the recommendation system pre-sets test cycles, selects visitor groups with the same or similar composition as test subjects, and randomly divides the test subjects into N groups. In each test cycle, these N groups of test subjects are tested using the N sets of parameter combinations currently generated by the parameter tuning device. That is, one group of test subjects corresponds to one set of parameter combinations. The recommendation system records the execution behavior indicators corresponding to the corresponding parameter combinations assigned to each group of test subjects in each test cycle. These indicators are used by the parameter tuning device to generate a new round of N sets of parameter combinations, which are then used for testing in the next test cycle.

[0094] In specific implementation, when the parameter tuning device receives a parameter adjustment request for a sub-module in the recommendation system, it can proactively obtain feedback metrics for N sets of parameter combinations from the previous round of the previous test period for that sub-module from the recommendation system. Alternatively, the recommendation system can send the feedback metrics for N sets of parameter combinations from the previous round of the previous test period for that sub-module to the parameter tuning device. The feedback metrics for each parameter combination are the execution behavior metrics corresponding to the respective parameter combination used by each test object. The N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm. The feedback metrics may include, but are not limited to, the following metrics: reading time, user click-through rate, user like rate, user collection rate, and whether a user comments, etc. This application embodiment does not limit these metrics.

[0095] S23. Generate a new set of N parameter combinations based on the feedback indicators of the previous round's N parameter combinations and the preset evolution strategy algorithm.

[0096] In specific implementation, the parameter tuning device generates a new set of N parameter combinations for the sub-module based on the feedback indicators of the previous round of N parameter combinations for the sub-module and the preset evolution strategy algorithm. The value of N can be set as needed, and this application embodiment does not limit it.

[0097] As one possible implementation method, it can be carried out according to... Figure 3 The process shown generates N sets of parameter combinations for a new round, including the following steps:

[0098] S231. Based on the feedback indicators and preset return functions of the N parameter combinations in the previous round, determine the return values ​​corresponding to each of the N parameter combinations in the previous round.

[0099] In specific implementation, it can be done according to the following: Figure 4 The process shown determines the reward value corresponding to each of the N parameter combinations in the previous round, including the following steps:

[0100] S2311. Sort each feedback index for each parameter combination in the previous round.

[0101] In practice, the parameter tuning device sorts each performance indicator corresponding to the previous round's parameter combination for each test group in descending order, determining the digit of each feedback indicator for each parameter combination from the previous round. Assuming N=3 (three parameter combinations), and the feedback indicators include user reading time, user click-through rate, and user collection rate, the first test group uses the first parameter combination, the second test group uses the second parameter combination, and the third test group uses the third parameter combination. The reading time of a group of test subjects can be characterized by the average reading time of all test subjects in that group for the content recommended by the recommendation system. The click-through rate of a group of test subjects can be calculated as follows: the ratio of the number of clicks made by all test subjects in that group for the content recommended by the recommendation system to the total number of recommended content. For example, if the recommendation system recommends 100 articles and the test subjects in that group click on 10 of them, the click-through rate is 10 / 100 = 0.1. Similarly, the collection rate of a group of test subjects can be calculated as follows: the ratio of the number of collections made by all test subjects in that group for the content recommended by the recommendation system to the total number of recommended content. Assuming the first group of test subjects uses the first set of parameter combinations and the corresponding user reading time is 15 seconds, the second group of test subjects uses the second set of parameter combinations and the corresponding user reading time is 16 seconds, and the third group of test subjects uses the third set of parameter combinations and the corresponding user reading time is 17 seconds, then the user reading times of each group of test subjects using the corresponding parameter combinations in descending order of user reading time are: 17 seconds (third set of parameter combinations), 16 seconds (second set of parameter combinations), and 15 seconds (first set of parameter combinations). The number of digits in the permutation of the user reading times corresponding to the first set of parameter combinations, the second set of parameter combinations, and the third set of parameter combinations are 3, 2, and 1, respectively. Assuming the first group of test subjects has a click-through rate (CTR) of 0.3 using the first parameter combination, the second group has a CTR of 0.1 using the second parameter combination, and the third group has a CTR of 0.2 using the third parameter combination, then the CTRs of each group of test subjects using the corresponding parameter combinations, ranked from highest to lowest, are: 0.3 (first parameter combination), 0.2 (third parameter combination), and 0.1 (second parameter combination). The number of digits in the CTR ranking for the first, second, and third parameter combinations are 1, 3, and 2, respectively.Assuming the first group of test subjects has a user collection rate of 0.2 when using the first set of parameter combinations, the second group of test subjects has a user collection rate of 0.4 when using the second set of parameter combinations, and the third group of test subjects has a user collection rate of 0.5 when using the third set of parameter combinations, then the user collection rates of each group of test subjects using the corresponding parameter combinations in descending order are: 0.5 (third set of parameter combinations), 0.4 (second set of parameter combinations), and 0.2 (first set of parameter combinations). The number of digits in the permutation of the user collection rates for the first, second, and third set of parameter combinations are 3, 2, and 1, respectively.

[0102] S2312. Determine the return value corresponding to each set of parameters in the previous round based on the number of positions of each feedback indicator in each set of parameters and the weight of each feedback indicator.

[0103] In specific implementation, the preset reward function set in the embodiments of this application can be:

[0104]

[0105] Among them, reward j This represents the reward for the j-th parameter combination in the previous round, where j = 1 to N;

[0106] ω i Let represent the weight corresponding to the i-th feedback indicator in the j-th parameter combination, where i = 1 to m, and m is the number of feedback indicators.

[0107] l i This represents the number of positions of the i-th feedback index in the j-th parameter combination.

[0108] The reward value corresponding to each parameter combination in the N parameter combinations of the previous round can be calculated using the above reward function.

[0109] Continuing with the previous example, let's take the first group of test subjects using the first set of parameters from the previous round. The digits for user reading time corresponding to the first set of parameters are 3, the digits for user click-through rate are 1, and the digits for user collection rate are 3. Assuming the weights for user reading time (ω1 = 0.4), user click-through rate (ω2 = 0.35), and user collection rate (ω3 = 0.25), then the return value of the first set of parameters from the previous round... Similarly, the return value of the second set of parameters in the previous round can be calculated. The return value of the third parameter combination in the previous round

[0110] S232. Arrange the N sets of parameters from the previous round in descending order of return value, and select the parameter combinations with a preset ratio from front to back.

[0111] In practice, the preset ratio can be set according to requirements, and this application embodiment does not limit this. In the example above, the return values ​​are arranged in descending order as follows: the return value of the third parameter combination is 0.825, the return value of the first parameter combination is 0.57, and the return value of the second parameter combination is 0.44. Assuming the preset ratio is 0.7, there are three parameter combinations in total, and two parameter combinations with higher return values ​​can be selected: the third parameter combination and the first parameter combination.

[0112] S233. Calculate the mean and variance of each parameter in each selected parameter combination.

[0113] S234. Based on the preset evolutionary strategy algorithm, the algorithm iterates according to the preset number of iterations to generate N sets of parameter combinations for a new round that satisfy the Gaussian distribution of the mean and variance of each parameter.

[0114] In practical implementation, to prevent excessively rapid convergence and failure to obtain the optimal solution, after calculating the variance of each parameter in each selected parameter combination, the following steps are also included:

[0115] The offset variance corresponding to the variance is determined based on the set perturbation variance, the set maximum perturbation round, and the current iteration round. The offset variance is then added to the variance of each parameter.

[0116] Specifically, the offset variance corresponding to the variance is determined by the following formula:

[0117]

[0118] Where σ′ represents the offset variance corresponding to the variance;

[0119] p represents the current iteration round;

[0120] Δt represents the maximum set perturbation round;

[0121] Δσ represents the set disturbance variance.

[0122] In this embodiment of the application, the preset evolutionary strategy algorithm can be the CEM algorithm.

[0123] In the initial stage, that is, when generating N sets of parameter combinations in the first round of iteration, an initial mean and an initial variance can be set for each parameter, and N sets of parameter combinations that conform to the Gaussian distribution of the initial mean and initial variance of each parameter can be randomly generated.

[0124] The following example illustrates this point. Taking the weights α1, α2, and α3 of the three ranking sub-models in the ranking module of the recommendation system—the prediction of whether a click is made, the prediction of the duration, and the prediction of whether a like is made—as the parameter combination to be optimized, the final ranking score of the ranking module is obtained by linearly weighting these three ranking sub-models and their respective weights.

[0125] The initial mean and initial variance of the weight α1 for the prediction model of whether to click are set to 0, the initial mean and initial variance of the weight α2 for the prediction model of duration are set to 0, and the initial mean and initial variance of the weight α3 for the prediction model of whether to like are set to 0. Assuming N=3, three sets of parameter combinations are generated for testing in each round. The preset ratio is set to 0.7, and then two sets of parameter combinations with higher return values ​​are selected each time.

[0126] The first iteration process is as follows:

[0127] The parameter tuning device randomly generates three sets of parameter combinations α1, α2, and α3 that conform to a multivariate normal distribution N[(0, 0, 0), (1, 1, 1)] (i.e., conform to a Gaussian distribution with mean (0, 0, 0) and variance (1, 1, 1)). The first set of parameter combinations is: α1 = 0.8, α2 = 1.0, α3 = 1.7; the second set of parameter combinations is: α1 = 1.1, α2 = 0.3, α3 = 0.2; and the third set of parameter combinations is: α1 = 1.2, α2 = 1.3, α3 = 1.0. In N[(0,0,0),(1,1,1)], (0,0,0) are the initial means of parameters α1, α2, and α3, respectively, and (1,1,1) are the initial variances of parameters α1, α2, and α3, respectively. From the three sets of parameter combinations generated above, it can be seen that α1, α2, and α3 all conform to the Gaussian distribution of their respective means and variances, and can be randomly generated using code.

[0128] The parameter tuning device stores the three sets of parameter combinations generated in the first round to the parameter server. The recommendation system reads these three sets of parameter combinations from the parameter server. Assuming there are 300 users (i.e., test subjects), these 300 users are randomly divided into three groups of 100 users each. The 100 users in the first group are assigned to use the first set of parameter combinations (α1 = 0.8, α2 = 1.0, α3 = 1.7) for testing. The 100 users in the second group are assigned to use the second set of parameter combinations (α1 = 1.1, α2 = 0.3, α3 = 0.2) for testing. The 100 users in the third group are assigned to use the third set of parameter combinations (α1 = 1.2, α2 = 1.3, α3 = 1.0) for testing. In the test, the score for predicting whether a click will occur is denoted as ctr, the score for predicting the duration is denoted as du, and the score for predicting whether a like will occur is denoted as up. The final score given by the first group of 100 users using the first set of parameters for the ranking sub-model is: 0.8*ctr + 1.0*du + 1.7*up. The final score given by the second group of 100 users using the second set of parameters for the ranking sub-model is: 1.1*ctr + 0.3*du + 0.2*up. The final score given by the third group of 100 users using the third set of parameters for the ranking sub-model is: 1.2*ctr + 1.3*du + 1.0*up. Because the fusion weights of the three ranking sub-models mentioned above are different, the scores given for each item (or content) are also different, resulting in different recommendation results. After the first test period, the recommendation system statistically records the performance indicators of each group of users using the corresponding parameter combinations. Assume the reward function uses two indicators: user click-through rate (CTR) and user collection rate. Assume the weight of CTR is pre-set as w1 = 0.8 and the weight of collection rate is w2 = 0.2. The first group of users had a CTR of 0.1 and a collection rate of 0.01; the second group had a CTR of 0.11 and a collection rate of 0.007; and the third group had a CTR of 0.08 and a collection rate of 0.012. The CTR and collection rate of the first group of users each have two digits in their ranking. Therefore, the reward value for the first parameter combination (α1 = 0.8, α2 = 1.0, α3 = 1.7) is... The click-through rate of the second group of users has 1 digit in the ranking, and the collection rate has 3 digits in the ranking. Therefore, the return value for the second parameter combination (α1 = 1.1, α2 = 0.3, α3 = 0.2) is... The click-through rate of the third group of users has 3 digits, and the collection rate has 1 digit. Therefore, the return value for the third parameter combination (α1 = 1.2, α2 = 1.3, α3 = 1.0) is... Arrange the reward values ​​in descending order as follows: 0.87 (reward value of the second parameter combination), 0.5 (reward value of the first parameter combination), and 0.47 (reward value of the third parameter combination). Select the two parameter combinations that are ranked first, namely the second parameter combination and the first parameter combination, as the initial values ​​for the next iteration.

[0129] The second round of iterations is as follows:

[0130] The mean and variance of parameters α1, α2, and α3 in the two selected parameter combinations: the second parameter combination (α1 = 1.1, α2 = 0.3, α3 = 0.2) and the first parameter combination (α1 = 0.8, α2 = 1.0, α3 = 1.7) are calculated as follows:

[0131] mean of α1 variance of α1

[0132] mean of α2 variance of α2

[0133] mean of α3 variance of α3 Furthermore, the parameter tuning device randomly generates three sets of parameter combinations α1, α2, and α3 that conform to a multivariate normal distribution N[(0.95, 0.65, 0.95), (0.0225, 0.1225, 0.5625)], which are as follows:

[0134] The first set of parameter combinations: α1 = 0.9746, α2 = 0.5674, α3 = 0.8638; the second set of parameter combinations: α1 = 0.9148, α2 = 0.6317, α3 = 1.3788; the third set of parameter combinations: α1 = 0.9342, α2 = 0.7445, α3 = 1.1540.

[0135] From the three sets of parameter combinations generated in this round, it can be seen that α1's value is within a very small range after only one iteration, namely (0.9148~0.9746). This is because the variance of α1 is too small, only 0.0225. To prevent convergence too quickly due to accidental factors, in this embodiment, after calculating the mean and variance of each parameter based on the selected optimal parameter combination in each round, an offset value (i.e., offset variance) is added to the variance of each parameter. Assuming the maximum perturbation round is set to 10 and the perturbation variance is set to 0.2, according to the formula for offset variance... It can be calculated that in the second round (i.e., the current round is 2) Adding the offset variance to the variance σ1 of α1: σ1 + 0.032 = 0.0225 + 0.032 = 0.0257; adding the offset variance to the variance σ2 of α2: σ2 + 0.032 = 0.1225 + 0.032 = 0.1545; adding the offset variance to the variance σ3 of α3: σ3 + 0.032 = 0.5625 + 0.032 = 0.5945. Then, when generating the parameter combinations for the second round, replace N[(0.95, 0.65, 0.95), (0.0225, 0.1225, 0.5625)] with N[(0.95, 0.65, 0.95), (0.0257, 0.1545, 0.5945)] to randomly generate the final three parameter combinations as follows:

[0136] The first set of parameter combinations: α1 = 1.1517, α2 = 0.5680, α3 = 0.8343; the second set of parameter combinations: α1 = 0.9636, α2 = 0.5882, α3 = 0.5310; the third set of parameter combinations: α1 = 1.2394, α2 = 0.8161, α3 = 0.7149.

[0137] It should be noted that the value of N mentioned above is relatively small. For ease of illustration, the value of N and the number of iterations can be set based on empirical values ​​during implementation. For example, N can be set to 50, and the number of iterations can be preset, for example, 100 iterations. This application embodiment does not limit this. If N is 50 and the number of iterations is 100, 50 sets of parameter combinations can be generated for 100 rounds. The recommendation system selects the optimal parameter combination from the final generated parameter combinations for official use.

[0138] In the example above, after the selected parameter combination was put into operation after the last iteration, the overall click-through rate of users increased by 0.5%, and the click-through rate increased by 0.26%, which enabled the final ranking score to better balance the effects among the three ranking sub-modules.

[0139] The parameter adjustment method provided in this application combines a preset evolutionary strategy algorithm to automatically generate parameter combinations for sub-modules in the recommendation system, thereby achieving automatic adjustment of parameters in the recommendation system without relying on human experience, improving parameter adjustment efficiency. Furthermore, since each set of parameter combinations generated in each round is assigned to different groups of test objects for testing, and a new set of parameter combinations is generated based on the execution behavior indicators corresponding to different parameter combinations for each group of test objects and the preset evolutionary strategy algorithm, the generated new set of parameter combinations can be further optimized, thus enabling the recommendation system to achieve better recommendation results.

[0140] Exemplary device

[0141] Having introduced the method of the exemplary embodiments of this application, the parameter adjustment device of the exemplary embodiments of this application will now be described.

[0142] like Figure 5 The diagram shown is a structural schematic of the parameter adjustment device 30 provided in an embodiment of this application. In one embodiment, the parameter adjustment device 30 includes: a response unit 301, an acquisition unit 302, and a generation unit 303.

[0143] The response unit 301 is used to respond to parameter adjustment requests for submodules in the recommendation system;

[0144] The acquisition unit 302 is used to acquire the feedback indicators of the N sets of parameter combinations in the previous round within the previous test cycle. The feedback indicator of each set of parameter combinations is the execution behavior indicator of each test object using the corresponding parameter combination. The N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm.

[0145] The generation unit 303 is used to generate a new round of N sets of parameter combinations based on the feedback indicators of the N sets of parameter combinations from the previous round and the preset evolutionary strategy algorithm.

[0146] In one possible implementation, the generating unit 303 is specifically used for:

[0147] Based on the feedback indicators and preset return functions of the N parameter combinations from the previous round, determine the return values ​​corresponding to each of the N parameter combinations from the previous round.

[0148] Arrange the N sets of parameters from the previous round in descending order of return value, and select a preset proportion of parameter combinations from front to back.

[0149] Calculate the mean and variance of each parameter in each selected parameter combination;

[0150] Based on the preset evolutionary strategy algorithm, the algorithm iterates according to the preset number of iterations to generate a new round of N sets of parameter combinations that satisfy the Gaussian distribution of the mean and variance of each parameter.

[0151] In one possible implementation, the generating unit 303 is specifically used for:

[0152] Each feedback index of each parameter combination from the previous round is sorted.

[0153] Based on the number of positions of each feedback indicator in each set of parameters and the weight of each feedback indicator, the return value corresponding to each set of parameters in the previous round is determined.

[0154] In one possible implementation, the preset reward function is:

[0155]

[0156] Among them, reward j This represents the reward for the j-th parameter combination in the previous round, where j = 1 to N;

[0157] ω i This represents the weight corresponding to the i-th feedback indicator in the j-th parameter combination, where i = 1 to m, and m is the number of feedback indicators.

[0158] l i This indicates the number of positions of the i-th feedback index in the j-th parameter combination.

[0159] In one possible implementation, the device further includes:

[0160] The determining unit is used to determine the offset variance corresponding to the variance after calculating the variance of each parameter of each selected parameter combination, based on the set perturbation variance, the set maximum perturbation round, and the current iteration round.

[0161] The processing unit is used to add the offset variance to the variance of each parameter respectively.

[0162] In one possible implementation, the determining unit is specifically used for:

[0163] The offset variance corresponding to the variance is determined by the following formula:

[0164]

[0165] Wherein, σ′ represents the offset variance corresponding to the variance;

[0166] p represents the current iteration round;

[0167] Δt represents the set maximum disturbance round;

[0168] Δσ represents the set disturbance variance.

[0169] Based on the same inventive concept as the parameter adjustment method described above, this application also provides an electronic device 40. For example... Figure 6 As shown, the electronic device 40 may include a processor 401 and a memory 402.

[0170] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0171] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 402 in this embodiment may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0172] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0173] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0174] While the spirit and principles of this application have been described with reference to several specific embodiments, it should be understood that this application is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A parameter adjustment method, characterized in that, include: Responding to parameter adjustment requests for submodules in the recommendation system; Obtain feedback metrics for N sets of parameter combinations from the previous round within the previous test period. The feedback metrics for each set of parameter combinations are the execution behavior metrics of each test object using the corresponding parameter combination. The N sets of parameter combinations are generated based on a preset evolutionary strategy algorithm. When the submodule is a sorting module, a set of parameter combinations is the weight of each sorting submodel included in the sorting module. The new round of N parameter combinations is generated based on the feedback indicators of the previous round's N parameter combinations and the preset evolutionary strategy algorithm.

2. The method according to claim 1, characterized in that, Based on the feedback indicators of the N parameter combinations from the previous round and the preset evolutionary strategy algorithm, a new set of N parameter combinations is generated, specifically including: Based on the feedback indicators and preset return functions of the N parameter combinations from the previous round, determine the return values ​​corresponding to each of the N parameter combinations from the previous round. Arrange the N sets of parameters from the previous round in descending order of return value, and select a preset proportion of parameter combinations from front to back. Calculate the mean and variance of each parameter in each selected parameter combination; Based on the preset evolutionary strategy algorithm, the algorithm iterates according to the preset number of iterations to generate a new round of N sets of parameter combinations that satisfy the Gaussian distribution of the mean and variance of each parameter.

3. The method according to claim 2, characterized in that, Based on the feedback indicators and preset return functions of the N parameter combinations from the previous round, determine the return value corresponding to each of the N parameter combinations from the previous round, specifically including: Each feedback index of each parameter combination from the previous round is sorted. Based on the number of positions of each feedback indicator in each set of parameters and the weight of each feedback indicator, the return value corresponding to each set of parameters in the previous round is determined.

4. The method according to claim 3, characterized in that, The preset reward function is: Among them, reward j This represents the reward for the j-th parameter combination in the previous round, where j = 1 to N; ω i This represents the weight corresponding to the i-th feedback indicator in the j-th parameter combination, where i = 1 to m, and m is the number of feedback indicators. l i This indicates the number of positions of the i-th feedback index in the j-th parameter combination.

5. The method according to claim 2, characterized in that, After calculating the variance of each parameter in each selected parameter combination, the following is also included: The offset variance corresponding to the set perturbation variance, the set maximum perturbation round, and the current iteration round are used to determine the variance. The offset variance is added to the variance of each parameter respectively.

6. The method according to claim 5, characterized in that, The offset variance corresponding to the set disturbance variance and the set maximum disturbance round number are determined, specifically including: The offset variance corresponding to the variance is determined by the following formula: Where, σ ′ This represents the offset variance corresponding to the variance. p represents the current iteration round; Δt represents the set maximum disturbance round; Δσ represents the set disturbance variance.

7. A parameter adjustment device, characterized in that, include: A response unit is used to respond to parameter adjustment requests for submodules in the recommendation system; The acquisition unit is used to acquire the feedback indicators of the N sets of parameter combinations in the previous round within the previous test period. The feedback indicator of each set of parameter combinations is the execution behavior indicator of each test object using the corresponding parameter combination. The N sets of parameter combinations are generated based on a preset evolution strategy algorithm. When the submodule is a sorting module, a set of parameter combinations is the weight of each sorting sub-model contained in the sorting module. The generation unit is used to generate a new set of N parameter combinations based on the feedback indicators of the N parameter combinations from the previous round and the preset evolutionary strategy algorithm.

8. The apparatus according to claim 7, characterized in that, The generation unit is specifically used for: Based on the feedback indicators and preset return functions of the N parameter combinations from the previous round, determine the return values ​​corresponding to each of the N parameter combinations from the previous round. Arrange the N sets of parameters from the previous round in descending order of return value, and select a preset proportion of parameter combinations from front to back. Calculate the mean and variance of each parameter in each selected parameter combination; Based on the preset evolutionary strategy algorithm, the algorithm iterates according to the preset number of iterations to generate a new round of N sets of parameter combinations that satisfy the Gaussian distribution of the mean and variance of each parameter.

9. The apparatus according to claim 8, characterized in that, The generation unit is specifically used for: Each feedback index of each parameter combination from the previous round is sorted. Based on the number of positions of each feedback indicator in each set of parameters and the weight of each feedback indicator, the return value corresponding to each set of parameters in the previous round is determined.

10. The apparatus according to claim 9, characterized in that, The preset reward function is: Among them, reward j This represents the reward for the j-th parameter combination in the previous round, where j = 1 to N; ω i This represents the weight corresponding to the i-th feedback indicator in the j-th parameter combination, where i = 1 to m, and m is the number of feedback indicators. l i This indicates the number of positions of the i-th feedback index in the j-th parameter combination.

11. The apparatus according to claim 8, characterized in that, Also includes: The determining unit is used to determine the offset variance corresponding to the variance after calculating the variance of each parameter of each selected parameter combination, based on the set perturbation variance, the set maximum perturbation round, and the current iteration round. The processing unit is used to add the offset variance to the variance of each parameter respectively.

12. The apparatus according to claim 11, characterized in that, The determining unit is specifically used for: The offset variance corresponding to the variance is determined by the following formula: Where, σ ′ This represents the offset variance corresponding to the variance. p represents the current iteration round; Δt represents the set maximum disturbance round; Δσ represents the set disturbance variance.

13. A recommendation system, characterized in that, The recommendation system comprises several sub-modules. When any sub-module needs to adjust its parameters, it uses N sets of parameter combinations generated by the parameter adjustment method described in any one of claims 1 to 6 to recommend information.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the parameter adjustment method according to any one of claims 1 to 6.

15. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of the parameter adjustment method according to any one of claims 1 to 6.

Images