An adaptive online sorting method and system for dynamic environments

Abstract

The present invention discloses an adaptive online sorting method and system for dynamic environments. The online sorting learning task is modeled as a multi-arm duel slot machine problem, and the query results are sorted using an adaptive online sorting method. First, a sorting model is determined and relevant parameters are set. Then, an adaptive online sorting method is used to update the parameters of the sorting model, and the query results are sorted using the updated sorting model. The adaptive online sorting method includes a meta-method and multiple expert methods. The present invention can adaptively sort the query results online, and is suitable for a dynamic environment where user query preferences change over time.

Description

Technical Field

[0001] The present invention relates to an online sorting method in the technical field of data mining and machine learning, and in particular to a method for performing adaptive online sorting in a dynamic environment, which can be applied to information retrieval scenarios such as web page search and product search. Background Art

[0002] Learning to rank refers to the use of machine learning techniques to rank query results. For a given query-document pair, the results related to the query input are ranked by extracting corresponding features and training a ranking model. Learning to rank can be divided into offline ranking methods and online ranking methods. Traditional offline ranking methods require collecting user query inputs and documents with relevant tags as training data, and then training a parameterized ranking model to predict the relevance of query results for ranking. The disadvantage of this method is that it is very time-consuming to collect these training data and is not suitable for large-scale applications. In addition, once the ranking model is trained, it is fixed, so the ranking model cannot be continuously updated and it is difficult to meet user needs. In order to solve the above problems, some recent studies have introduced online ranking methods.

[0003] Online ranking methods can continuously update and optimize ranking models based on user interaction behaviors, and do not need to store a large amount of offline data. Online ranking methods are widely used in information retrieval scenarios such as web search and product search. For example, in web search, online ranking methods can update the ranking model in time according to user interaction feedback (clicking on the web links of interest) after displaying the query results, and display the web links that are more relevant to the user's query in the front, and the web links with less relevance in the back, thereby improving the quality of search results. Because online ranking methods need to balance between exploring new ranking models and using the current optimal ranking model, a common method is to model the online ranking learning task as a multi-arm duel slot machine problem, where each arm corresponds to a ranking model parameter. In each round, the online ranking method updates the parameters of the optimized ranking model by comparing different arms. Current online ranking methods generally assume that the user's query preference satisfies a static probability distribution. However, in many real-life scenarios, the user's query preference usually changes over time. In order to better handle ranking tasks in real life, an adaptive online ranking method is needed to continuously learn the user's query preference. Summary of the invention

[0004] Purpose of the invention: Current online ranking methods are only applicable to static environments where user query preferences are fixed. However, in many real-world scenarios, user query preferences change over time, which is a dynamic environment that is constantly changing and cannot be predicted in advance. In view of the problems and shortcomings of the prior art, the present invention provides an adaptive online ranking method and system for dynamic environments.

[0005] Technical solution: An adaptive online ranking method for dynamic environments, used in information retrieval scenarios such as web page search and product search. The specific process is, first, determine the ranking model and set relevant parameters. Then, use an adaptive method to update the parameters of the ranking model, and use the updated ranking model to sort the query results. Among them, the adaptive method includes a meta-method and multiple expert methods. Each expert method is configured with different exploration step parameters for a possible user preference, and the parameters of the expert method are updated by gradient descent; the meta-method receives the parameter results of all expert methods and configures different weights for each expert method; then the parameter results of each expert method are weighted and summed to form the parameters of the current ranking model, and then the exploration ranking model parameters are calculated, and the parameters of the ranking model are updated according to the comparison results of the parameters of the current ranking model and the exploration ranking model, and the query results are sorted with the updated ranking model. Finally, the weight of each expert method is updated.

[0006] The adaptive method includes a meta-method and an expert method.

[0007] The specific steps of the meta-method are:

[0008] Step 101, setting a step size parameter α; the step size is used to update the weight of each expert method;

[0009] Step 102, setting the number of meta-method running rounds T;

[0010] Step 103, setting the number of expert methods N;

[0011] Step 104, setting the exploration step parameter δ;

[0012] Step 105, setting the step size parameters γ1,…,γ of each expert method N ;

[0013] Step 106, initialize the weight of each expert method in the first round

[0014] Step 107, in the tth round, t=1, 2, ..., T, perform the following steps:

[0015] Step 108, receiving the output of the i-th expert method

[0016] Step 109: Calculate the current sorting model parameters is the weight of the i-th expert method in the t-th round;

[0017] Step 110, from the unit sphere The uniform sampling vector u t As a direction for exploration;

[0018] Step 111, create an exploration ranking model with parameters W′ t =Π w [w t +δu t ];

[0019] Step 112, compare parameter w′ t and w t According to the comparison results, update the parameters of the sorting model;

[0020] Step 113, sorting the query results using the updated sorting model;

[0021] Step 114, construct an alternative loss function l t (w);

[0022] Step 115, update the weight of each expert method

[0023] Step 116: and u t Send to each expert method.

[0024] The specific steps of each expert method are:

[0025] Step 200, initialization in refers to the parameters of the i-th expert method in the first round;

[0026] Step 201, in the tth round, t=1, 2, ..., T, perform the following steps:

[0027] Step 202: Sent to metamethod;

[0028] Step 203, receiving from the meta method and u t ;

[0029] Step 204, update output

[0030] The setting method of the step size parameter α in step 101 is:

[0031] The number of expert methods in step 103

[0032] The step size parameter is explored in step 104 Where d is the parameter decision set Dimension.

[0033] The step size parameter used by each expert method in step 105 is Where R is the parameter decision set The upper bound of

[0034] The projection operator Π in step 112 W The specific definition of [·] is

[0035] In step 113, the parameter w′ t and w t The corresponding sorting results are cross-displayed, and the parameter w′ is compared through user feedback on the sorting results. t and w t The advantages and disadvantages.

[0036] The alternative loss function l constructed in step 114 is t The specific definition of (w) is

[0037] In step 116 is the indicator function, w′ t >w t Represents parameter w′ t Better than w t .

[0038] An adaptive online ranking system for dynamic environments, used in information retrieval scenarios such as web page search and product search. It includes: a ranking model determination module, a meta module, and multiple expert modules;

[0039] The sorting model determination module is used to determine the sorting model and set relevant parameters;

[0040] Each of the expert modules is configured with different exploration step parameters for a possible user preference, and the parameters of the expert modules are updated by gradient descent;

[0041] The meta-module receives the parameter results of all expert modules, and configures different weights for each expert module; then the parameter results of each expert module are weighted and summed to form the parameters of the current sorting model, and then the parameters of the exploratory sorting model are calculated, and the parameters of the sorting model are updated according to the comparison results of the parameters of the current sorting model and the exploratory sorting model, and the query results are sorted using the updated sorting model. Finally, the weight of each expert module is updated.

[0042] The implementation process of the meta-module is as follows: set the step parameter α, the number of meta-module operation rounds T, the number of expert modules N, the exploration step parameter δ, and the utilization step parameters γ1,…,γ N ;

[0043] Initialize the weights of each expert module in the first round

[0044] In the tth round, receive the output of the i-th expert module Calculate the current sorting model parameters is the weight of the i-th expert module in the t-th round; The uniform sampling vector u t As the exploration direction; create an exploration sorting model with parameters Comparison parameter w′ t and w t According to the comparison results, update the parameters of the sorting model; sort the query results with the updated sorting model; construct an alternative loss function l t (w); Update the weight of each expert module Will and u t Sent to each expert module.

[0045] The implementation process of the expert module is as follows: Initialization in Refers to the parameters of the i-th expert module in the first round;

[0046] In round t, Send to meta-module; Receive from meta-module and u t ; Update output

[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the adaptive online sorting method for dynamic environment as described above is implemented.

[0048] A computer-readable storage medium stores a computer program for executing the adaptive online sorting method for dynamic environments as described above.

[0049] Beneficial effects: Compared with the prior art, the present invention can adaptively sort query results online, and is suitable for a dynamic environment where user query preferences change over time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a meta-method according to an embodiment of the present invention;

[0051] Figure 2 The expert method workflow diagram of the embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0053] The adaptive online ranking method for dynamic environments is applicable to information retrieval scenarios such as web page search, product search, etc. The following takes web page search in a search website as an example to introduce the specific workflow of the adaptive online ranking method for dynamic environments in an embodiment of the present invention.

[0054] First, determine the ranking model and set relevant parameters. Then, use an adaptive method to update the parameters of the ranking model, and use the updated ranking model to rank the query results. The adaptive method includes a meta-method and multiple expert methods. Each expert method is configured with different exploration step parameters for a possible user preference, and the parameters of the expert method are updated by gradient descent; the meta-method receives the parameter results of all expert methods and configures different weights for each expert method; then the parameter results of each expert method are weighted and summed to form the parameters of the current ranking model, and then the exploration ranking model parameters are calculated. The parameters of the ranking model are updated according to the comparison results of the parameters of the current ranking model and the exploration ranking model, and the query results are ranked by the updated ranking model. Finally, the weight of each expert method is updated.

[0055] The workflow of the metamethod is as follows Figure 1 As shown. First, determine the online sorting model, where w represents the parameters of the sorting model.

[0056] Next, determine the number of running rounds T and set the step size parameter Set the number of expert methods Set the exploration step size parameter

[0057] Then, set the step size parameter of each expert method Initialize the weights of each expert method

[0058] Finally, we start the online operation of each round. In each round, we first collect the user's query, which refers to the content that the user wants to search, such as the keyword entered. The query and the related web links are used as input to the ranking model. Each expert method determines its own parameters based on the input. At the same time, the parameters Sent to the meta-method. The meta-method receives the output of the i-th expert method Calculate the current sorting model parameters And from the unit sphere The uniform sampling vector u t As an exploration direction, create an exploration ranking model with parameters Then the parameter w′ t and w t The corresponding web page link ranking results are cross-displayed, and the parameter w′ is compared through the user's click feedback on the ranking results. t and w t Constructing an alternative loss function in is the indicator function, w′ t >w t Represents parameter w′ t Better than w t , is the feasible domain of the ranking model parameters. According to the comparison results of the current ranking model and the exploration ranking model, the parameters of the ranking model are updated and the ranking of the query results is given. Then, the alternative loss function is calculated and the weight of each expert method is updated. Afterwards and u t Send to each expert method.

[0059] The workflow diagram of each expert method is as follows Figure 2 As shown. First initialize In each round, the expert method first sends the output of the current round to the meta-method, and then receives the output from the meta-method. and u t , and finally each expert method updates the output

[0060] The expert method runs simultaneously with the metamethod, sending and receiving parameters in each round, so the expert method and metamethod run the same number of rounds.

[0061] An adaptive online ranking system for dynamic environments, used in information retrieval scenarios such as web page search and product search. It includes: a ranking model determination module, a meta module, and multiple expert modules;

[0062] The sorting model determination module is used to determine the sorting model and set relevant parameters;

[0063] Each expert module is configured with different exploration step parameters for a possible user preference, and the parameters of the expert module are updated using gradient descent.

[0064] The meta-module receives the parameter results of all expert modules and configures different weights for each expert module. It then performs a weighted summation of the parameter results of each expert module to form the parameters of the current sorting model, and then calculates the parameters of the exploration sorting model. It updates the parameters of the sorting model based on the comparison results of the parameters of the current sorting model and the exploration sorting model, and uses the updated sorting model to sort the query results. Finally, it updates the weight of each expert module.

[0065] The implementation process of the meta-module is:

[0066] First, the online ranking model and the parameters of the ranking model are determined.

[0067] Next, determine the number of running rounds T and set the step size parameter Set the number of expert modules Set the exploration step size parameter

[0068] Then, set the utilization step size parameters of each expert module Initialize the weights of each expert module

[0069] Finally, start the online operation of each round. In each round, first collect the user's query, which refers to the content that the user wants to query, such as the keyword entered. The query and the related web links are used as input to the ranking model. Each expert module determines its own parameters based on the input. At the same time, the parameters Sent to the meta-method. The meta-module receives the output of the i-th expert module Calculate the current sorting model parameters And from the unit sphere The uniform sampling vector u t As an exploration direction, create an exploration ranking model with parameters Then the parameter w′ t and w t The corresponding web page link ranking results are cross-displayed, and the parameter w′ is compared through the user's click feedback on the ranking results. t and w t Constructing an alternative loss function in is the indicator function, w′ t >w t Represents parameter w′ t Better than w t , is the feasible domain of the ranking model parameters. According to the comparison results of the current ranking model and the exploration ranking model, the parameters of the ranking model are updated and the ranking of the query results is given. Then, the alternative loss function is calculated and the weight of each expert module is updated. Afterwards and u t Sent to each expert module.

[0070] The execution process of each expert module is:

[0071] First initialize In each round, the expert module first sends the output of the current round to the meta-module, and then receives the output from the meta-module. and u t , and finally each expert module updates the output

[0072] Obviously, those skilled in the art should understand that the above-mentioned steps of the adaptive online sorting method for dynamic environment or the modules of the adaptive online sorting system for dynamic environment can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. An adaptive online sorting method for dynamic environments, characterized in that: For information retrieval scenarios, first, determine the ranking model and set relevant parameters; then, use an adaptive method to update the parameters of the ranking model, and use the updated ranking model to rank the query results; the adaptive method includes a meta-method and multiple expert methods; each expert method is configured with different exploration step parameters for a possible user preference, and the parameters of the expert method are updated by gradient descent; the meta-method receives the parameter results of all expert methods and configures different weights for each expert method; then the parameter results of each expert method are weighted and summed to form the parameters of the current ranking model, and then the exploration ranking model parameters are calculated, and the parameters of the ranking model are updated according to the comparison results of the parameters of the current ranking model and the exploration ranking model, and the query results are ranked by the updated ranking model; finally, the weight of each expert method is updated; The specific steps of the meta-method are: Step 101, setting a step size parameter α; the step size is used to update the weight of each expert method; Step 102, setting the number of meta-method running rounds T; Step 103, setting the number of expert methods N; Step 104, setting the exploration step parameter δ; Step 105, setting the step size parameters γ1,...,Y of each expert method N ; Step 106, initialize the weight of each expert method in the first round Step 107, in the tth round, t=1, 2, ..., T, perform the following steps: Step 108, receiving the output of the i-th expert method Step 109: Calculate the current sorting model parameters is the weight of the i-th expert method in the t-th round; Step 110, from the unit sphere The uniform sampling vector u t As an exploration direction, d is the dimension of the parameter decision set W; Step 111, create an exploration ranking model with parameter w′ t =Π w [w t +δu t ], is the parameter decision set, Π w [·] represents the projection operator; Step 112, compare parameter w′ t and w t According to the comparison results, update the parameters of the sorting model; Step 113, sorting the query results using the updated sorting model; Step 114, construct an alternative loss function l t (w); Step 115, update the weight of each expert method Step 116: and u t Send to each expert method; The specific steps of each expert method are: Step 200, initialization in refers to the parameters of the i-th expert method in the first round; Step 201, in the tth round, t=1, 2, ..., T performs the following steps: Step 202: Sent to metamethod; Step 203, receiving from the meta method and u t ; Step 204, update output 2. The adaptive online sorting method for dynamic environment according to claim 1, characterized in that: The setting method of the step size parameter α in step 101 is: The number of expert methods in step 103 The step size parameter is explored in step 104 Where d is the dimension of the parameter decision set w; the step size parameter used by each expert method in step 105 is Where R is the parameter decision set The upper bound of .

3. The adaptive online sorting method for dynamic environment according to claim 1, characterized in that: The projection operator Π in step 111 W The specific definition of [·] is Π W [u] = argmin v∈W ||uv||, In step 113, the parameter w′ t and w t The corresponding sorting results are cross-displayed, and the parameter w′ is compared through user feedback on the sorting results. t and w t The pros and cons of The alternative loss function l constructed in step 114 is t The specific definition of (w) is In step 116, II is the indicator function, w′ t >w t Represents parameter w′ t Better than w t .

4. An adaptive online sorting system for dynamic environments, characterized in that: Used in information retrieval scenarios, including a ranking model determination module, a meta-module, and multiple expert modules; The sorting model determination module is used to determine the sorting model and set relevant parameters; Each of the expert modules is configured with different exploration step parameters for a possible user preference, and the parameters of the expert modules are updated by gradient descent; The meta-module receives the parameter results of all expert modules and configures different weights for each expert module; then the parameter results of each expert module are weighted and summed to form the parameters of the current sorting model, and then the exploration sorting model parameters are calculated, and the parameters of the sorting model are updated according to the comparison results of the parameters of the current sorting model and the exploration sorting model, and the query results are sorted using the updated sorting model; finally, the weight of each expert module is updated; The implementation process of the meta-module is as follows: set the step parameter α, the number of meta-module operation rounds T, the number of expert modules N, the exploration step parameter δ, and the utilization step parameters γ1,...,γ N ; Initialize the weights of each expert module in the first round In the tth round, receive the output of the i-th expert module Calculate the current sorting model parameters is the weight of the i-th expert module in the t-th round; The uniform sampling vector u t As the exploration direction; create an exploration sorting model with parameters Comparison parameter w′ t and w t According to the comparison results, update the parameters of the sorting model; sort the query results with the updated sorting model; construct an alternative loss function l t (w); Update the weight of each expert module Will and u t Sent to each expert module, d is the parameter decision set The dimension, Π W [·] represents the projection operator; The implementation process of the expert module is as follows: Initialization in refers to the parameters of the i-th expert module in the first round, is the parameter decision set; In round t, Send to meta-module; Receive from meta-module and u t ; Update output 5. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the adaptive online sorting method for dynamic environments as described in any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the adaptive online sorting method for dynamic environments as described in any one of claims 1 to 3.

Images