Search sorting model training method, search sorting method, device and program

Abstract

The embodiment of the invention provides a training method of a search sorting model, a search sorting method, equipment and a program. The training method comprises the following steps: acquiring a plurality of training samples; based on a plurality of behavior indexes associated with the optimization target, determining overall point-to-point losses corresponding to the plurality of training samples in each behavior index; determining pairwise loss of a plurality of behavior indexes among the training samples and pairwise loss of object attribute influence among the training samples, wherein the pairwise loss of the object attribute influence is the pairwise loss corresponding to the influence of the object attribute on the plurality of behavior indexes; determining overall pairwise loss according to the pairwise loss of the plurality of behavior indexes among the training samples and the pairwise loss influenced by the object attributes; determining a comprehensive loss function according to the overall point-to-point loss and the overall paired loss corresponding to each behavior index; and training a search sorting model based on the comprehensive loss function. According to the embodiment of the invention, the search sorting model can be optimized, and the service capability of an online service platform is improved.

Description

TECHNICAL FIELD

[0001] The embodiments of the present application relate to the technical field of computer Internet technology, and in particular, to a search ranking model training method, a search ranking method, a device and a program. BACKGROUND

[0002] An online service platform can provide a search service for a user, thereby determining a plurality of recommended objects in accordance with a search request of the user based on the search request; based on specific functions of the online service platform and different search scenarios, the recommended objects can have different forms, such as recommended hotels in a hotel search scenario of a travel service platform, recommended traffic tickets in a traffic ticket search scenario of the travel service platform, recommended goods in a goods search scenario of an e-commerce platform, and recommended videos in a video search scenario of a video multimedia platform. Among them, the travel service platform is an online service platform that provides travel services for users, such as an OTP (Online Travel Platform) platform, an OTA (Online Travel Agency) platform, and the like.

[0003] After determining the plurality of recommended objects, the online service platform such as the travel service platform can determine a search ranking of the recommended objects, and the search ranking refers to the ranking of the recommended objects in the search results displayed to the user; specifically, after determining the plurality of recommended objects, the travel service platform can use a search ranking model to determine a search ranking score of each recommended object, thereby ranking the recommended objects based on the search ranking score of the recommended objects to determine the ranking of the recommended objects in the search results.

[0004] As a core component of the online service platform such as the travel service platform, the search ranking model is crucial to the search service of the online service platform such as the travel service platform, and therefore, how to provide a training scheme of the search ranking model to optimize the search ranking model, thereby providing support for improving the service capability of the online service platform, has become a technical problem that needs to be solved by the technical personnel in the field. SUMMARY

[0005] Therefore, the embodiments of the present application provide a search ranking model training method, a search ranking method, a device and a program to optimize the search ranking model and provide support for improving the service capability of the online service platform.

[0006] To achieve the above object, the embodiments of the present application provide the following technical scheme.

[0007] In a first aspect, the embodiments of the present application provide a search ranking model training method, comprising:

[0008] obtaining a plurality of training samples;

[0009] determine the overall pairwise loss of the plurality of training samples according to the pairwise loss of the plurality of behavior indicators and the pairwise loss of the object attribute influence;

[0010] determine the pairwise loss of the plurality of behavior indicators between the training samples, and determine the pairwise loss of the object attribute influence between the training samples; wherein the optimization target of the search ranking model is further associated with at least one object attribute of the recommended object, and the pairwise loss of the object attribute influence is the pairwise loss corresponding to the influence of the object attribute on the plurality of behavior indicators;

[0011] determine the overall pairwise loss of the plurality of training samples according to the pairwise loss of the plurality of behavior indicators and the pairwise loss of the object attribute influence;

[0012] determine the comprehensive loss function according to the overall point-to-point loss of the plurality of training samples corresponding to each behavior indicator and the overall pairwise loss;

[0013] train the search ranking model based on the comprehensive loss function.

[0014] In a second aspect, an embodiment of the present application provides a search ranking method, comprising:

[0015] obtain a search request of a user;

[0016] determine a plurality of recommended objects corresponding to the search request;

[0017] for any one recommended object, determine the model input feature of the recommended object based on the recommended object related feature and the user related feature of the user;

[0018] deliver the model input feature of each recommended object to the search ranking model to obtain the search ranking score of each recommended object determined by the search ranking model; wherein the search ranking model is trained based on the training method of the search ranking model of the first aspect described above;

[0019] determine the ranking of each recommended object in the search result based on the search ranking score of each recommended object, and display it to the user.

[0020] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, the memory stores computer execution instructions, and the processor invokes the computer execution instructions to execute the training method of the search ranking model of the first aspect described above, or the search ranking method of the second aspect described above.

[0021] In a fourth aspect, an embodiment of the present application provides a computer program product comprising computer execution instructions, which, when executed, implement the training method of the search ranking model according to the first aspect above, or the search ranking method according to the second aspect above.

[0022] The training method of the search ranking model provided by the embodiment of the present application determines the overall point-to-point loss of each behavior indicator corresponding to a plurality of training samples based on a plurality of behavior indicators of a user associated with an optimization target of the search ranking model, so as to consider the point-to-point loss of the behavior indicators associated with the optimization target when determining the loss function of the search ranking model, so as to ensure that the predicted value of the behavior indicators predicted by the model is close to the true value, and improve the accuracy of the model. Meanwhile, the embodiment of the present application introduces a hierarchical pairwise loss, which is divided into a plurality of behavior indicators of the pairwise loss and an object attribute influence of the pairwise loss. Specifically, the embodiment of the present application can determine the pairwise loss of a plurality of behavior indicators between training samples, so as to ensure that the sorting prediction of the model at the behavior indicator level conforms to the logic. The object attribute influence of the pairwise loss between the training samples is determined, and the object attribute influence of the pairwise loss is the pairwise loss corresponding to the influence of the object attribute of the recommended object on a plurality of behavior indicators, and the optimization target of the search ranking model is also associated with the object attribute of the recommended object. Then, the embodiment of the present application can optimize the sorting influence of the object attribute on the behavior indicators through the object attribute influence of the pairwise loss, so as to ensure that the model at the behavior indicator level is reasonable while optimizing the influence of the object attribute of the recommended object. Then, the overall pairwise loss, i.e., the hierarchical pairwise loss, is determined based on the pairwise loss of a plurality of behavior indicators between the training samples and the object attribute influence of the pairwise loss. Finally, the comprehensive loss function is determined by fusing the overall point-to-point loss corresponding to each behavior indicator and the overall pairwise loss. The search ranking model is trained by using the comprehensive loss function. The trained search ranking model can comprehensively consider the prediction accuracy of the behavior indicators, the rationality of the sorting, and the balance between the behavior indicators and the object attribute influence, so as to optimize the search ranking model.

[0023] In summary, the training method of the search ranking model provided by the embodiment of the present application can optimize the search ranking model, so that the search ranking of the recommended object determined based on the search ranking model conforms to the demand of the optimization target, and provides support for improving the service capability of the online service platform. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiment or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creating any creative labor.

[0025] Figure 1 A flowchart of a training method of a search ranking model provided by an embodiment of the present application.

[0026] Figure 2 An example diagram of a structure of a search ranking model and a loss function involved.

[0027] Figure 3 A flowchart of determining a pairwise loss of a plurality of behavior indicators provided by an embodiment of the present application.

[0028] Figure 4 A flowchart of determining a pairwise loss of an object attribute influence provided by an embodiment of the present application.

[0029] Figure 5 A flowchart of a search ranking method provided by an embodiment of the present application.

[0030] Figure 6 A block diagram of a training device of a search ranking model provided by an embodiment of the present application.

[0031] Figure 7 A block diagram of a search ranking device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0033] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant national and regional laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0034] Training the search ranking model can be regarded as a learning to rank technique, learning a set of model parameters so that the search ranking model can determine the search ranking scores of the recommendation objects according to various input features such as user-related features and recommendation object-related features, and then the recommendation objects are ranked according to the search ranking scores and displayed to the user in the search results, aiming to achieve the optimization target expected to be achieved by the online service platform. The optimization target can be a quantifiable result that the online service platform hopes to improve by using the search ranking model, and can be set according to the actual needs of the online service platform. For example, the optimization target can be to improve the GMV (Gross Merchandise Volume) of the online service platform, such as the online service platform determining the search ranking scores of the recommendation objects by the search ranking model and ranking the recommendation objects in the search results according to the search ranking scores of the recommendation objects, hoping to achieve the effect of improving the GMV, for example, the effect of maximizing the GMV; the GMV represents the total sales or total transaction amount of the online service platform within a certain period of time.

[0035] It should be noted that the optimization target is related to a plurality of behavior indicators of the user and at least one object attribute of the recommendation object; the behavior indicator of the user refers to a quantitative measure formed by the interactive behavior of the user to the recommendation object in the recommendation or search scenario, used to measure the interest degree or conversion tendency of the user to the recommendation object, which can be a statistical result of historical behavior data or a prediction value of user behavior probability by a model, such as click rate, conversion rate, etc.; the object attribute of the recommendation object refers to the characteristic information of the recommendation object that has an impact on the optimization target, for example, the price, the number of likes, the number of views, etc. of the recommendation object. For example, taking the optimization target of maximizing the GMV as an example, the GMV is related to a plurality of behavior indicators of the user such as click rate and conversion rate, and the GMV is related to at least one object attribute of the recommendation object such as price. Further, the at least one object attribute of the recommendation object related to the GMV is not limited to price, but can also be, for example, the number of likes, the number of views, etc. of the recommendation object.

[0036] Among them, the click rate (CTR, Click Through Rate) refers to the probability that the recommendation object is clicked by the user; the conversion rate (CVR, Conversion Rate) refers to the probability that the user purchases or completes other target behaviors on the recommendation object, and improving the conversion rate means that more users will complete the transaction, thereby improving the GMV; the price (Price) refers to the pricing of the recommendation object, such as the price of the recommended hotel in the hotel search scenario of the travel service platform, the price of the recommended transportation ticket in the transportation ticket search scenario of the travel service platform, etc.

[0037] For further understanding, taking the optimization goal of maximizing GMV and the correlation of CTR, CVR and price as an example, it is assumed that a user u submits a search keyword q to an online service platform through a search request, and a search session s can be initiated between the user u and the online service platform, wherein the search session refers to a search interaction process between the user and the online service platform, involving a process from the user submitting a search keyword to the user interacting with a search result (such as clicking, purchasing a recommended object in the search result, etc.), and the user can directly exit the search session without taking actions such as clicking and purchasing a recommended object in the search result; based on the above assumption, in the case of a given price of each recommended object in the search result, the maximization of GMV can be defined by the following formula:

[0038]

[0039] wherein the capital S represents a set of search sessions, containing multiple search sessions (search session is represented as small s); i represents the i th recommended object (such as the i th commodity) in the search session s; E i represents an indication function of whether the i th recommended object is displayed, such as the i th recommended object is displayed, then E i is 1, and the i th recommended object is not displayed, then E i is 0; it should be noted that the recommended objects in the search result are displayed to the user in order, so not all recommended objects in the search result will be seen or interacted by the user, such as the recommended objects at the back of the page are not displayed without the user sliding the search result; C i represents an indication function of whether the i th recommended object is clicked, such as the i th recommended object is clicked, then C i is 1, and the i th recommended object is not clicked, then C i is 0; Price i represents an indication function of whether the i th recommended object is purchased, such as the i th recommended object is purchased, then Price i is 1, and the i th recommended object is not purchased, then Price i is 0.

[0040] To achieve optimization goals (such as maximizing GMV), during the model training phase (the phase of training the search ranking model), one training method for the search ranking model is the sample weighting method. The sample weighting method divides the training samples used to train the search ranking model into positive samples and negative samples. Positive samples include converted samples (such as samples corresponding to recommended objects that users have purchased), and negative samples include unconverted samples (such as samples corresponding to recommended objects included in the search results but not purchased by the user). Thus, the sample weighting method trains the search ranking model by applying a higher loss weight to positive samples (such as using the price of the recommended object corresponding to the positive sample as the loss weight of the positive sample) and a unit loss weight to negative samples (such as using 1 as the loss weight of the negative sample).

[0041] However, the training method of the above-mentioned sample weighting method is essentially an approximate model to achieve the optimization goal, and does not fully model the problem of achieving the optimization goal (such as not fully modeling the problem of maximizing GMV). Therefore, it has limitations in applicability, resulting in unsatisfactory model performance. Specifically, the training method of the above-mentioned sample weighting method is only equivalent to modeling the optimization goal (such as maximizing GMV) when the proportion of positive samples is low. Therefore, when the proportion of positive samples is high, the model performance of the above-mentioned sample weighting method may not be ideal, and it has limitations in applicability, that is, it cannot be applied to situations where the proportion of positive samples is high.

[0042] It's important to note that the loss function, often simply called loss, is used to measure the difference between the model's predictions and the actual results when training a search ranking model, and to guide the optimization of the model's parameters. Loss weights, on the other hand, are the weights of the loss function used to adjust the contribution of each sample to the loss function. For example, in the sample weighting method described above, positive and negative samples are assigned different loss weights, meaning that positive and negative samples can have different levels of attention during model training.

[0043] To achieve the optimization goal, during the model usage phase (the phase of using the search ranking model), one way the search ranking model determines the search ranking score is through a formula. For example, taking GMV optimization, and considering that GMV is associated with click-through rate (CTR), conversion rate (CVR), and price, the trained search ranking model can predict the CTR and conversion rate of recommended objects. The predicted CTR can be expressed as pCTR (Predicted Click Through Rate), and the predicted conversion rate can be expressed as pCVR (Predicted Conversion Rate). Therefore, the search ranking model can use the formula to calculate pCTR × pCVR × Price. αto obtain a search ranking score of the recommended object, and then rank the search result according to the search ranking score of the recommended object. It should be noted that in the model use stage, the learned search ranking model is used for real-time user search response of the online service platform, that is, after the online service platform obtains a search request of a user and determines a plurality of recommended objects corresponding to the search request, the learned search ranking model is used to determine a search ranking score of each recommended object to determine the ranking of each recommended object in the search result, so as to display the search result to the user to respond to the search request of the user in real time.

[0044] In the above formula method, a is used as a hyperparameter to adjust the influence degree of the price on the search ranking score, however, the optimal value of a is difficult to determine, and the adaptability to different scenarios is limited, for example, when the formula method is applied in different scenarios, the value of a may need to be adjusted frequently, which increases the complexity of the search ranking model.

[0045] At the same time, the above formula method is difficult to balance between conversion rate and price, that is, it is difficult to balance the relationship between conversion rate and price, for example, improving the conversion rate may cause the price to drop, and improving the price may cause the conversion rate to drop; that is, improving the conversion rate may require adjusting the price, thereby causing the price to drop, because a lower price may attract more users to click and make a purchase conversion; similarly, improving the price may reduce the conversion rate, because users may consider that the recommended object is overpriced, resulting in a decrease in purchase intention.

[0046] Based on the above description, the embodiments of the present application provide an improved hierarchical loss framework for training a search ranking model, which is composed of pointwise loss and hierarchical pairwise loss. Specifically, the embodiments of the present application consider a plurality of behavior indicators of a user associated with an optimization goal of the search ranking model and at least one object attribute of a recommended object, wherein the priority of the behavior indicators is higher than the priority of the object attribute, and the plurality of behavior indicators can also have priorities; thus, when training the search ranking model, the embodiments of the present application use not only the overall pointwise loss corresponding to each behavior indicator, but also the hierarchical pairwise loss composed of the pairwise loss of the plurality of behavior indicators and the pairwise loss affected by the object attribute, wherein the pairwise loss affected by the object attribute is the pairwise loss corresponding to the influence of the object attribute on the plurality of behavior indicators; further, in each training process of the search ranking model, the embodiments of the present application take the strategy of achieving the optimization goal without sacrificing the high-priority behavior indicators, and combine the overall pointwise loss corresponding to each behavior indicator and the hierarchical pairwise loss (composed of the pairwise loss of the plurality of behavior indicators and the pairwise loss affected by the object attribute) to determine a comprehensive loss function; further, training the search ranking model by using the comprehensive loss function can make the trained search ranking model meet the demand of achieving the optimization goal and balance the relationship between the behavior indicators and the object attribute of the recommended object, and has high applicability.

[0047] It should be noted that the pointwise loss is a loss function based on a single training sample, which is used to calculate the error between the predicted value and the true value of a single training sample. For example, the pointwise loss of a single training sample in CTR is the error between the pCTR (predicted click rate) of the single training sample and the true CTR, and the pointwise loss of a single training sample in CTCVR (Click Through Conversion Rate, click conversion rate) is the error between the pCTCVR (Predicted Click Through Conversion Rate, predicted click conversion rate) of the single training sample and the true CTCVR. The pointwise loss can be in the form of binary cross-entropy loss. Wherein, CTCVR represents the probability that a recommended object clicked by a user completes conversion, and CTCVR can be determined by combining the CTR (click rate) and CVR (conversion rate) of the recommended object.

[0048] The pairwise loss is a kind of ranking loss function, which is used to optimize the relative order between training samples. The pairwise loss calculates the relative ranking error of two training samples by comparing the predicted values of the two training samples, so as to ensure that the ranking of the training samples follows the actual order.

[0049] Based on the above idea, as an optional implementation, Figure 1An exemplary optional flowchart of a training method of a search ranking model provided by an embodiment of the present application is shown, which can be applied to a computer device, such as a computer device used by an online service platform, such as a travel service platform, for training a search ranking model.

[0050] With reference to Figure 1 The training method of the search ranking model provided by an embodiment of the present application can include the following steps.

[0051] In step S110, a plurality of training samples are obtained.

[0052] A training sample is a data instance used for training a model in machine learning. In the present application, the training sample can express interaction information between a user and a recommended object (such as a recommended hotel, a recommended transportation ticket, a recommended commodity, etc.), so that each training sample contains user-related features and recommended object-related features. Specifically, based on whether the user has performed a behavior operation corresponding to a behavior indicator on the recommended object, the sample type of the training sample can be divided into positive samples and negative samples, wherein the positive samples correspond to the user performing a behavior operation on the recommended object, and the negative samples correspond to the user not performing a behavior operation on the recommended object. The behavior operation corresponding to the behavior indicator is, for example, a purchase operation corresponding to a conversion rate (CVR), a click operation corresponding to a click rate (CTR), etc.

[0053] Further, the present application can set a plurality of behavior indicators associated with an optimization target, wherein a main behavior indicator exists, such as a behavior indicator with the highest priority among a plurality of behavior indicators with priorities, so that the main behavior indicator corresponds to the search ranking score, that is, the predicted value of the main behavior indicator is used as the search ranking score output by the search ranking model. Further, the above-mentioned behavior operation can be a behavior operation corresponding to the main behavior indicator, that is, the positive sample can correspond to a recommended object on which the behavior operation of the main behavior indicator is performed, and the negative sample can correspond to a recommended object on which the behavior operation is not performed.

[0054] For example, taking the optimization target of maximizing GMV as an example, the present application can set behavior indicators such as CTR, CTCVR, etc. associated with GMV, and CTCVR is the main behavior indicator corresponding to the search ranking score, that is, the predicted click conversion rate (pCTCVR) determined by the search ranking model is used as the search ranking score. Correspondingly, the positive sample is, for example, a converted sample, and the negative sample is, for example, a non-converted sample.

[0055] It should be noted that setting the behavior indicators such as CTR, CTCVR is only an optional example, and the embodiments of the present application can set specific optimization targets and specific types of behavior indicators associated with the optimization targets according to specific scenarios, and are not limited to CTR and CTCVR. For example, in the content recommendation scenario of short videos, the embodiments of the present application can set the optimization target as improving the user stay time (such as maximizing the user stay time), and associate the click rate, the playback rate and other behavior indicators. Similarly, setting CTCVR as the main behavior indicator is only an optional example, and the embodiments of the present application can set the type of the main behavior indicator corresponding to the search ranking score according to specific scenarios, and are not limited to CTCVR.

[0056] In an optional implementation, the online service platform can collect historical search results and user behavior on the recommended objects in the historical search results, thereby forming positive samples and negative samples, and dividing them into multiple training batches. One training batch can include multiple training samples, and the multiple training samples of each training batch contain a certain number of positive samples and negative samples. For example, the proportion of positive samples and negative samples in each training batch is set according to specific requirements. It should be noted that in the training process of machine learning, the training data is divided into multiple training batches for training processing to improve training efficiency and stability. Specifically, the training batch is a division of training data in the training process. In each iteration of the model, the training data in one training batch is processed for training, so that the training data of multiple training batches can help the model to iterate multiple times to improve the training efficiency and stability.

[0057] As an optional implementation, the multiple training samples obtained in step S110 can be multiple training samples in one training batch, which are used to perform one iteration on the search ranking model.

[0058] In an optional implementation, one training sample can include user-related features of a user and recommended object-related features of a recommended object, and distinguish whether the user has performed a behavior operation on the recommended object through the sample types of positive and negative samples. The user-related features and the recommended object-related features of the training sample can be regarded as training input features of the training sample, that is, training data.

[0059] In an optional implementation, the user-related features are used to describe the personal information, behavior and preferences of the user, to help the model understand the interests, needs, habits and the like of the user, and then provide personalized recommendations. For example, the user-related features can include but are not limited to:

[0060] User Profile, representing the basic information of the user, such as the age, gender, location, etc. of the user; further, the user information can also include the preference information of the user, such as the type of the recommended object preferred by the user, the brand of the recommended object preferred by the user, the price of the recommended object preferred by the user, etc.

[0061] User Query information, representing the search keywords and other search conditions carried by the user in the search request, such as the search keywords input by the user in the search box;

[0062] User Behavior Items, representing the features related to the historical behavior of the user, such as the historical clicks, historical purchases, and historical browsed recommended objects of the user.

[0063] In an optional implementation, the recommended object related features are used to describe the features of the recommended object, so as to understand the characteristics of the recommended object by the model, and further optimize the recommended result. For example, the recommended object related features can include but are not limited to:

[0064] Information of the recommended object, representing the basic information of the recommended object, such as the type, brand, and price of the recommended object;

[0065] Context information of the recommended object, representing the background information of the recommended object, which helps to understand the recommended scene through the timeliness and situational information; for example, the relationship information between the price of the recommended object and the time can be included in the context information of the recommended object as the time background information, so as to understand the influence of time on the price of the recommended object.

[0066] In step S120, based on the multiple behavior indicators of the user associated with the optimization target of the search ranking model, the overall point-to-point loss of the multiple training samples corresponding to each behavior indicator is determined respectively, to obtain the overall point-to-point loss corresponding to each behavior indicator.

[0067] As an optional implementation, for any behavior indicator, the embodiment of the present application can determine the overall point-to-point loss of the plurality of training samples in the behavior indicator, i.e., the overall point-to-point loss corresponding to the behavior indicator; wherein the overall point-to-point loss of the plurality of training samples in a behavior indicator is obtained by combining the point-to-point loss of the plurality of training samples in the behavior indicator, such as taking the mean of the point-to-point loss of the plurality of training samples in the behavior indicator to obtain the overall point-to-point loss of the plurality of training samples in the behavior indicator. That is, the embodiment of the present application can determine the point-to-point loss of each training sample in each behavior indicator, and for any behavior indicator, determine the overall point-to-point loss of the plurality of training samples in the behavior indicator according to the point-to-point loss of the plurality of training samples in the behavior indicator, to obtain the overall point-to-point loss corresponding to the behavior indicator.

[0068] In an optional implementation, for any behavior indicator, the embodiment of the present application can obtain the predicted value of each training sample in the behavior indicator determined by the search ranking model, so as to determine the point-to-point loss of each training sample in the behavior indicator based on the predicted value and the true value of each training sample in the behavior indicator, and then combine the point-to-point loss of each training sample in the behavior indicator to determine the overall point-to-point loss corresponding to the behavior indicator. It should be noted that the true value can be a label annotated in advance for the training sample (such as a label annotated by a human for the training sample), so the true value is also called a label value.

[0069] Further, taking the plurality of training samples as B training samples (such as B training samples contained in a training batch, i.e., B is the batch size of a training batch), then for any behavior indicator, after determining the point-to-point loss of the B training samples in the behavior indicator respectively, the overall point-to-point loss corresponding to the behavior indicator can be obtained by combining the point-to-point loss of the B training samples in the behavior indicator respectively; for example, for any behavior indicator, the mean of the point-to-point loss of the B training samples in the behavior indicator respectively can be taken to obtain the overall point-to-point loss corresponding to the behavior indicator, i.e., the overall point-to-point loss corresponding to a behavior indicator is the mean of the point-to-point loss of the plurality of training samples in the behavior indicator in a training batch, so that the overall point-to-point loss corresponding to a behavior indicator can be expressed as:

[0070]

[0071] Wherein, l(·) represents a negative log-likelihood loss function, taking the predicted value and the true value as input; for any behavior indicator, The predicted value of the i-th training sample in the behavior indicator determined by the search ranking model is y i The true value of the i-th training sample in the behavior indicator is y

[0072] For example, taking CTR as an example of the behavior index, the point-to-point loss of the i-th training sample on CTR can be expressed as:

[0073] wherein, is the predicted value of the i-th training sample on CTR (i.e., the pCTR of the i-th training sample) determined by the search ranking model, y i CTR is the true value of the i-th training sample on CTR;

[0074] Further, the point-to-point loss of the B training samples on CTR is averaged, and the overall point-to-point loss of CTR is obtained, that is, the overall point-to-point loss of CTR is the average of the point-to-point loss of multiple training samples in a training batch on CTR, which is expressed as:

[0075]

[0076] For example, taking CTCVR as an example of the behavior index, the point-to-point loss of the i-th training sample on CTCVR can be expressed as:

[0077] wherein, is the predicted value of the i-th training sample on CTCVR (i.e., the pCTCVR of the i-th training sample) determined by the search ranking model, y i CTCVR is the true value of the i-th training sample on CTCVR; the predicted value of a training sample on CTCVR can be determined by combining the predicted value of the training sample on CTR and the predicted value on CTR, such as multiplying the predicted value of the training sample on CTR and the predicted value on CTR to obtain the predicted value of the training sample on CTCVR;

[0078] Further, the point-to-point loss of the B training samples on CTCVR is averaged, and the overall point-to-point loss of CTCVR is obtained, that is, the overall point-to-point loss of CTCVR is the average of the point-to-point loss of multiple training samples in a training batch on CTCVR, which is expressed as:

[0079]

[0080] For further understanding, Figure 2 An example of the structure of the search ranking model and the example diagram of the loss function involved are shown as follows: Figure 2As shown, when training the search ranking model, the user personal information, the user search information, the user behavior item, the information of the recommended object, the context information of the recommended object, and the like included in the training sample are processed by the embedding layer of the search ranking model respectively, so as to obtain the feature vectors corresponding to the respective training input features of the training sample, such as the feature vector corresponding to the user personal information, the feature vector corresponding to the user search information, the feature vector corresponding to the user behavior item, the feature vector corresponding to the information of the recommended object, and the feature vector corresponding to the context information of the recommended object. It should be noted that the respective training input features included in the training sample can be regarded as sparse features (i.e., discrete features), and the embedding layer is used for processing, so as to map the high-dimensional discrete features to low-dimensional dense vectors, thereby obtaining the feature vectors corresponding to the respective training input features of the training sample.

[0081] The concatenate layer of the search ranking model is used to concatenate the feature vectors corresponding to the respective training input features of the training sample processed by the embedding layer, so as to form the sample feature vector of the training sample.

[0082] The deep cross network of the search ranking model is used for feature interaction learning, so as to improve the expression ability of the model. The deep cross network is composed of a deep network and a cross network. The deep network adopts a multi-layer neural network to learn the combination relationship of nonlinear features, such as an MLP (Multi Layer Perceptron) layer used to learn the combination relationship of nonlinear features. The MLP layer can be composed of multiple fully connected layers and can learn the nonlinear relationship of features. The cross network models the cross relationship of features through a cross layer, so as to capture the mutual influence between features.

[0083] Taking the CTCVR as an example, the CTCVR is determined by combining the CTR and the CVR, and the CTCVR is taken as the main behavior index. In the search ranking model, the embodiment of the present application can set the deep cross network of the CTR and the deep cross network of the CVR. Thus, the deep cross network of the CTR performs feature interaction learning of the training sample in the CTR based on the sample feature vector of the training sample concatenated by the concatenate layer, so as to obtain the feature interaction learning result of the training sample in the CTR. The deep cross network of the CVR performs feature interaction learning of the training sample in the CVR based on the sample feature vector of the training sample concatenated by the concatenate layer, so as to obtain the feature interaction learning result of the training sample in the CVR.

[0084] The CTR prediction layer of the search ranking model can determine the prediction value of the training sample in CTR, i.e., pCTR (predicted click rate), based on the feature interaction learning result of the training sample in CTR obtained by the deep cross network of CTR. The CVR prediction layer of the search ranking model can determine the prediction value of the training sample in CVR, i.e., pCVR (predicted conversion rate), based on the feature interaction learning result of the training sample in CVR obtained by the deep cross network of CVR. For example, the CTR prediction layer and the CVR prediction layer can be respectively composed of multiple rectified linear units (ReLU). ReLU is an activation function that can be used for nonlinear transformation in a neural network. For example, the CTR prediction layer is composed of multiple ReLU, which performs nonlinear transformation on the output of the deep cross network of CTR to obtain the pCTR of the training sample. The CVR prediction layer is composed of multiple ReLU, which performs nonlinear transformation on the output of the deep cross network of CVR to obtain the pCVR of the training sample.

[0085] Further, the CTCVR prediction layer of the search ranking model can combine the pCTR and the pCVR of the training sample to obtain the prediction value of the training sample in CTCVR, i.e., pCTCVR (predicted click conversion rate). For example, the CTCVR prediction layer can be composed of a point multiplication unit, which multiplies the pCTR and the pCVR of the training sample to obtain the pCTCVR of the training sample. For example, the pCTCVR obtained by the search ranking model can be used as the search ranking score.

[0086] It should be noted that the process of obtaining the prediction value of each behavior indicator by the search ranking model described above is not limited to the model training stage. The process of obtaining the prediction value of each behavior indicator of the recommended object by the search ranking model in the model using stage can be similarly referred to, except that the training sample is input in the model training stage to obtain the prediction value of each behavior indicator of the training sample, and the user-related features when the user searches and the related features of each recommended object in the search results are used as input features in the model using stage to determine the prediction value of each behavior indicator of each recommended object in the search results. Further, the prediction value of each behavior indicator obtained by the search ranking model includes the prediction value of the search ranking score, such as pCTCVR, which can be used as the search ranking score.

[0087] Further combining Figure 2As shown, in the model training stage, after the search ranking model obtains the pCTR of the training sample for any training sample, the point-to-point loss of the training sample in CTR can be determined based on the pCTR of the training sample and the true value of CTR of the training sample (i.e., the true CTR of the training sample, which can be a pre-labeled label of the training sample); then, the overall point-to-point loss of CTR is obtained by combining the point-to-point losses of multiple training samples in CTR (such as taking the average of the point-to-point losses of multiple training samples in CTR)

[0088] Similarly, after the search ranking model obtains the pCTCVR of the training sample for any training sample, the point-to-point loss of the training sample in CTCVR can be determined based on the pCTCVR of the training sample and the true value of CTCVR of the training sample (i.e., the true CTCVR of the training sample, which can be a pre-labeled label of the training sample); then, the overall point-to-point loss of CTCVR is obtained by combining the point-to-point losses of multiple training samples in CTCVR (such as taking the average of the point-to-point losses of multiple training samples in CTCVR)

[0089] In step S130, the pair loss of multiple behavior indicators between training samples is determined.

[0090] In the embodiments of the present application, the pair loss is used to optimize the rationality of ranking, so that the trained search ranking model can achieve the optimization goal, such as maximizing GMV. Based on the multiple behavior indicators of the user associated with the optimization goal and the at least one object attribute of the recommendation object associated with the optimization goal, the embodiments of the present application set a hierarchical pair loss (i.e., the pair loss is multi-layered), which is divided into a pair loss of multiple behavior indicators at the indicator level and a pair loss of object attributes affected by the attribute level. This part mainly introduces the related content of the pair loss of multiple behavior indicators at the indicator level.

[0091] The pair loss of multiple behavior indicators represents the pair loss of multiple behavior indicators between training samples, which is used to optimize the predicted ranking relationship of behavior indicators between training samples to conform to the true ranking relationship; that is, the pair loss of multiple behavior indicators is the pair loss between training samples calculated based on multiple behavior indicators, and the purpose is to optimize the predicted ranking of each behavior indicator. For example, the pair loss of multiple behavior indicators can be the pair loss corresponding to CTR and CTCVR, so as to optimize the predicted ranking relationship of CTR and CTCVR between training samples to conform to the true ranking relationship.

[0092] To facilitate the understanding of the pair loss, the calculation principle of the pair loss is introduced as follows. The pair loss is the ranking loss between training samples. Therefore, for multiple training samples in a training batch, any two training samples form a sample pair, the pair loss of each sample pair is determined, and then the pair loss of multiple sample pairs (such as taking the average of the pair loss of multiple sample pairs) is combined to obtain the overall pair loss, such as the overall pair loss of a training batch;

[0093] For example, assuming that there are B training samples in a training batch, each two training samples form a sample pair, and the pair loss is calculated, and then the average of the pair loss of all sample pairs in a training batch is taken to obtain the overall pair loss of a training batch, which is represented as:

[0094]

[0095] wherein, represents the true value of the i-th training sample under the current target O k , represents the true value of the j-th training sample under the current target O k , represents the predicted value of the i-th training sample under the current target O k , represents the predicted value of the j-th training sample under the current target O k ;

[0096] I(·) represents an indicator function, such as is used to ensure that the currently calculated pair loss occurs between sample pairs that meet the indicator function. Specifically, m represents the optimized target number, which ranges from 1 to k-1. For any two training samples i and j, it is checked whether the ranking of the true values on the optimized targets (O 1 to O k-1 ) is consistent. If the true value of the i-th training sample on the optimized target is not less than (greater than or equal to) the true value of the j-th training sample on the optimized target, the pair loss calculation of the current target O k is performed. It should be noted that the target referred to here is the target considered for optimization of the pair loss, such as CTR, CTCVR, price, etc.

[0097] represents the pair loss of the i-th training sample and the j-th training sample under the current target O k , which is used to measure the ranking error of the i-th training sample and the j-th training sample under the current target O k . Specifically, if is greater than but the model prediction does not meet greater than The corresponding pair loss is calculated.

[0098] Based on the formula structure of the overall pair loss of the above training batch, the embodiment of the application can express the overall pair loss of the training batch in a more concise formula form, specifically:

[0099]

[0100] wherein, represents the within batch, that is, traversing all sample pairs in a training batch, averaging the hierarchical ranking loss generated by each pair of samples, and ensuring comprehensive coverage of comparison;

[0101] represents the hierarchical loss indicator, which is used to indicate whether the sample pair satisfies the hierarchical relationship based on the first k-1 targets. Only when the true value of the i-th training sample is not lower than that of the j-th training sample on all processed targets, the comparison of the current target (the k-th target) is effective;

[0102] Specifically, represents whether the i-th training sample is higher than the j-th training sample on the first k-1 targets; represents whether the true value of the i-th training sample should be ranked in front of the j-th training sample under the current k-th target;

[0103] represents the ranking loss feedback, which means that the ranking comparison loss is made by using the predicted value of the model on the target O1, and the predicted difference is mapped to the interval (0, 1) by using the Sigmoid function, and then the consistency loss (such as cross entropy) is made with the ideal ranking label, that is, the hierarchical ranking constraint will be fed back to the first target O1 to improve the ranking quality of the target; wherein, σ is the symbol of the Sigmoid function, which is used to map real numbers to the interval (0, 1) in the ranking task or binary classification task of machine learning.

[0104] Further, assuming that the training data set has N training samples, and the N training samples of the training data set are divided into multiple training batches, each training batch contains B training samples, the training process corresponds to the iterative calculation of multiple training batches, until the entire training data set is traversed; further, the overall pair loss of the training data set can be the average of the overall pair loss of all training batches, such as calculating the average after summing the overall pair loss of all training batches.

[0105] Based on the determination manner of the pair loss between the training samples as described above, in an optional implementation, the embodiment of the present application can set an index layer trigger condition, so as to determine the pair loss of the plurality of behavior indexes between the training samples that meet the index layer trigger condition, such as determining the pair loss corresponding to CTR and CTCVR between the training samples that meet the index layer trigger condition. That is, in the embodiment of the present application, the determination of the pair loss of the plurality of behavior indexes has an additional trigger condition (that is, the index layer trigger condition), and the index layer trigger condition determines whether the pair loss of the plurality of behavior indexes is generated between the training samples, such as determining whether the pair loss corresponding to CTR and CTCVR is generated between the training samples. In an optional implementation, the index layer trigger condition is related to the sample type relationship between the training samples, the size relationship of the predicted value of the main behavior index, and the like. These related factors can be set as the indication function corresponding to the index layer trigger condition to constrain the sample pair condition of generating the pair loss of the plurality of behavior indexes.

[0106] As an optional implementation, for one positive sample and one negative sample, if the predicted value of the main behavior index of the positive sample is less than the predicted value of the main behavior index of the negative sample, the index layer trigger condition is met, and the pair loss of the plurality of behavior indexes between the positive sample and the negative sample needs to be determined.

[0107] In an optional implementation, Figure 3 An exemplary flowchart for determining the pair loss of the plurality of behavior indexes provided by the embodiment of the present application is shown as follows. Figure 3 As shown in the flowchart, the flowchart can include the following steps.

[0108] Step S310, obtaining a positive-negative sample pair formed by a positive sample and a negative sample.

[0109] The index layer trigger condition set by the embodiment of the present application restricts the calculation of the pair loss of the plurality of behavior indexes in the sample pair of a specific sample type relationship, specifically, the positive-negative sample pair formed by the positive sample and the negative sample. In an optional implementation, the positive sample and the negative sample in one training batch can be paired two by two to form a plurality of positive-negative sample pairs, and each positive-negative sample pair contains one positive sample and one negative sample.

[0110] Step S320, for the positive-negative sample pair, if the predicted value of the main behavior index of the positive sample is less than the predicted value of the main behavior index of the negative sample, the pair loss of the plurality of behavior indexes of the positive-negative sample pair is determined.

[0111] When the predicted value of the positive sample of a positive-negative sample pair in the main behavior index is less than the predicted value of the negative sample in the main behavior index, it indicates that the model does not assign a higher predicted value to the positive sample in the main behavior index, and there is an error in the ranking decision. At this time, the index layer trigger condition is met, and the pair loss of multiple behavior indexes of the positive-negative sample pair is calculated to optimize the prediction of the main behavior index by the model.

[0112] For example, taking the main behavior index as CTCVR, and a positive-negative sample pair containing sample A1 and sample A2 as an example, the index layer trigger condition is considered to be met in the following cases, and the pair loss of multiple behavior indexes is calculated for the positive-negative sample pair:

[0113] Sample A1 is a positive sample (such as a converted sample), and sample A2 is a negative sample (such as a non-converted sample), and pCTCVR of sample A1 < pCTCVR of sample A2;

[0114] Specifically, in the above case, the model based on the prediction of pCTCVR tends to recommend sample A2, but sample A2 is a negative sample, and there is a prediction error; for example, the model assigns a higher pCTCVR to a non-converted sample, and a lower pCTCVR to a converted sample, which is considered to be an error in prediction, and therefore the index layer trigger condition is met, triggering the calculation of the pair loss of multiple behavior indexes, and by determining the pair loss corresponding to CTR and CTCVR, the model tends to predict a higher pCTCVR for the recommended object of the positive sample.

[0115] It should be further explained that for a negative-negative sample pair in which both training samples are negative samples, the index layer trigger condition is considered not to be met, and the pair loss of multiple behavior indexes is not calculated, because the pair loss of multiple behavior indexes is mainly used to optimize the ranking prediction of behavior indexes, such as optimizing the ranking prediction of CTCVR, and both negative samples are not converted, and the model does not need to optimize the ranking prediction of CTCVR for the two negative samples; for a positive-positive sample pair in which both training samples are positive samples, the index layer trigger condition is considered not to be met, and the pair loss of multiple behavior indexes is not calculated, because the positive sample is considered to have performed a behavior operation, and therefore the influence of object attributes such as price on ranking is more concerned between positive samples. Therefore, the pair loss of multiple behavior indexes of the present application is mainly used to optimize the ranking prediction of CTCVR and other behavior indexes between positive samples and negative samples.

[0116] After determining that the sample pair (e.g., positive and negative sample pair) needs to calculate the pair loss of multiple behavior indicators based on the index layer trigger condition, the pair loss of multiple behavior indicators of the sample pair can be aggregated by the pair loss of the sample pair in each behavior indicator, such as the pair loss of the sample pair in each behavior indicator is aggregated in the form of summation or weighted summation, to form the pair loss of multiple behavior indicators of the sample pair; and whether the pair loss of the sample pair in a behavior indicator exists is determined based on whether the predicted ranking relationship of the sample pair in the behavior indicator is consistent with the real ranking relationship, so that the pair loss of the sample pair in a certain behavior indicator can not be calculated, and the corresponding loss is 0, such as when the predicted ranking relationship of the sample pair in a certain behavior indicator is consistent with the real ranking relationship, the pair loss of the behavior indicator is not calculated and is considered as 0.

[0117] That is, the pair loss of multiple behavior indicators is used to ensure that the ranking relationship (i.e., predicted ranking relationship) of the sample pair predicted by the model in each behavior indicator conforms to the real ranking relationship. Taking the example that the multiple behavior indicators include CTR and CTCVR, the pair loss of multiple behavior indicators is used to ensure that the predicted ranking relationship of CTR of the sample pair conforms to the real ranking relationship, and the predicted ranking relationship of CTCVR conforms to the real ranking relationship. Specifically, for the sample pair formed by the ith training sample and the jth training sample, if the sample pair satisfies the index layer trigger condition, and the ith training sample is higher than the jth training sample in the real CTR, the ith training sample should also be higher than the jth training sample in the pCTR, and similarly, if the ith training sample is higher than the jth training sample in the real CTCVR, the ith training sample should also be higher than the jth training sample in the pCTCVR. However, the prediction of the model can cause the predicted ranking relationship of CTR and CTCVR to be inconsistent with the real ranking relationship, for example, the real CTR of the ith training sample is higher than that of the jth training sample, but the model predicts that the pCTR of the ith training sample is lower than that of the jth training sample, and for example, the real CTCVR of the ith training sample is higher than that of the jth training sample, but the model predicts that the pCTCVR of the ith training sample is lower than that of the jth training sample.

[0118] When the predicted ranking relationship of a certain behavior indicator in the sample pair is different from the real ranking relationship, the pair loss of the sample pair in the behavior indicator is determined by the embodiment of the present application, so that the pair loss of the sample pair in each behavior indicator is aggregated into the pair loss of multiple behavior indicators of the sample pair, so that the predicted ranking relationship of CTR of the sample pair conforms to the real situation, and the predicted ranking relationship of CTCVR conforms to the real situation, thereby improving the rationality of the model.

[0119] In an optional implementation, assuming that a pair of samples satisfying the index layer trigger condition comprises an i-th training sample and a j-th training sample, and taking the pair loss of the plurality of behavior indexes as an example involving CTR and CTCVR, the pair loss of the plurality of behavior indexes of the i-th training sample and the j-th training sample can involve two parts:

[0120] The pair loss of the i-th training sample and the j-th training sample in CTR, if the real CTR of the i-th training sample is higher than that of the j-th training sample, but the prediction error of pCTR exists (that is, the pCTR of the i-th training sample is lower than that of the j-th training sample), the pair loss of the i-th training sample and the j-th training sample related to CTR is calculated;

[0121] The pair loss of the i-th training sample and the j-th training sample in CTCVR, if the real CTCVR of the i-th training sample is higher than that of the j-th training sample, but the prediction error of pCTCVR exists (that is, the pCTCVR of the i-th training sample is lower than that of the j-th training sample), the pair loss of the i-th training sample and the j-th training sample related to CTCVR is calculated;

[0122] Therefore, the pair loss of the i-th training sample and the j-th training sample in CTR and the pair loss in CTCVR are aggregated to form the pair loss of the plurality of behavior indexes between the i-th training sample and the j-th training sample.

[0123] The above describes the pair loss of the plurality of behavior indexes, and the following describes the pair loss of the object attribute influence.

[0124] Returning to Figure 1 As shown in FIG. 1, in step S140, the pair loss of the object attribute influence between the training samples is determined; wherein the optimization target of the search ranking model is further associated with at least one object attribute of the recommended object, and the pair loss of the object attribute influence is the pair loss corresponding to the influence of the object attribute on the plurality of behavior indexes.

[0125] The pair loss of the object attribute influence represents the pair loss caused by the influence of the object attribute of the recommended object on the plurality of behavior indexes, that is, the pair loss corresponding to the influence of the object attribute on the plurality of behavior indexes; that is, at least one object attribute (such as price, etc.) of the recommended object has a greater influence on the behavior index, and the influence loss (in the form of pair loss) of each object attribute on the plurality of behavior indexes needs to be calculated to optimize the rationality of the ranking. For example, the pair loss of the object attribute influence can be the pair loss corresponding to the influence of the price of the training samples on CTR and CTCVR, so as to optimize the influence of the price and other object attributes on the ranking of CTR and CTCVR, so that the ranking result conforms to the maximization of GMV.

[0126] Further, the at least one object attribute of the recommended object associated with the optimization target can specifically be a plurality of object attributes, such as a price, a number of likes, a number of views, and the like; each object attribute can have a corresponding pair loss of object attribute influence on the plurality of behavior indicators, i.e., one object attribute can have one corresponding pair loss of object attribute influence, so that a plurality of object attributes have a plurality of corresponding pair losses of object attribute influence, representing the influence of each object attribute on the plurality of behavior indicators, such as a pair loss of object attribute influence of the price of the recommended object on the CTR and the CTCVR, a pair loss of object attribute influence of the number of likes of the recommended object on the CTR and the CTCVR, and the like.

[0127] Based on the determination manner of the pair loss between the training samples as introduced above, in an optional implementation, the embodiments of the present application can set a property influence layer trigger condition, so as to determine the pair loss of object attribute influence for the training samples that meet the property influence layer trigger condition, such as determining the pair loss of object attribute influence of the price and the like of the training samples that meet the property influence layer trigger condition on the CTR and the CTCVR. Specifically, the determination of the pair loss of object attribute influence has an additional trigger condition (i.e., the property influence layer trigger condition), and the property influence layer trigger condition determines whether the pair loss of object attribute influence is generated between the training samples, wherein the property influence layer trigger condition is related to the sample type relationship between the training samples, the size relationship of the predicted value of the main behavior indicator, and the size relationship of the object attribute, and these related factors can be set as an indication function corresponding to the property influence layer trigger condition to constrain the sample pair that generates the pair loss of object attribute influence.

[0128] As an optional implementation, Figure 4 An example of the flowchart for determining the pair loss of object attribute influence provided by the embodiments of the present application is shown as follows. Figure 4 As shown in the figure, the flowchart can include the following steps.

[0129] In step S410, a positive-positive sample pair is obtained, and the positive-positive sample pair is formed by two positive samples.

[0130] The property influence layer trigger condition set by the embodiments of the present application restricts the calculation of the pair loss of object attribute influence in the sample pair of a specific sample type relationship, specifically, the positive-positive sample pair formed by two positive samples. In an optional implementation, the positive samples and the positive samples in one training batch can be paired two by two to form a plurality of positive-positive sample pairs, and each positive-positive sample pair contains two positive samples.

[0131] In step S420, for the positive-positive sample pair, the size relationship of the predicted value of the main behavior indicator of the two positive samples and the size relationship of the object attribute of the two positive samples are determined.

[0132] Step S430, if the size relationship of the two positive samples in the predicted value of the main behavior index and the size relationship of the object attribute do not match the optimization target, the pair loss of the object attribute influence of the positive-positive sample pair is determined.

[0133] In an optional implementation, for the positive-positive sample pair formed by the two positive samples, the size relationship of the two positive samples in the predicted value of the main behavior index refers to the size relationship of the predicted value (such as pCTCVR) of the two positive samples in the main behavior index predicted by the search ranking model, that is, which of the two positive samples has a larger predicted value in the main behavior index, such as which positive sample has a larger pCTCVR; the size relationship of the object attribute (such as price) of the two positive samples, that is, which of the two positive samples has a larger object attribute value, such as which positive sample has a higher price.

[0134] Further, for the positive-positive sample pair in which both training samples are positive samples, the size relationship of the predicted value of the main behavior index and the size relationship of the object attribute are compared to determine whether the size relationship of the predicted value of the main behavior index and the size relationship of the object attribute match the optimization target (such as whether they match the maximum GMV). If the size relationship of the predicted value of the main behavior index and the size relationship of the object attribute do not match the optimization target, it means that the decision of the model does not correctly achieve the optimization target (such as maximizing GMV), which is considered to satisfy the attribute influence layer trigger condition, and the pair loss of the object attribute influence needs to be calculated for the sample pair to optimize the ranking.

[0135] For example, taking the optimization target as maximizing GMV, the main behavior index as CTCVR, and the object attribute as price as an example, for the positive-positive sample pair in which both training samples are positive samples, if the pCTCVR of the positive sample with a lower price is greater than the pCTCVR of the positive sample with a higher price, the size relationship of the pCTCVR of the positive-positive sample pair and the size relationship of the price do not match the maximum GMV, which means that the model may not correctly consider the influence of the price on the GMV, and thus is considered to satisfy the attribute influence layer trigger condition, and the pair loss of the object attribute influence needs to be calculated for the positive-positive sample pair (such as calculating the pair loss corresponding to the influence of the price on CTR and CTCVR). Specifically, if the pCTCVR of the positive sample with a higher price is less than the pCTCVR of the positive sample with a lower price, the model may prefer to recommend the recommended object corresponding to the positive sample with a lower price, but since both training samples are positive samples, from the perspective of GMV, the recommended object corresponding to the positive sample with a higher price, although the CTCVR is slightly lower, may have a higher contribution to GMV, and thus should be preferentially recommended. At this time, the pair loss of the object attribute influence needs to be calculated to prompt the model to adjust the prediction so that the ranking of the recommended object with a higher price is more in line with the maximum GMV.

[0136] As an example, taking CTCVR as the behavior indicator and price as the object attribute, a sample pair containing sample Al and sample A2, the embodiment of the present application considers that the attribute influence layer triggering condition is met in the following cases, and the pair-wise loss of the object attribute influence is calculated for the sample pair:

[0137] Both sample Al and sample A2 are positive samples, pCTCVR of sample Al > pCTCVR of sample A2, but the price of sample Al < the price of sample A2;

[0138] Specifically, in the above case, the model based on the prediction of pCTCVR tends to recommend sample Al with lower price, which is not conducive to the maximization of GMV, thereby triggering the calculation of the pair-wise loss of the object attribute influence, and by determining the pair-wise loss corresponding to the influence of price on CTR and CTCVR, the model tends to predict higher pCTCVR for positive samples (converted) with higher price.

[0139] It needs to be further explained that for a negative-negative sample pair in which both training samples are negative samples, the embodiment of the present application considers that the attribute influence layer triggering condition is not met, and the pair-wise loss of the object attribute influence is not calculated, because the pair-wise loss of the object attribute influence is mainly used to optimize the contribution of the object attribute such as price to the optimization target such as GMV, and both negative samples are not converted, and do not involve the contribution of the optimization target such as GMV, so the model does not need to consider the pair-wise loss of the object attribute influence between the two negative samples; for a positive-negative sample pair in which one training sample is a positive sample and the other training sample is a negative sample, the embodiment of the present application considers that the attribute influence layer triggering condition is not met, and the pair-wise loss of the object attribute influence is not calculated, because the prediction optimization of the behavior indicator such as CTCVR between the positive sample and the negative sample is more concerned. Thus, the pair-wise loss of the object attribute influence of the embodiment of the present application is mainly for the influence of the object attribute such as price on the ranking of the behavior indicator such as CTCVR between positive samples.

[0140] After triggering the attribute influence layer condition based on the attribute influence layer trigger condition, determining the sample pair (such as a positive-positive sample pair) that meets the attribute influence layer trigger condition, the embodiments of the present application can calculate the pair loss of the object attribute influence for the sample pair. The pair loss of the object attribute influence is mainly determined based on the influence of the object attribute (such as price) on multiple behavior indicators (such as CTR, CTCVR), ensuring that the contribution of the object attribute to the optimization target (such as GMV) conforms to the logic, so that the ranking optimization conforms to the implementation of the optimization target (such as GMV maximization). As an optional implementation of determining the pair loss of the object attribute influence for the sample pair, the embodiments of the present application can determine the pair loss of the object attribute on each behavior indicator, and aggregate the pair loss of the object attribute on each behavior indicator to form the pair loss of the object attribute influence for the sample pair, that is, for the sample pair, the pair loss of the object attribute influence is aggregated by the pair loss of the object attribute on each behavior indicator.

[0141] For example, assuming that the pair of sample pairs that meet the attribute influence layer trigger condition include the i th training sample and the j th training sample, the pair loss of the object attribute (such as price) on a certain behavior indicator (such as CTR or CTCVR) can be expressed as:

[0142]

[0143] wherein, is the predicted value of the i th training sample at a certain behavior indicator, is the predicted value of the j th training sample at a certain behavior indicator, P i is the object attribute (such as price) of the i th training sample, P j is the object attribute (such as price) of the j th training sample, so that I(P i <P j ) represents an indicator function, which is true when P i <P j ; and further, the pair loss of the object attribute on each behavior indicator is aggregated by summation or weighted summation, and the pair loss of the object attribute influence for the sample pair can be obtained.

[0144] By setting the attribute influence layer trigger condition, the embodiments of the present application can optimize the ranking of the recommended object by the model, so that the ranking result not only considers the behavior indicators such as CTCVR, but also more reasonably balances the influence of the object attributes such as price on the optimization target such as GMV, thereby optimizing the overall recommendation strategy.

[0145] It needs to be further explained that in the embodiments of the present application, the CTCVR and other behavior indicators can be used as high-priority targets, and the price and other object attributes can be used as low-priority targets, so that the embodiments of the present application can offset the loss of low-priority targets in the case of loss of high-priority targets, thereby preventing low-priority targets from interfering with high-priority targets during model training. Specifically, the calculation of the pairwise loss of multiple behavior indicators of the sample pair occurs between the positive sample and the negative sample, and is calculated based on whether the predicted ranking relationship of the behavior indicators is consistent with the true ranking relationship, and is not affected by the price and other object attributes; that is, the pairwise loss of multiple behavior indicators ensures that the predicted ranking of high-priority targets such as CTCVR and CTR of the positive sample corresponds to the true situation, thereby ensuring that the ranking of the recommended object with high CTR and high CTCVR is earlier, and is not affected by the interference of low-priority targets such as price, so that the model can prevent the prediction of behavior indicators such as CTR and CTCVR from being inaccurate due to excessive optimization of object attributes such as price, and avoid recommending recommended objects with low CTR, low CTCVR but high price. Moreover, the calculation of the pairwise loss of the object attribute influence of the sample pair is between the positive samples, such as optimizing the influence of the price and other object attributes on the ranking, so that for the positive samples with high CTR and CTCVR, the influence of low-priority targets such as price on behavior indicators such as CTR and CTCVR is considered, so that the model can determine the loss of the influence of object attributes such as price on CTR and CTCVR ranking on the premise of ensuring the rationality of the prediction of behavior indicators (CTR and CTCVR), in order to optimize GMV.

[0146] Back to Figure 1 As shown in FIG. 1 1, in step S150, the overall pairwise loss of the multiple training samples is determined according to the pairwise loss of the multiple behavior indicators and the pairwise loss of the object attribute influence.

[0147] After determining the pairwise loss of multiple behavior indicators between training samples and the pairwise loss of object attribute influence, the embodiments of the present application can take the mean of the pairwise loss of multiple behavior indicators between training samples to obtain the mean pairwise loss of multiple behavior indicators, such as taking the mean of the pairwise loss of multiple behavior indicators of multiple sample pairs that meet the index layer trigger condition to obtain the mean pairwise loss of multiple behavior indicators; at the same time, the mean of the pairwise loss of object attribute influence between training samples is taken to obtain the mean pairwise loss of object attribute influence, such as taking the mean of the pairwise loss of object attribute influence of multiple sample pairs that meet the attribute influence layer trigger condition to obtain the mean pairwise loss of object attribute influence;

[0148] Further, the pair loss mean of the plurality of behavior indicators and the pair loss mean of the object attribute influence are aggregated in a sum or the like manner to form the overall pair loss, i.e., the hierarchical pair loss.

[0149] For further understanding, combined with Figure 2 As shown in FIG. 6, taking the CTR, CTCVR and the price as examples of the plurality of behavior indicators and the object attribute, the pair loss mean of the plurality of behavior indicators of the plurality of training samples is represented as which is obtained by taking the mean of the pair loss of the plurality of behavior indicators between the training samples; the pair loss mean of the object attribute influence of the plurality of training samples is represented as which is obtained by taking the mean of the pair loss of the object attribute influence between the training samples; further, the pair loss mean of the plurality of behavior indicators and the pair loss mean of the object attribute influence of the plurality of training samples are added to obtain the overall pair loss, which is represented as:

[0150]

[0151] Step S160, according to the overall point-to-point loss of each behavior indicator and the overall pair loss of the plurality of training samples, determine the comprehensive loss function.

[0152] As an optional implementation, after obtaining the overall point-to-point loss of each behavior indicator and the overall pair loss, the embodiment of the present application can perform weighted sum processing on the overall point-to-point loss of each behavior indicator and the overall pair loss based on the preset weight corresponding to the point-to-point loss of each behavior indicator and the weight corresponding to the overall pair loss, thereby obtaining the comprehensive loss function.

[0153] For example, taking the CTR, CTCVR and the price as examples of the plurality of behavior indicators and the object attribute, the embodiment of the present application can perform weighted sum processing on the overall point-to-point loss of CTR, the overall point-to-point loss of CTCVR and the overall pair loss to obtain the comprehensive loss function Loss, which is represented as:

[0154]

[0155] Wherein, α is the weight corresponding to the point-to-point loss of CTCVR, and β is the weight corresponding to the overall pair loss, i.e., the weight corresponding to the hierarchical pair loss.

[0156] It should be noted that the specific values of the weights corresponding to the point-to-point loss of each behavior indicator and the weights corresponding to the overall pairwise loss can be set according to actual conditions, such as the importance of the point-to-point loss of the behavior indicator to search ranking and the importance of the overall pairwise loss to search ranking, and the specific values of the weights of each behavior indicator and the weight of the overall pairwise loss are not limited by the embodiments of the present application.

[0157] In step S170, the search ranking model is trained based on the comprehensive loss function.

[0158] In optional implementations, the embodiments of the present application can determine a comprehensive loss function once using multiple training samples in a training batch, thereby adjusting the model parameters of the search ranking model once, and iteratively adjusting the search ranking model multiple times through multiple training batches until the training is completed. As an optional implementation, the embodiments of the present application can adjust the model parameters of the search ranking model multiple times by minimizing the comprehensive loss function, such as calculating the gradient of the comprehensive loss function by gradient descent, and updating the model parameters of the search ranking model based on the gradient, so that the model can more accurately predict CTR, CTCVR, and reasonably consider the influence of object attributes such as price.

[0159] The training method of the search ranking model provided by the embodiments of the present application can optimize the search ranking model, so that the search ranking of the recommended object determined based on the search ranking model meets the demand of achieving the optimization target, and provides support for improving the service capability of the online service platform.

[0160] The embodiments of the present application also provide a search ranking method, which can be applied to an online service platform such as a travel service platform. Specifically, the online service platform responds to a user search request and determines the ranking of a recommended object in a search result by using the search ranking model trained by the training method of the search ranking model provided by the embodiments of the present application. The following description can be referred to each other.

[0161] In optional implementations, the online service platform such as a travel service platform can include an online server on the service side and a client on the user side, and the search ranking method can be applied to the online server on the service side, or applied to the client on the user side, or the online server and the client cooperate to execute the search ranking method. As an optional implementation, Figure 5 An optional flowchart of the search ranking method provided by the embodiments of the present application is shown by way of example, and the flowchart is described with reference to Figure 5 The flowchart can include the following steps.

[0162] In step S510, a user's search request is obtained.

[0163] In step S520, a plurality of recommended objects corresponding to the search request are determined.

[0164] The search request of the user can carry a search condition such as a search keyword, so that the online service platform can search for a plurality of recommended objects corresponding to the search request based on the search condition such as the search keyword in the search request.

[0165] In step S530, for any recommended object, a model input feature of the recommended object is determined based on the recommended object-related feature and the user-related feature of the user.

[0166] As an optional implementation, for any recommended object, the model input feature of the recommended object can be determined based on the user-related feature (such as user personal information, user search information, user behavior items, etc.) of the user who proposes the search request and the recommended object-related feature (such as information of the recommended object, context information of the recommended object, etc.) of the recommended object. The specific content of the user-related feature and the recommended object-related feature can be referred to the description in the corresponding part of the foregoing description, which will not be expanded here.

[0167] In step S540, the model input feature of each recommended object is transmitted to the search ranking model to obtain a search ranking score of each recommended object determined by the search ranking model.

[0168] The model input feature of each recommended object can be transmitted to the search ranking model, so that the search ranking model determines the search ranking score of each recommended object based on the model input feature of each recommended object, such as determining the pCTCVR of each recommended object. The related content can be referred to the description of the model training stage of the search ranking model, which will not be expanded here. The related content of the search ranking model can also be referred to the foregoing description, which will not be expanded here. Figure 2

[0169] In step S550, the ranking of each recommended object in the search result is determined based on the search ranking score of each recommended object, and is displayed to the user.

[0170] After obtaining the search ranking score of each recommended object, the embodiment of the present application can determine the ranking of each recommended object in the search result according to the search ranking score of each recommended object, such as ranking the recommended objects in descending order of the search ranking score, so that the recommended objects ranked in the search result can be displayed to the user to respond to the search request of the user.

[0171] In an optional implementation, the search ranking score of the recommended object is a predicted value of the recommended object in a main behavior indicator determined by the search ranking model, wherein the optimization target of the search ranking model is associated with a plurality of behavior indicators of the user, the plurality of behavior indicators have priorities, and the main behavior indicator is a behavior indicator with the highest priority.

[0172] ​It can be understood that, in the model training process, the loss function considers high-priority targets such as CTR, CTCVR, and low-priority targets such as price, so that the trained search ranking model can not only consider the accuracy of the predicted values of behavior indicators such as CTR and CTCVR in the model use stage, but also consider the influence of price on CTR and CTCVR when predicting the predicted value of the main behavior indicator CTCVR (as the search ranking score) to optimize the search ranking score of the recommended object, such as optimizing the pCTCVR of the recommended object. That is, the related loss of low-priority targets such as price can be propagated to the prediction of the search ranking score in the model use stage after the model is trained in the model training stage, so that the search ranking model can reflect the influence of object attributes such as price although the predicted value of the main behavior indicator (such as pCTCVR) is used as the search ranking score, so that the search ranking can achieve the optimization goal of maximizing GMV.

[0173] Further, taking the optimization of GMV as an example, the search ranking model provided by the embodiment of the present application is compared with the traditional GMV optimization method (such as formula method, sample weighting method, etc.) on the test set of the offline data set, and experiments are compared on different backbone models. The search ranking model provided by the embodiment of the present application is superior to the traditional GMV optimization method such as formula method, sample weighting method, etc. in the prediction of CTCVR index; at the same time, in the online contrast experiment, the search ranking model provided by the embodiment of the present application improves the CTCVR and GMV of the recommended object in the search scene, and the model effect is effective.

[0174] The embodiment of the present application considers the multi-behavior indicators such as CTR and CTCVR associated with the optimization goal and the object attributes such as price, proposes a hierarchical loss framework, which is specifically composed of point-to-point loss of behavior indicators, and hierarchical pair loss composed of pair loss of multiple behavior indicators and pair loss of object attribute influence, so as to train the optimization search ranking model through the hierarchical loss framework, realize the optimization of search ranking, and achieve the optimization goal of maximizing GMV without reducing the behavior indicators such as CTCVR; further, by setting the index layer trigger condition and the attribute influence layer trigger condition, the sample pairs generating the pair loss of multiple behavior indicators and the sample pairs generating the pair loss of object attribute influence are determined, the optimization process of high-priority targets such as CTR and CTCVR is avoided from being disturbed by low-priority targets such as price, and the optimization is ensured to be reasonable. Therefore, the embodiment of the present application can optimize the search ranking model, so that the search ranking of the recommended object determined based on the search ranking model meets the demand of achieving the optimization goal, and balances the relationship between the behavior indicators such as CTCVR and the object attributes such as price, thereby providing support for improving the service capability of the online service platform.

[0175] In a further optional implementation, the embodiment of the present application further provides a training apparatus of a search ranking model, as an optional implementation, Figure 6 An exemplary block diagram of the training apparatus of the search ranking model provided by the embodiment of the present application is shown, which can be applied to a computer device for training a search ranking model, such as a computer device for training a search ranking model used by an online service platform, such as a travel service platform, with reference to Figure 6 The apparatus can comprise:

[0176] The sample obtaining module 610 is configured to obtain a plurality of training samples.

[0177] The point-to-point loss determining module 620 is configured to determine a plurality of overall point-to-point losses of the plurality of training samples corresponding to a plurality of behavior indicators respectively based on a plurality of behaviors of a user associated with an optimization target of the search ranking model.

[0178] The behavior indicator pair loss determining module 630 is configured to determine a plurality of behavior indicator pair losses between the training samples.

[0179] The attribute influence pair loss determining module 640 is configured to determine an object attribute influence pair loss between the training samples, wherein the optimization target of the search ranking model is further associated with at least one object attribute of a recommended object, and the object attribute influence pair loss is a pair loss corresponding to an influence of the object attribute on the plurality of behavior indicators.

[0180] The overall pair loss determining module 650 is configured to determine an overall pair loss of the plurality of training samples according to the plurality of behavior indicator pair losses and the object attribute influence pair loss.

[0181] The comprehensive loss determining module 660 is configured to determine a comprehensive loss function according to the overall point-to-point losses of the plurality of training samples corresponding to the plurality of behavior indicators respectively and the overall pair loss.

[0182] The training executing module 670 is configured to train the search ranking model based on the comprehensive loss function.

[0183] The related content of each functional module described above can refer to the foregoing description, which will not be expanded here.

[0184] In a further optional implementation, the embodiment of the present application further provides a search ranking apparatus, as an optional implementation, Figure 7 An exemplary block diagram of the search ranking apparatus provided by the embodiment of the present application is shown, which can be applied to an online service platform, such as an online service platform, such as a travel service platform, with reference to Figure 7 The apparatus can comprise:

[0185] The request obtaining module 710 is configured to obtain a search request of a user.

[0186] The recommended object determining module 720 is configured to determine a plurality of recommended objects corresponding to the search request.

[0187] The model input feature determining module 730 is configured to determine, for any recommended object, a model input feature of the recommended object based on a recommended object related feature and a user related feature of the user.

[0188] The score determining module 740 is configured to pass the model input feature of each recommended object to a search ranking model to obtain a search ranking score of each recommended object determined by the search ranking model, wherein the search ranking model is trained based on the training method of the search ranking model provided in the embodiments of the present application.

[0189] The ranking and displaying module 750 is configured to determine the ranking of each recommended object in the search result based on the search ranking score of each recommended object, and display the ranking to the user.

[0190] The related content of each functional module described above can be referred to the foregoing description, which will not be expanded here.

[0191] In further optional implementations, the embodiments of the present application further provide a computer device, such as a computer device used for training the search ranking model, or a computer device (such as a server device) corresponding to an online service platform, which can include at least one memory and at least one processor. The memory stores computer executable instructions, and the processor invokes the computer executable instructions stored in the memory to execute the training method of the search ranking model provided in the embodiments of the present application, or the search ranking method provided in the embodiments of the present application.

[0192] In further optional implementations, the embodiments of the present application further provide a storage medium, which stores computer executable instructions. When the computer executable instructions are executed (such as when the computer executable instructions are executed by the processor), the training method of the search ranking model provided in the embodiments of the present application, or the search ranking method provided in the embodiments of the present application is implemented.

[0193] In further optional implementations, the embodiments of the present application further provide a computer program product, which includes computer executable instructions. When the computer executable instructions are executed (such as when the computer executable instructions are executed by the processor), the training method of the search ranking model provided in the embodiments of the present application, or the search ranking method provided in the embodiments of the present application is implemented.

[0194] The above describes a plurality of embodiment schemes provided by the embodiments of the present application. The optional modes introduced by each embodiment scheme can be combined with each other and cross-referenced in the case of no conflict, thereby extending a plurality of possible embodiment schemes, which can be considered as the embodiments disclosed and published by the embodiments of the present application.

[0195] Although the embodiments of the present application are disclosed as above, the present application is not limited to this. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and therefore the protection scope of the present application should be subject to the range defined by the claims.

Claims

1. A method for training a search ranking model, the method comprising: The method comprises: obtaining a plurality of training samples; determining a plurality of behavior indicators of a user associated with an optimization target of a search ranking model, and determining a plurality of overall point-to-point losses of the plurality of training samples corresponding to each behavior indicator respectively; determining a plurality of pair losses of the plurality of behavior indicators between the training samples, and determining a pair loss of an object attribute influence between the training samples; wherein the optimization target of the search ranking model is further associated with at least one object attribute of a recommended object, and the pair loss of the object attribute influence is a pair loss corresponding to an influence of the object attribute on the plurality of behavior indicators; determining an overall pair loss of the plurality of training samples according to the pair losses of the plurality of behavior indicators and the pair loss of the object attribute influence; determining a comprehensive loss function according to the overall point-to-point losses of the plurality of training samples corresponding to each behavior indicator respectively and the overall pair loss; training the search ranking model based on the comprehensive loss function.

2. The method of claim 1, wherein, The plurality of behavior indicators have priorities, wherein a behavior indicator with the highest priority is a main behavior indicator, and a predicted value of the main behavior indicator is used as a search ranking score output by the search ranking model.

3. The method of claim 2, wherein, The sample types of the training samples are divided into positive samples and negative samples, the positive samples correspond to recommended objects that have performed a behavior operation of the main behavior indicator, and the negative samples correspond to recommended objects that have not performed the behavior operation; wherein the pair losses of the plurality of behavior indicators are determined by an index layer trigger condition, and the pair loss of the object attribute influence is determined by an attribute influence layer trigger condition; the index layer trigger condition is related to a sample type relationship between the training samples, and a size relationship of the predicted value of the main behavior indicator; and the attribute influence layer trigger condition is related to the sample type relationship between the training samples, the size relationship of the predicted value of the main behavior indicator, and a size relationship of the object attribute.

4. The method according to claim 2 or 3, characterized in that, The determination of the pair losses of the plurality of behavior indicators between the training samples comprises: obtaining a positive-negative sample pair formed by a positive sample and a negative sample; if the predicted value of the positive sample in the main behavior indicator is less than the predicted value of the negative sample in the main behavior indicator, determining a pair loss of the plurality of behavior indicators of the positive-negative sample pair; wherein the pair loss of the plurality of behavior indicators of the positive-negative sample pair is aggregated from pair losses of the positive-negative sample pair in each behavior indicator, and whether the pair loss of the positive-negative sample pair in a behavior indicator exists is determined based on whether a predicted ranking relationship of the positive-negative sample pair in the behavior indicator matches a real ranking relationship.

5. The method according to claim 2 or 3, characterized in that, The determination of the pair loss of the object attribute influence between the training samples comprises: obtaining a positive-positive sample pair formed by two positive samples; determining a size relationship of the predicted value of the main behavior indicator of the two positive samples, and a size relationship of the object attribute of the two positive samples; if the size relationship of the predicted value of the main behavior indicator of the two positive samples does not match the size relationship of the object attribute of the two positive samples, determining a pair loss of the object attribute influence of the positive-positive sample pair; wherein for the positive-positive sample pair, the pair loss of the object attribute influence is aggregated from pair losses of the object attribute on each behavior indicator.

6. The method of claim 1, wherein, The determining the overall pairwise loss of the plurality of training samples according to the pairwise loss of the plurality of behavior indicators and the pairwise loss of the object attribute influence includes: taking the mean of the pairwise loss of the plurality of behavior indicators between the training samples to obtain the mean of the pairwise loss of the plurality of behavior indicators; taking the mean of the pairwise loss of the object attribute influence between the training samples to obtain the mean of the pairwise loss of the object attribute influence; aggregating the mean of the pairwise loss of the plurality of behavior indicators and the mean of the pairwise loss of the object attribute influence to form the overall pairwise loss of the plurality of training samples; The determining the comprehensive loss function according to the overall point-to-point loss of each behavior indicator and the overall pairwise loss includes: weighting and summing the overall point-to-point loss of each behavior indicator and the overall pairwise loss based on the weight corresponding to the point-to-point loss of each behavior indicator and the weight corresponding to the overall pairwise loss to obtain the comprehensive loss function.

7. A method of search ranking, characterized by, The method includes: obtaining a search request of a user; determining a plurality of recommended objects corresponding to the search request; for any recommended object, determining a model input feature of the recommended object based on a recommended object related feature and a user related feature of the user; delivering the model input feature of each recommended object to a search ranking model to obtain a search ranking score of each recommended object determined by the search ranking model; wherein the search ranking model is trained based on the training method of the search ranking model of any one of claims 1-6; determining the ranking of each recommended object in the search result based on the search ranking score of each recommended object and displaying to the user.

8. The method of claim 7, wherein, The search ranking score of the recommended object is the predicted value of the recommended object in the main behavior indicator determined by the search ranking model, wherein the optimization target of the search ranking model is associated with a plurality of behavior indicators of the user, the plurality of behavior indicators have priority, and the main behavior indicator is the behavior indicator with the highest priority.

9. A computer device, comprising: The device includes a memory and a processor, the memory stores computer execution instructions, and the processor invokes the computer execution instructions to execute the training method of the search ranking model of any one of claims 1-6 or the search ranking method of any one of claims 7-8.

10. A computer program product, characterised in that, The computer execution instructions are executed to implement the training method of the search ranking model of any one of claims 1-6 or the search ranking method of any one of claims 7-8.

Images