Method and apparatus for updating recommendation model

Abstract

The application discloses a recommendation model updating method and device. A specific embodiment of the method comprises: determining a recommendation score of each candidate object in a candidate object set based on a pre-trained recommendation model; adjusting the recommendation score of a candidate object that is likely to be underestimated by the recommendation model in the candidate object set to obtain a corrected score; generating recommendation information according to a final score of each candidate object in the candidate object set; and updating the recommendation model according to feedback information for the recommendation information. The application makes the updated recommendation model capable of solving the underestimation problem for long-tail objects and improving the exploration efficiency of the recommendation model when facing the exploitation-exploration dilemma.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, in particular to a method and device for updating a recommendation model. BACKGROUND

[0002] The application of deep learning technology greatly promotes the development of recommendation system. The basic idea of the recommendation system using deep learning technology is to learn user preferences and object attributes from user historical behaviors by using the expression ability of deep neural network. Deep learning model often uses user feedback information on recommended objects for training in a supervised learning manner. Taking a click rate prediction model as an example, whether a product is exposed depends on the estimation result of the click rate estimation model; and the click of the exposed product affects the update training of the click rate estimation model. Thus, a training-estimation iterative closed loop for the click rate estimation model is formed. In this closed loop, the click rate estimation model updated iteratively gradually focuses on a small number of high click rate products; on the contrary, a large number of long-tail products are difficult to obtain sufficient exposure. This leads to the problem that the products whose click rates are underestimated by the click rate prediction model are difficult to obtain sufficient exposure opportunities, and thus lose their deserved click volume, resulting in the problem of underestimation of the click rate of long-tail products. SUMMARY

[0003] Embodiments of the present application provide a method and device for updating a recommendation model.

[0004] In a first aspect, embodiments of the present application provide a method for updating a recommendation model, comprising: determining a recommendation score of each candidate object in a candidate object set based on a pre-trained recommendation model; adjusting the recommendation score of a candidate object in the candidate object set that is likely to be underestimated by the recommendation model to obtain a corrected score; generating recommendation information according to the final score of each candidate object in the candidate object set; and updating the recommendation model according to feedback information for the recommendation information.

[0005] In some embodiments, the above determining a recommendation score of each candidate object in a candidate object set based on a pre-trained recommendation model comprises: for each candidate object in the candidate object set, performing the following operations: obtaining a representation vector and an initial score of the candidate object by the recommendation model; and representing the uncertainty of the initial score by the length of the representation vector to determine the recommendation score of the candidate object.

[0006] In some embodiments, the above representing the uncertainty of the initial score by the length of the representation vector to determine the recommendation score of the candidate object comprises: constructing normal distribution information corresponding to the candidate object with the initial score as the mean value and the length of the representation vector as the standard deviation; and sampling the recommendation score of the candidate object from the normal distribution information.

[0007] In some embodiments, the adjusting the recommendation score of the candidate object that is likely to be underestimated by the recommendation model in the candidate object set to obtain a corrected score comprises: for each candidate object in the candidate object set, in response to determining that the recommendation score of the candidate object is not greater than a first threshold value, and the number of recommendations of the candidate object in the historical recommendation information corresponding to the recommendation information is not greater than a second threshold value, adjusting the recommendation score of the candidate object based on a preset manner to obtain the corrected score of the candidate object, wherein the number of recommendations and the adjustment range of the corrected score compared to the recommendation score are negatively correlated.

[0008] In some embodiments, the adjusting the recommendation score of the candidate object based on the preset manner to obtain the corrected score of the candidate object comprises: obtaining an adjustment coefficient based on the number of recommendations of the candidate object and a preset parameter; and obtaining the corrected score of the candidate object according to the adjustment coefficient and the recommendation score of the candidate object.

[0009] In some embodiments, the method further comprises: performing a subsequent recommendation task by using the updated recommendation model.

[0010] In some embodiments, the recommendation score of each candidate object in the candidate object set represents the click rate of the user on the candidate object when the candidate object is recommended.

[0011] In a second aspect, the embodiments of the present application provide an updating device of a recommendation model, comprising: a determination unit configured to determine a recommendation score of each candidate object in a candidate object set based on a pre-trained recommendation model; a correction unit configured to adjust the recommendation score of a candidate object that is likely to be underestimated by the recommendation model in the candidate object set to obtain a corrected score; a generation unit configured to generate recommendation information according to the final score of each candidate object in the candidate object set; and an updating unit configured to update the recommendation model according to feedback information corresponding to the recommendation information.

[0012] In some embodiments, the determination unit is further configured to: for each candidate object in the candidate object set, perform the following operations: obtaining a representation vector and an initial score of the candidate object by using the recommendation model; and determining the recommendation score of the candidate object by taking the length of the representation vector as the uncertainty of the initial score.

[0013] In some embodiments, the determination unit is further configured to: construct normal distribution information corresponding to the candidate object by taking the initial score as the mean value and taking the length of the representation vector as the standard deviation; and sampling the recommendation score of the candidate object from the normal distribution information.

[0014] In some embodiments, the correction unit is further configured to: for each candidate object in the set of candidate objects, in response to determining that the recommendation score of the candidate object is not greater than a first threshold, and that a number of recommendations of the candidate object in the historical recommendation information corresponding to the recommendation information is not greater than a second threshold, adjusting the recommendation score of the candidate object based on a preset manner to obtain a corrected score of the candidate object, wherein the number of recommendations and the adjustment range of the corrected score compared to the recommendation score are negatively correlated.

[0015] In some embodiments, the correction unit is further configured to: obtain an adjustment coefficient based on the number of recommendations of the candidate object and a preset parameter; and obtain the corrected score of the candidate object according to the adjustment coefficient and the recommendation score of the candidate object.

[0016] In some embodiments, the apparatus further includes an execution unit configured to execute a subsequent recommendation task by using the updated recommendation model.

[0017] In some embodiments, the recommendation score of each candidate object in the set of candidate objects represents a click rate of a user with respect to the candidate object in a case where the candidate object is recommended.

[0018] In a third aspect, an embodiment of the present application provides a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0020] The method and apparatus for updating a recommendation model provided by the embodiments of the present application can determine a recommendation score of each candidate object in a set of candidate objects based on a pre-trained recommendation model; adjust the recommendation score of a candidate object that is likely to be underestimated by the recommendation model to obtain a corrected score; generate recommendation information according to the final score of each candidate object in the set of candidate objects; and update the recommendation model according to feedback information with respect to the recommendation information, so that the updated recommendation model can solve the problem of underestimation of long-tail objects, and improve the exploration efficiency of the recommendation model in the face of exploitation-exploration dilemma. BRIEF DESCRIPTION OF DRAWINGS

[0021] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments thereof, read in conjunction with the accompanying drawings:

[0022] Figure 1is an exemplary system architecture diagram in which an embodiment of the present application can be applied;

[0023] Figure 2 is a flow chart of an embodiment of a method for updating a recommendation model according to the present application;

[0024] Figure 3 is a schematic diagram of an application scenario of a method for updating a recommendation model according to the present embodiment;

[0025] Figure 4 is a flow chart of another embodiment of a method for updating a recommendation model according to the present application;

[0026] Figure 5 is a flow chart of an embodiment of a method for modeling uncertainty according to the present application;

[0027] Figure 6 is a flow chart of another embodiment of a method for modeling uncertainty according to the present application;

[0028] Figure 7 is a structural diagram of an embodiment of an updating device for a recommendation model according to the present application;

[0029] Figure 8 is a structural schematic diagram of a computer system suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application will be further described below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for explaining the related application, and are not intended to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for the convenience of description.

[0031] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0032] Figure 1 An exemplary architecture 100 to which the method and device for updating a recommendation model of the present application can be applied is shown.

[0033] As shown in Figure 1 , the system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The terminal devices 101, 102, 103 are communicatively connected to form a topological network, and the network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0034] The terminal devices 101, 102, and 103 can be hardware devices or software that support network connection to perform data interaction and data processing. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices that support network connection, information acquisition, interaction, display, processing, and the like, including but not limited to smart phones, tablet computers, e-book readers, laptop computers, desktop computers, and the like. When the terminal devices 101, 102, and 103 are software, they can be installed in the above-listed electronic devices. They can be implemented as multiple software or software modules, for example, to provide distributed services, or as a single software or software module. No specific limitation is made herein.

[0035] The server 105 can be a server that provides various services, such as a background processing server that acquires request information sent by a user through a terminal device 101, 102, or 103 and recommends objects to the user. The server is provided with a recommendation model, and by adjusting the recommendation score of a candidate object that can be underestimated by the recommendation model, the recommendation opportunity of the underestimated candidate object (for example, a long-tail commodity) is increased. Optionally, the server can feed back the determined recommended object to the terminal device. As an example, the server 105 can be a cloud server.

[0036] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or as a single software or software module. No specific limitation is made herein.

[0037] It should also be noted that the recommendation model updating method provided by the embodiments of the present application can be executed by the server, or by the terminal device, or by the server and the terminal device in cooperation with each other. Accordingly, the recommendation model updating apparatus includes various parts (for example, units) that can be all arranged in the server, or all arranged in the terminal device, or arranged in the server and the terminal device respectively.

[0038] It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0039] With reference back to Figure 2FIG. 200 shows a flow of one embodiment of the updating method of the recommendation model, including the following steps:

[0040] Step 201, determining the recommendation score of each candidate object in the candidate object set based on the pre-trained recommendation model.

[0041] In this embodiment, the execution subject (e.g. Figure 1 of the updating method of the recommendation model, such as a terminal device or a server, can determine the recommendation score of each candidate object in the candidate object set based on the pre-trained recommendation model.

[0042] The recommendation model selects the target object matching the target user from the candidate object set based on the operation request of the target user. The recommendation score of each candidate object in the candidate object set is used to represent the possibility of the target user triggering the candidate object if the candidate object is recommended to the target user, and the recommendation score of each candidate object is positively correlated with the possibility of being triggered. In a specific application scenario, the way the target user triggers the candidate object may be, for example, clicking, reviewing, browsing, etc.

[0043] The recommendation model can use a machine learning method to learn the preference information of the target user and the attribute information of the candidate object from the historical behavior of the target user about the candidate object. Specifically, first, the above execution subject collects the historical behavior of each user about the object (for example, collects historical behavior data about the commodity object from the database of the e-commerce platform), and constructs a training sample set. Among them, the training sample in the training sample set includes the attribute information of the object and the historical behavior of the user about the candidate object. Then, using a machine learning method, taking the attribute information of the object as input and the historical behavior of the user about the object as expected output, the recommendation model is trained to predict the possibility of the user triggering the object according to the attribute information of the object, and then recommend the object to the user.

[0044] Among them, the recommendation model can use various deep learning models that can realize the recommendation function, including but not limited to recurrent neural networks, convolutional neural networks, and residual neural networks.

[0045] The candidate objects in the candidate object set can be any object, including but not limited to various commodities, books, video data, advertisements, etc. The recommendation model is used to determine the recommendation score of each object in the object set according to the search, browsing, etc. request information of the target user.

[0046] Step 202, adjusting the recommendation score of the candidate object that may be underestimated by the recommendation model in the candidate object set to obtain a corrected score.

[0047] In this embodiment, the execution subject can adjust the recommendation score of the candidate object that is likely to be underestimated by the recommendation model in the candidate object set, to obtain a corrected score.

[0048] Whether a candidate object is recommended depends on the estimation result (recommendation score) of the recommendation model, and the feedback information of the target user for the recommended candidate object affects the update training of the recommendation model. Thus, a training-estimation iterative closed loop about the recommendation model is formed. In this closed loop, the gradually updated iterative recommendation model tends to focus on a small number of objects with high recommendation rates; in contrast, a large number of long-tail objects (such as long-tail commodities) are difficult to obtain sufficient recommendation times, resulting in problems such as insufficient exposure (recommendation times) of long-tail objects, inaccurate recommendation scores, and lack of feedback information.

[0049] However, the recommendation model is trained based on a deep learning model, and the information processing process thereof is not interpretable. The estimation of the recommendation model for the candidate objects in the candidate object set may be underestimated. It can be understood that the candidate objects with higher recommendation scores are more likely to be recommended, and thus the feedback information for the candidate objects with higher recommendation scores is sufficient, and it is basically impossible to be underestimated by the recommendation model. The candidate objects with lower recommendation scores are less likely to be recommended, and thus the feedback information for the candidate objects with lower recommendation scores is lacking, and they may be underestimated by the recommendation model in the process of multiple update iterations.

[0050] As an example, the execution subject can set a recommendation score threshold, determine the candidate objects in the candidate object set whose recommendation scores are lower than the recommendation score threshold as the candidate objects that are likely to be underestimated by the recommendation model, and increase the recommendation scores of the candidate objects that are underestimated by the recommendation model.

[0051] For example, the recommendation score of the underestimated candidate object can be adjusted based on the principle that the recommendation score is negatively correlated with the adjustment amplitude, the recommendation score of the underestimated candidate object can be adjusted by multiplying a preset adjustment coefficient, or the recommendation score of the underestimated candidate object can be adjusted by adding a preset value.

[0052] In some optional implementations of this embodiment, the execution subject can perform the step 202 in the following manner:

[0053] For each candidate object in the candidate object set, in response to determining that the recommendation score of the candidate object is not greater than a first threshold value, and the number of recommendations of the candidate object in the historical recommendation information corresponding to the recommendation information is not greater than a second threshold value, the recommendation score of the candidate object is adjusted based on a preset manner to obtain a corrected score of the candidate object. The number of recommendations and the adjustment amplitude of the corrected score compared with the recommendation score are negatively correlated.

[0054] The first threshold value and the second threshold value are used to control the adjustment of the orientation of the candidate object with the recommended score, and can be set according to actual conditions, which are not limited herein. In practice, the determination of the recommended times can be limited by a preset time length, and the recommended times of the candidate object in a historical time period represented by the preset time length are determined as the recommended times.

[0055] In some optional implementations of the embodiment, the execution subject can perform the correction process of the recommended score by the following manner: first, obtain an adjustment coefficient based on the recommended times of the candidate object and a preset parameter; and then, obtain a corrected score of the candidate object according to the adjustment coefficient and the recommended score of the candidate object.

[0056] As an example, the preset parameter includes a first parameter a and a second parameter β. The first parameter a and the second parameter β are both greater than zero, and are used to control the correction strength of the recommended score, and the specific values thereof can be set according to actual conditions. The adjustment coefficient can be calculated by the following formula:

[0057]

[0058] wherein, c i represents the recommended times of the candidate object.

[0059] Further, the recommended score of the candidate object is multiplied by the adjustment coefficient to obtain the corrected score of the candidate object.

[0060] The correction strategy of the recommended score adjusts the recommended score according to the recommended times of the candidate object. The adjustment object is the candidate object with a lower recommended score and fewer recommended times. The adjustment manner is to increase the scores of these candidate objects, and the smaller the recommended times, the greater the increase amplitude, so as to support the potential underestimated candidate objects and give them a certain recommendation opportunity.

[0061] In the training-estimation iteration process of the pre-trained recommendation model, an exploration-exploitation dilemma will be encountered, that is, in the iteration process, is it to fully utilize the estimation results (recommended score) of the existing recommendation model, or to explore more information to better update the recommendation model.

[0062] Since the above correction strategy directly intervenes between the recommended score and the generation of the recommended information in the subsequent step 203, it is almost applicable to the exploration strategies of all existing deep learning models. Through the above universal and interpretable correction strategy, the exploration efficiency of the recommendation model can be improved, and the reward of the recommendation model represented by the feedback information involved in the cumulative step 204 in the iteration process is maximized.

[0063] Step 203, generating recommended information according to the final score of each candidate object in the candidate object set.

[0064] In this embodiment, the execution subject can generate the recommendation information according to the final score of each candidate object in the candidate object set.

[0065] For each candidate object in the candidate object set, when the recommendation score of the candidate object is not adjusted, the final score of the candidate object is the recommendation score; and when the recommendation score of the candidate object is adjusted, the final score of the candidate object is the corrected score.

[0066] In this embodiment, the execution subject can determine the candidate object with a higher final score as the to-be-recommended object, or directly determine the candidate object with the highest final score as the to-be-recommended object, to generate the recommendation information.

[0067] Through the adjustment of the recommendation score of the candidate object that is likely to be underestimated in step 202, the final score of the underestimated candidate object can become a candidate object with a higher score or the highest score, thereby having a chance to become a to-be-recommended object.

[0068] In step 204, the recommendation model is updated according to the feedback information for the recommendation information.

[0069] In this embodiment, the execution subject can update the recommendation model according to the feedback information for the recommendation information.

[0070] The feedback information may, for example, be operation information of a user for a candidate object represented by the recommendation information, such as a click, a browse, a view, a purchase, or the like, which is a positive reward for the recommendation model, or operation information of ignoring or the like, which is a negative reward. It can be understood that updating the recommendation model based on positive incentive information can make the updated recommendation model focus on the candidate object represented by the recommendation information, which may be a candidate object underestimated by the recommendation model after adjusting the recommendation score by the correction strategy. Thus, the updated recommendation model can solve the problem of underestimation of long-tail goods.

[0071] As an example, the execution subject can update the recommendation model according to the feedback information based on a preset quantity threshold, when feedback information exceeding the preset quantity threshold is collected.

[0072] As another example, the execution subject can collect feedback information in a preset time period based on a preset time period, and update the recommendation model according to the feedback information.

[0073] The updating process of the recommendation model can refer to the training process of the recommendation model, and use a machine learning algorithm to update the recommendation model, with attribute information of an object represented by the recommendation information as input and feedback information as expected output.

[0074] In the application of the recommendation model, the aforementioned execution entity can perform an iterative process of training and prediction of the recommendation model based on the update process shown in steps 201-204 above. When updating the model, the updated recommendation model obtained in the previous iteration is used as the pre-trained recommendation model for the current update process.

[0075] In some optional implementations of this embodiment, after obtaining the updated recommendation model, the aforementioned execution entity can perform subsequent recommendation tasks using the updated recommendation model. These recommendation tasks can include recommending items such as products, advertisements, videos, and books to target users. The updated recommendation model can address the underestimation of long-tail objects.

[0076] In some optional implementations of this embodiment, the recommendation score of each candidate object in the candidate object set represents the click-through rate of the user for that candidate object when it is recommended.

[0077] In this implementation, the higher the click-through rate of the candidate object predicted by the recommendation model, the greater the likelihood that the candidate object will be recommended to the target user.

[0078] See also Figure 3 , Figure 3 This is a schematic diagram 300 illustrating an application scenario of the recommendation model update method according to this embodiment. Figure 3 In the application scenario, target user 301, while browsing an e-commerce platform through terminal device 302, initiates a search request to server 303. Server 303 is deployed with a pre-trained recommendation model 304. Upon receiving the search request, server 303 first determines the recommendation score for each candidate object in the candidate object set 305 based on the pre-trained recommendation model 304. Then, it adjusts the recommendation scores of candidate objects in the candidate object set that might be underestimated by the recommendation model, obtaining a corrected score. Next, based on the final score of each candidate object in the candidate object set, it generates recommendation information and feeds it back to terminal device 302. For each candidate object in the candidate object set, when its recommendation score is not adjusted, its final score is the recommended score; when its recommendation score is adjusted, its final score is the corrected score. Finally, based on the feedback from target user 301 regarding the recommendation information, the recommendation model is updated.

[0079] The method provided by the above embodiments of the present application determines the recommendation score of each candidate object in the candidate object set based on the pre-trained recommendation model, adjusts the recommendation score of the candidate object that is likely to be underestimated by the recommendation model in the candidate object set to obtain a corrected score, generates recommendation information according to the final score of each candidate object in the candidate object set, and updates the recommendation model according to the feedback information for the recommendation information, so that the updated recommendation model can solve the underestimation problem for long-tail objects and improve the exploration efficiency of the recommendation model in the face of exploitation-exploration dilemma.

[0080] In some optional implementations of the present embodiment, the above execution subject can model the uncertainty of the initial score of each candidate object obtained by the recommendation model to determine the recommendation score of each candidate object. Specifically, the above execution subject can perform the above step 201 in the following manner:

[0081] For each candidate object in the candidate object set, the following operations are performed:

[0082] First, the representation vector and the initial score of the candidate object are obtained by the recommendation model.

[0083] The deep learning model can be decomposed into a representation layer and an affine layer. When the model is estimated for a sample, the attribute information of the candidate object is first processed by the representation layer to obtain the representation vector (i.e., deep representation) of the candidate object; then, the affine layer is used to perform affine transformation on the representation vector to obtain the final model output, i.e., the initial score of the candidate object.

[0084] Second, the length of the representation vector is used to represent the uncertainty of the initial score to determine the recommendation score of the candidate object.

[0085] In the present embodiment, after obtaining the representation vector of each candidate object, the length of each representation vector is calculated. As an example, for each candidate object in the candidate object set, the execution subject can calculate the L2 norm of the representation vector of the candidate object as the length of the representation vector of the candidate object. It can be understood that the lengths of the representation vectors of different candidate objects are generally different.

[0086] The initial score obtained by the recommendation model has uncertainty. For each candidate object in the candidate object set, the greater the length of the representation vector of the candidate object, the greater the uncertainty of the initial score of the candidate object; the smaller the length of the representation vector of the candidate object, the smaller the uncertainty of the initial score of the candidate object.

[0087] The principle of representing the uncertainty of the initial score of the candidate object by the representation vector of the candidate object is that, mathematically, the representation vector is the gradient of the model output with respect to the affine layer parameters. The greater the gradient (the greater the L2 norm), the more sensitive the model's estimate on the candidate object, and a small perturbation of the model parameters will cause a dramatic change in the estimate.

[0088] For each candidate object in the candidate object set, the above execution subject can process the initial score of the candidate object according to the length of the representation vector of the candidate object, to obtain a recommended score of the candidate object. As an example, when the length of the representation vector of the candidate object is small, that is, the uncertainty of the initial score of the candidate object is small, the initial score of the candidate object is directly determined as the recommended score; when the length of the representation vector of the candidate object is large, that is, the uncertainty of the initial score of the candidate object is large, a value is randomly selected in a preset numerical range in which the initial score of the candidate object is located as the recommended score of the candidate object.

[0089] In some optional implementations of the embodiment, the above execution subject can perform the above second step by the following way:

[0090] First, the normal distribution information corresponding to the candidate object is constructed with the initial score as the mean value and the length of the representation vector as the standard deviation; and then, the recommended score of the candidate object is sampled from the normal distribution information.

[0091] As an example, the above execution subject can randomly sample from the normal distribution information to obtain the recommended score of the candidate object.

[0092] The existing uncertainty modeling methods of deep learning models all need to make a large modification to the structure, parameters, training process or estimation process of the deep learning model, and it is difficult to reuse the existing model. For example, the Bayesian neural network needs to add a corresponding variance parameter for each weight parameter of the model; the Monte Carlo Dropout needs to add a Dropout layer. Since a large number of deep learning models have been deployed in real application scenarios, if the existing models cannot be reused, it will greatly affect the application range of the uncertainty modeling method of the deep learning model.

[0093] Compared with the uncertainty modeling methods such as Bayesian neural network and Monte Carlo Dropout, the above uncertainty modeling method based on deep representation does not change the parameters, structure and training method of the existing deep learning model at all, and only adds a sampling process for the representation vector in the estimation process, thereby improving the universality of the uncertainty modeling method.

[0094] With reference to the foregoing description, the uncertainty modeling method based on deep representation can be applied to the following application scenarios. Figure 4, shows a schematic flow 400 of one embodiment of the updating method of the recommendation model according to the present application, comprising the following steps:

[0095] Step 401, for each candidate object in the candidate object set, the following operations are performed:

[0096] Step 4011, obtaining, by the recommendation model, the representation vector and the initial score of the candidate object.

[0097] Step 4012, constructing normal distribution information corresponding to the candidate object with the initial score as the mean value and the length of the representation vector as the standard deviation.

[0098] Step 4013, sampling the recommendation score of the candidate object from the normal distribution information.

[0099] Step 4014, in response to determining that the recommendation score of the candidate object is not greater than the first threshold value, and the number of recommendations of the candidate object in the historical recommendation information corresponding to the recommendation information is not greater than the second threshold value, obtaining an adjustment coefficient based on the number of recommendations of the candidate object and a preset parameter.

[0100] Step 4015, obtaining the corrected score of the candidate object according to the adjustment coefficient and the recommendation score of the candidate object.

[0101] Step 402, generating the recommendation information according to the final score of each candidate object in the candidate object set.

[0102] For each candidate object in the candidate object set, when the recommendation score of the candidate object is not adjusted, the final score of the candidate object is the recommendation score; when the recommendation score of the candidate object is adjusted, the final score of the candidate object is the corrected score.

[0103] Step 403, updating the recommendation model according to the feedback information for the recommendation information.

[0104] As can be seen from the present embodiment, compared with Figure 2 Compared with the corresponding embodiment, the flow 400 of the updating method of the recommendation model in the present embodiment specifically illustrates the uncertainty modeling process of the initial score, the correction process of the recommendation score, improves the universality of the uncertainty modeling, solves the underestimation problem for long-tail objects, and improves the exploration efficiency of the recommendation model when facing the exploitation-exploration dilemma.

[0105] Continuing to refer to Figure 5 , shows a schematic flow 500 of one embodiment of the uncertainty modeling method according to the present application, comprising the following steps:

[0106] Step 501, obtaining, by the recommendation model, the representation vector and the initial score of the candidate object.

[0107] In this embodiment, the subject performing the uncertainty modeling method (e.g., the server in Figure 1 The recommendation model can obtain the representation vector and the initial score of the candidate object by the recommendation model. The candidate object can be each candidate object in a preset candidate object set.

[0108] The deep learning model corresponding to the recommendation model can be decomposed into a representation layer and an affine layer. When the model is estimated on the sample, the attribute information of the candidate object is first processed by the representation layer to obtain the representation vector (i.e., deep representation) of the candidate object; then, the affine layer is used to perform affine transformation on the representation vector to obtain the final model output, i.e., the initial score of the candidate object.

[0109] In step 502, the length of the representation vector is used to represent the uncertainty of the initial score, and the initial score of the candidate object is modeled with uncertainty.

[0110] In this embodiment, the subject performing the uncertainty modeling method (e.g., the server in

[0111] The initial score obtained by the recommendation model has uncertainty. When the length of the representation vector of the candidate object is greater, it indicates that the uncertainty of the initial score of the candidate object is greater; when the length of the representation vector of the candidate object is smaller, it indicates that the uncertainty of the initial score of the candidate object is smaller.

[0112] The principle of using the representation vector of the candidate object to represent the uncertainty of the initial score of the candidate object is that, mathematically, the representation vector is the gradient of the model output with respect to the parameters of the affine layer. The greater the gradient (the greater the L2 norm), the more sensitive the model is to the candidate object, and a small perturbation of the model parameters will cause a dramatic change in the estimated value.

[0113] The subject performing the uncertainty modeling method can process the initial score of the candidate object according to the length of the representation vector of the candidate object to obtain the recommendation score of the candidate object. As an example, when the length of the representation vector of the candidate object is small, i.e., the uncertainty of the initial score of the candidate object is small, the initial score of the candidate object is directly determined as the recommendation score; when the length of the representation vector of the candidate object is large, i.e., the uncertainty of the initial score of the candidate object is large, a value within a preset numerical range of the initial score of the candidate object is randomly selected as the recommendation score of the candidate object.

[0114] In some optional implementations of this embodiment, the subject performing the uncertainty modeling method can perform the above step 502 in the following manner:

[0115] First, the initial score is taken as the mean value, and the length of the representation vector is taken as the standard deviation to construct the normal distribution information corresponding to the candidate object, and the initial score of the candidate object is modeled for uncertainty.

[0116] The uncertainty modeling method does not change the parameters, structure and training method of the existing deep learning model at all, and only adds a sampling process for the representation vector in the prediction process, improving the universality of the uncertainty modeling method.

[0117] The method provided by the above embodiments of the present application obtains the representation vector and the initial score of the candidate object through the recommendation model, takes the length of the representation vector as the uncertainty of the initial score, models the initial score of the candidate object for uncertainty, and improves the universality of the uncertainty modeling.

[0118] With reference to Figure 6 , another embodiment of the uncertainty modeling method according to the present application is shown, and the schematic flow 600 includes the following steps:

[0119] In step 601, the representation vector and the initial score of the candidate object are obtained through the recommendation model.

[0120] In step 602, the initial score is taken as the mean value, and the length of the representation vector is taken as the standard deviation to construct the normal distribution information corresponding to the candidate object, and the initial score of the candidate object is modeled for uncertainty.

[0121] In some optional implementations of the present embodiment, the above execution subject can also perform the following steps to complete the iteration process of the training and prediction of the recommendation model. Including:

[0122] In step 603, the recommendation score of the candidate object is sampled from the normal distribution information.

[0123] In step 604, the recommendation information is generated according to the recommendation score of each candidate object in the candidate object set.

[0124] In the present embodiment, the above execution subject can obtain the recommendation score of each candidate object in the candidate object set in the manner shown in steps 601-603.

[0125] In step 605, the recommendation model is updated according to the feedback information for the recommendation information.

[0126] Steps 604-605 can be performed in the manner of steps 203-204, which will not be described here.

[0127] With reference to Figure 7As an implementation of the method shown in the above figures, the application provides an embodiment of an updating device of a recommendation model, which corresponds to the method embodiment shown in Figure 2 The device can be applied in various electronic devices.

[0128] As shown in Figure 7 The updating device of the recommendation model comprises: a determination unit 701 configured to determine a recommendation score of each candidate object in a candidate object set based on a pre-trained recommendation model; a correction unit 702 configured to adjust the recommendation score of a candidate object in the candidate object set that is likely to be underestimated by the recommendation model to obtain a corrected score; a generation unit 703 configured to generate recommendation information according to a final score of each candidate object in the candidate object set; and an updating unit 704 configured to update the recommendation model according to feedback information for the recommendation information.

[0129] In some embodiments, the determination unit 701 is further configured to, for each candidate object in the candidate object set: obtain, by the recommendation model, a representation vector and an initial score of the candidate object; and determine the recommendation score of the candidate object by taking the length of the representation vector as a representation of the uncertainty of the initial score.

[0130] In some embodiments, the determination unit 701 is further configured to: construct normal distribution information corresponding to the candidate object by taking the initial score as a mean value and the length of the representation vector as a standard deviation; and sample the recommendation score of the candidate object from the normal distribution information.

[0131] In some embodiments, the correction unit 702 is further configured to, for each candidate object in the candidate object set: in response to determining that the recommendation score of the candidate object is not greater than a first threshold value and that the number of recommendations of the candidate object in historical recommendation information corresponding to the recommendation information is not greater than a second threshold value, adjust the recommendation score of the candidate object based on a preset manner to obtain the corrected score of the candidate object, wherein the number of recommendations and the adjustment range of the corrected score compared to the recommendation score of the candidate object are negatively correlated.

[0132] In some embodiments, the correction unit 702 is further configured to: obtain an adjustment coefficient based on the number of recommendations of the candidate object and a preset parameter; and obtain the corrected score of the candidate object according to the adjustment coefficient and the recommendation score of the candidate object.

[0133] In some embodiments, the above device further comprises an execution unit (not shown in the figure) configured to perform a subsequent recommendation task by using the updated recommendation model.

[0134] In some embodiments, the recommendation score of each candidate object in the candidate object set represents a click rate of the user on the candidate object in a case that the candidate object is recommended.

[0135] In this embodiment, the determination unit in the updating apparatus of the recommendation model determines the recommendation score of each candidate object in the candidate object set based on the pre-trained recommendation model; the correction unit adjusts the recommendation score of the candidate object that is likely to be underestimated by the recommendation model in the candidate object set to obtain a corrected score; the generation unit generates the recommendation information according to the final score of each candidate object in the candidate object set; and the updating unit updates the recommendation model according to the feedback information on the recommendation information, so that the updated recommendation model can solve the underestimation problem for long-tail objects and improve the exploration efficiency of the recommendation model in the face of exploitation-exploration dilemma.

[0136] Reference will be made to the following Figure 8 which shows a structure schematic diagram of a computer system 800 of the device (for example Figure 1 shown in the figures 101, 102, 103, 105. Figure 8 The device shown is only an example and should not bring any limitation to the function and use range of the embodiments of the present application.

[0137] As shown in the figures Figure 8 , the computer system 800 includes a processor (for example, CPU, central processing unit) 801, which can perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) 802 or the programs loaded from the storage portion 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The processor 801, the ROM 802 and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0138] The following components are connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 810 as necessary, so that a computer program read therefrom is installed in the storage portion 808 as necessary.

[0139] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product which includes a computer program tangibly embodied on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 809 and / or installed from the removable media 811. When the computer program is executed by the processor 801, the above-described functions defined in the methods of the present application are performed.

[0140] Note that the computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal that propagates in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination thereof.

[0141] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0142] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0143] The units described in the embodiments of the present application can be implemented by software, or by hardware. The units described can also be implemented by a processor, for example, can be described as: a processor, comprising a determining unit, a correcting unit, a generating unit and an updating unit. In some cases, the names of the units do not constitute a limitation on the units themselves, for example, the correcting unit can also be described as: a unit for adjusting the recommendation score of the candidate object that can be underestimated by the recommended model in the candidate object set, to obtain a corrected score.

[0144] As another aspect, the application further provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, which, when executed by the device, cause the computer device to: determine a recommendation score of each candidate object in a candidate object set based on a pre-trained recommendation model; adjust the recommendation score of a candidate object that can be underestimated by the recommendation model in the candidate object set to obtain a corrected score; generate recommendation information according to the final score of each candidate object in the candidate object set; and update the recommendation model according to feedback information for the recommendation information.

[0145] The above description is merely the preferred embodiments of the present application and the description of the technical principles of the application. It should be understood by those skilled in the art that the inventive scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the inventive concept. For example, the above features can be replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form the technical solutions.

Claims

1. A method for updating a recommendation model, comprising: determining, based on a pre-trained recommendation model, a recommendation score of each candidate object in a candidate object set, the recommendation model being configured to determine a recommendation score of each object in an object set according to request information of a target user; for each candidate object in the candidate object set, in response to determining that the recommendation score of the candidate object is not greater than a first threshold value and that a recommendation frequency of the candidate object in historical recommendation information corresponding to the recommendation information is not greater than a second threshold value, obtaining an adjustment coefficient based on the recommendation frequency of the candidate object and a preset parameter, and obtaining a corrected score of the candidate object according to the adjustment coefficient and the recommendation score of the candidate object, wherein the recommendation frequency and the adjustment range of the corrected score compared with the recommendation score are negatively correlated; generating recommendation information according to the final score of each candidate object in the candidate object set; and updating the recommendation model according to feedback information corresponding to the recommendation information.

2. The method of claim 1, wherein, The determination of the recommendation score of each candidate object in the candidate object set based on the pre-trained recommendation model comprises: for each candidate object in the candidate object set, performing the following operations: obtaining a representation vector and an initial score of the candidate object by using the recommendation model; representing the uncertainty of the initial score by the length of the representation vector to determine the recommendation score of the candidate object.

3. The method of claim 2, wherein, The determination of the recommendation score of the candidate object by representing the uncertainty of the initial score by the length of the representation vector comprises: constructing normal distribution information corresponding to the candidate object by taking the initial score as a mean value and the length of the representation vector as a standard deviation; and sampling the recommendation score of the candidate object from the normal distribution information.

4. The method of any one of claims 1-3, wherein, The method further comprises: performing a subsequent recommendation task by using the updated recommendation model.

5. The method of any one of claims 1-3, wherein, The recommendation score of each candidate object in the candidate object set represents a click rate of a user with respect to the candidate object in a recommended case.

6. An apparatus for updating a recommendation model, comprising: a determination unit configured to determine, based on a pre-trained recommendation model, a recommendation score of each candidate object in a candidate object set, the recommendation model being configured to determine a recommendation score of each object in an object set according to request information of a target user; a correction unit configured to, for each candidate object in the candidate object set, in response to determining that the recommendation score of the candidate object is not greater than a first threshold value and that a recommendation frequency of the candidate object in historical recommendation information corresponding to the recommendation information is not greater than a second threshold value, obtain an adjustment coefficient based on the recommendation frequency of the candidate object and a preset parameter, and obtain a corrected score of the candidate object according to the adjustment coefficient and the recommendation score of the candidate object, wherein the recommendation frequency and the adjustment range of the corrected score compared with the recommendation score are negatively correlated; a generation unit configured to generate recommendation information according to the final score of each candidate object in the candidate object set; and an updating unit configured to update the recommendation model according to feedback information corresponding to the recommendation information.

7. The apparatus of claim 6, wherein, The determination unit is further configured to: For each candidate object in the candidate object set, the following operations are performed: obtaining, by the recommendation model, a representation vector and an initial score of the candidate object; representing uncertainty of the initial score by a length of the representation vector, and determining a recommendation score of the candidate object.

8. The apparatus of claim 7, wherein, The determining unit is further configured to: construct, with the initial score as a mean value and the length of the representation vector as a standard deviation, normal distribution information corresponding to the candidate object; and sample the recommendation score of the candidate object from the normal distribution information.

9. The apparatus of any one of claims 6-8, wherein, Further comprising: a performing unit configured to perform a subsequent recommendation task by the updated recommendation model.

10. The apparatus of any one of claims 6-8, wherein, The recommendation score of each candidate object in the candidate object set represents a click rate of a user with respect to the candidate object in a case where the candidate object is recommended.

11. A computer readable medium having stored thereon a computer program, wherein, The program, when executed by a processor, implements the method of any one of claims 1-5. 12.An electronic device, comprising: one or more processors; a memory device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-5.

Images