Recommendation model training method and device, recommendation method and device and electronic equipment

Abstract

The invention provides a recommendation model training method and device, a recommendation method and device and electronic equipment. The recommendation model training method comprises the steps that parameters in a recommendation model are initialized, the recommendation model structurally comprises a probability space output unit and an expert network, and the expert network is used for predicting at least one task; a sample used for training a recommendation model is obtained, target object features in the sample are input into a probability space output unit to obtain a task probability space, and the sample is correspondingly marked with a label corresponding to at least one task; inputting the target object features and the object features in the sample into an expert network to obtain a prediction score corresponding to the at least one task; and sampling at least one task based on the task probability space, and updating parameters in the recommendation model based on the predicted score corresponding to the sampled task and the tag corresponding to the sampled task.

Description

Technical Field

[0001] The present application relates to the technical field of recommendation systems, and in particular to a recommendation model training method, a recommendation method, a device, an electronic device and a storage medium. Background Art

[0002] The recommendation system is built on the basis of massive data mining, and efficiently provides targeted decision support and information services for the target objects to improve the target object experience and commercial benefits. Multi-task optimization (for example, clicks, likes, playback time, etc.) is the core of the recommendation algorithm optimization, which directly affects the content distribution efficiency of the recommendation algorithm and the retention rate of the target object. In a related technology, the recommendation algorithm consists of two parts: 1. Training a multi-task model to learn the estimated scores of each task on the object to be recommended; 2. Selecting a target fusion function, using the target fusion function to fuse the estimated scores of each task to obtain a fusion score, and filtering and sorting the objects to be recommended according to the fusion score. The above-mentioned related technology does not take into account the personalized consumption needs of the target object at the task level, and the target fusion function is usually selected based on experience. The effect of calculating the fusion score of the task through the target fusion function is poor. Therefore, the recommendation method in the above-mentioned related technology cannot effectively recommend the most suitable object to be recommended to the target object based on the characteristics of the target object. Summary of the invention

[0003] The embodiments of the present application provide a recommendation model training method, a recommendation method, an apparatus, an electronic device and a storage medium to solve the problems existing in the related art.

[0004] In a first aspect, an embodiment of the present application provides a recommendation model training method, comprising: initializing parameters in the recommendation model, the structure of the recommendation model comprising a probability space output unit and an expert network, the expert network being used to predict at least one task; obtaining samples for training the recommendation model, inputting target object features in the samples into the probability space output unit to obtain a task probability space, the samples being marked with labels corresponding to the at least one task; inputting the target object features and item features in the samples into the expert network to obtain a prediction score corresponding to the at least one task; sampling the at least one task based on the task probability space, and updating the parameters in the recommendation model based on the prediction score corresponding to the sampled task and the label corresponding to the sampled task.

[0005] In a second aspect, an embodiment of the present application provides a recommendation method, comprising: outputting evaluation scores corresponding to objects to be sorted based on a recommendation model trained by the recommendation model training method described in the first aspect; and sorting and displaying the objects to be sorted according to the evaluation scores corresponding to the objects to be sorted.

[0006] In a third aspect, an embodiment of the present application provides a recommendation model training device, comprising: an initialization module, configured to initialize the parameters in the recommendation model, the structure of the recommendation model comprising a probability space output unit and an expert network, the expert network being used to predict at least one task; a task probability space acquisition module, configured to obtain samples for training the recommendation model, input the target object features in the samples into the probability space output unit to obtain a task probability space, the samples are marked with a label corresponding to the at least one task; a prediction score acquisition module, configured to input the target object features and the item features in the samples into the expert network to obtain a prediction score corresponding to the at least one task; a parameter updating module, configured to sample the at least one task based on the task probability space, and update the parameters in the recommendation model based on the prediction score corresponding to the sampled task and the label corresponding to the sampled task.

[0007] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0008] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0009] Compared with the prior art, this application has the following advantages:

[0010] According to the embodiment of the present application, the recommendation model outputs a corresponding task probability space for the target object based on the target object characteristics in the sample to characterize the personalized consumption needs of the target object. Using the task probability space to sample tasks and updating the parameters in the recommendation model based on the predicted scores and labels of the sampled tasks can enable the recommendation model to output higher predicted scores for tasks preferred by the target object, thereby effectively recommending more suitable recommended objects to the target object.

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0013] Figure 1 is a schematic diagram showing the basic structure of a recommendation model according to an embodiment of the present application;

[0014] Figure 2 is a flow chart showing a recommendation model training method according to an embodiment of the present application;

[0015] Figure 3 is a structural block diagram showing a recommendation model training device according to an embodiment of the present application; and

[0016] Figure 4 is a block diagram showing an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0018] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0019] Figure 1 Schematic diagram showing the basic structure of a recommendation model according to an embodiment of the present application. Figure 1 As shown, the recommendation model includes an embedding layer, a fusion function unit, a probability space output unit, an expert network, and a target fusion function. The embedding layer can convert the sparse high-dimensional features of the sample into a dense low-dimensional feature vector, and the features of the sample can include target object features and item features. The fusion function unit can output the weight parameters and bias parameters required to construct the target fusion function based on the low-dimensional feature vector corresponding to the target object features. The expert network is used to predict at least one task (for example, clicks, likes, viewing time, etc.), and the expert network can output the prediction scores corresponding to each task based on the low-dimensional feature vector corresponding to the target object features and the low-dimensional feature vector corresponding to the item features. The probability space output unit can output the probability space of the task based on the low-dimensional feature vector corresponding to the target object features, and the probability space reflects the preference of the target object for the task. The target fusion function generates a fusion score based on the prediction scores corresponding to each task.

[0020] An embodiment of the present application provides a recommendation model training method 200, which is applied to an electronic device, which may be an electronic device such as a server, a computer, a mobile phone, or a smart watch. Figure 2 The recommendation model training method 200 of the present application is described. Figure 2 is a flow chart showing a method for training a recommendation model according to an embodiment of the present application. Figure 2 As shown, the recommendation model training method 200 may include the following steps S201 to S204. Steps S201 to S204 will be described in detail below in conjunction with a specific embodiment.

[0021] First, proceed to step S201. In step S201, initialize the parameters in the recommendation model, the structure of the recommendation model includes a probability space output unit and an expert network, and the expert network is used to predict at least one task.

[0022] The training process of the recommendation model is to update the parameters in the recommendation model, and to make the recommendation model achieve better performance by continuously iterating and updating the parameters. Before updating the parameters of the recommendation model, the parameters of the recommendation model need to be initialized, for example, random initialization or Xavier initialization can be performed. This application does not limit the initialization method.

[0023] Next, step S202 is entered. In step S202, samples for training the recommendation model are obtained, and the target object features in the samples are input into the probability space output unit to obtain a task probability space, wherein the samples are marked with labels corresponding to the at least one task.

[0024] The target object represents the object to which the item is recommended, and there can be multiple objects. The samples used to train the recommendation model can be prepared in advance and stored in the database. The samples can include target object features and item features. The items can be commodities on e-commerce platforms, videos on video platforms, etc. The samples can also include labels corresponding to the tasks that the recommendation model needs to predict. Tasks can be clicks, likes, playback time, forwarding, etc. For example, if a target object clicks on an item but does not like it, in this sample, the label for clicks is 1 and the label for likes is 0.

[0025] In one embodiment, the structure of the recommendation model also includes an embedding layer, and the target object features in the sample are input into the probability space output unit to obtain the task probability space, including: converting the target object features into a low-dimensional feature vector through the embedding layer; and inputting the low-dimensional feature vector corresponding to the target object features into the probability space output unit to obtain the task probability space.

[0026] The target object features and item features in this application are discrete data, and discrete features are usually represented by onehot encoding. Taking the occupation of the target object in the target object feature as an example, if there are 20 occupations of the target object using an application, the one hot encoding of the target object occupation is a 20-dimensional vector. The one hot encoding of a target object whose occupation is a teacher is [1,0,0,0,…,0], which is a high-dimensional sparse vector with too high computational complexity. Taking the item ID in the item feature as an example, if there are 100 items in total, the one hot encoding of the item ID is a 100-dimensional vector, and the onehot encoding dimension will increase with the increase in the number of goods, which may cause a dimensional explosion. Therefore, the high-dimensional sparse vector formed by one hot encoding is not suitable for direct input into subsequent deep neural networks for training.

[0027] The target object features in the sample are input into the embedding layer for processing, and the original high-dimensional sparse vector corresponding to the target object features can be processed by low-dimensional embedding mapping, thereby converting the high-dimensional sparse vector into a low-dimensional feature vector. The distance between the low-dimensional feature vectors can represent the similarity of the target objects. The smaller the distance between the low-dimensional feature vectors, the more similar the eigenvalues ​​corresponding to the low-dimensional feature vectors are. Through the processing of the embedding layer, it is helpful to capture the similarity relationship between the target object features and reduce the calculation amount of the recommendation model.

[0028] After obtaining the low-dimensional feature vectors corresponding to the target object features, these low-dimensional feature vectors can be input into the probability space output unit, which can output the probability space corresponding to the task that the recommendation model needs to predict. The probability space output unit can be a deep neural network. Since the deep neural network can fit any function, the function of the probability space output unit is to fit a function that can output the task probability space. The input of the probability space output unit is the low-dimensional feature vector corresponding to the target object features, and the output is the probability space corresponding to the task. The task can be clicks, likes, viewing time, forwarding, etc. Take the tasks of click, like, and watch time as an example. For these three tasks, based on the input target object features (for example, occupation is teacher, interest is watching movies, and education is undergraduate), the probability space output by the probability space output unit can be, for example, {click: 0.1, like: 0.8, watch time: 0.1}. In this probability space, the probability of like is much greater than the probability of click and watch time. The target object (that is, the target object with undergraduate education, occupation is teacher, and interest is watching movies) prefers to like, and can be considered as a target object driven by likes, that is, the target object tends to watch content related to the content that has been liked before. The probability space output unit outputs the task probability space for the target object according to the target object features of different target objects, realizes the binding of target objects and tasks, and the recommended content can fully consider the personalized needs of the target object.

[0029] In one embodiment, the target object feature includes at least one of the following features: target object attribute feature, target object behavior sequence feature, context feature, and the item feature includes at least one of the following features: item attribute feature, item statistical feature, and item pre-trained feature.

[0030] The target object attribute features may be the target object's occupation, interests, education, etc. The target object behavior sequence features may be the target object's click sequence, like sequence, forwarding sequence, favorite sequence, etc. The context features may be the terminal brand information used by the target object, the operating system used by the target object, the time when the target object requests the recommended scene, the geographical location of the target object, etc. Items may be commodities on e-commerce platforms, videos on video platforms, notes and tweets on social networks, etc. Item attribute features may be the category and ID of items, etc. Item statistics include the click rate, like rate, average viewing time, etc. of items after being exposed within a period of time. The pre-trained features of items may be, for example, low-dimensional feature vectors obtained by pre-training the title of the item. The pre-trained model may use, for example, the BERT model.

[0031] Next, the process proceeds to step S203. In step S203, the target object features and the item features in the sample are input into the expert network to obtain a prediction score corresponding to the at least one task.

[0032] In one embodiment, the target object features and the item features in the sample are input into the expert network to obtain a prediction score corresponding to the at least one task, including: converting the item features into a low-dimensional feature vector through the embedding layer, concatenating the low-dimensional feature vector corresponding to the target object features and the low-dimensional feature vector corresponding to the item features; and inputting the concatenated low-dimensional feature vector into the expert network to obtain a prediction score corresponding to the at least one task.

[0033] After the item features are converted into low-dimensional feature vectors through the embedding layer, the low-dimensional feature vectors corresponding to the item features can be concatenated with the low-dimensional feature vectors corresponding to the target object features. For example, the low-dimensional vector corresponding to the item features is a 512*10 two-dimensional vector, and the low-dimensional vector corresponding to the target object features is a 512*20 two-dimensional vector. The two two-dimensional vectors are concatenated to obtain a 512*30 two-dimensional vector.

[0034] The structure of the expert network can adopt the structure of MMoE (Multi-gate Mixture-of-Experts) or the structure of PLE (Progressive Layered Extraction), and this application does not impose any restrictions on this. Taking the tasks of clicks, likes, and viewing time as an example, the number of expert networks can be 4, of which 3 expert networks correspond to the 3 tasks to be predicted, and the other expert network corresponds to the features that can be shared between the 3 tasks. The expert network can be a deep neural network used to fit a function that can output the prediction scores corresponding to each task. The expert network can output the prediction scores of each task based on the low-dimensional vectors corresponding to the features of the target object and the low-dimensional vectors corresponding to the features of the item. That is, the expert network can output the prediction scores of an item on each task for a specific target object. For example, the expert network can output the prediction scores of a short video in terms of clicks, likes, and viewing time for a target object with a bachelor's degree, a profession of teacher, and an interest in watching movies.

[0035] In one embodiment, the structure of the recommendation model also includes a fusion function unit, and method 200 also includes: inputting the low-dimensional feature vector corresponding to the target object feature into the fusion function unit to construct a target fusion function; inputting the prediction score corresponding to the at least one task into the target fusion function to obtain a fusion score; calculating a first loss function based on the fusion score and the label corresponding to the at least one task, and updating the parameters in the fusion function unit through the first loss function.

[0036] The fusion function unit constructs the target fusion function based on the low-dimensional feature vector corresponding to the target object features with the idea of ​​metalearning. Metalearning is also called "learning to learn", which means using past knowledge and experience to guide the learning of new tasks so that the neural network has the ability to learn to learn. Usually, the model's parameter initialization is obtained by manual setting or random initialization. These two types of parameter initialization methods cannot effectively obtain a better model initialization point. Metalearning models parameter initialization in the form of learning, allowing the model to learn to perform effective parameter initialization for different tasks.

[0037] In one embodiment, the inputting of the low-dimensional vector corresponding to the target object feature into the fusion function unit to construct a target fusion function includes: inputting the low-dimensional vector corresponding to the target object feature into the fusion function unit to obtain weight parameters and bias parameters; and constructing the target fusion function through the weight parameters and the bias parameters.

[0038] The fusion function unit can be a deep neural network, and the fusion function unit can output weight parameters and bias parameters based on the low-dimensional feature vector corresponding to the target object feature. Taking the function y=wx+b as an example, w is the weight parameter and b is the bias parameter. The weight parameter refers to the parameter connecting different neurons, and the size of the weight parameter represents the strength of the signal transmitted between different neurons. In the present application, the weight parameter can be the weight when the prediction points of each task are fused. The bias parameter is an offset used to adjust the output result. After obtaining the weight parameter and the bias parameter, the weight parameter and the bias parameter can be used to construct the target fusion function, which can also be a deep neural network. For example, the weight parameter of the target fusion function is a 3*1 matrix, then the input of the target fusion function is 3 parameters and the output is 1 parameter. Based on the idea of ​​meta-learning, the fusion function unit obtains the weight parameters and bias parameters required to construct the target fusion function through the target object features, realizes the binding of the target object and the target fusion function, and enables the target fusion function to obtain a better initialization point.

[0039] After the target fusion function is constructed, the target fusion function can be used to calculate the fusion score of at least one task. Taking the tasks of click, like, and watch time as an example, the expert network outputs prediction scores of 0.3, 0.6, and 0.4 for these three tasks based on the target object features and item features. The prediction scores of these three tasks are input into the target fusion function to obtain a fusion score. The fusion score can be obtained by multiplying the prediction scores corresponding to each task by different weights and then adding them together. The fusion score reflects the degree of interest of a specific target object in an item in the case of multiple tasks. After obtaining the fusion scores of these three tasks, the first loss function can be calculated based on this fusion score and the labels corresponding to the three tasks. The first loss function can be calculated in the form of a cross entropy loss function or a square loss function. This application does not limit this. The loss function is used to measure the difference between the predicted value and the expected value in a deep neural network. The higher the output value (loss) of the loss function, the greater the difference. The training of a deep neural network can be understood as a process of minimizing the loss as much as possible. After obtaining the first loss function, the gradient descent method can be used to update the parameters in the fusion function unit so that the fusion function unit outputs better weight parameters and bias parameters, and thus a better target fusion function can be constructed.

[0040] According to the embodiment of the present application, the target object features are used as the input of the fusion function unit based on the idea of ​​meta-learning, and the weight parameters and bias parameters for the target object are output to construct a target fusion function, so that different target objects have different target fusion functions, that is, for different target objects, the objects to be recommended have different fusion scores, and the recommendation effect for different target objects is better.

[0041] Next, the process proceeds to step S204. In step S204, the at least one task is sampled based on the task probability space, and the parameters in the recommendation model are updated based on the prediction scores corresponding to the sampled tasks and the labels corresponding to the sampled tasks.

[0042] Taking the tasks of click, like, and watch time as an example, the probability space for these three tasks can be, for example, {click: 0.1, like: 0.8, watch time: 0.1}. Based on this probability space, these three tasks are sampled, and the frequency of each task in the sampling result is positively correlated with the probability value in the probability space, that is, there is a greater probability (i.e., 80%) to sample the like task. Sampling can be implemented, for example, using the tf.multinomial method in the tensorflow model library.

[0043] In one embodiment, updating the parameters in the recommendation model based on the predicted scores corresponding to the sampled tasks and the labels corresponding to the sampled tasks includes: calculating a second loss function based on the predicted scores corresponding to the sampled tasks and the labels corresponding to the sampled tasks, and updating the parameters in the expert network through the second loss function.

[0044] Taking the tasks of click, like, and watch time as an example, the probability space for these three tasks can be, for example, {click: 0.1, like: 0.8, watch time: 0.1}. Sampling is performed based on this probability space. If the like task is sampled, the second loss function can be calculated based on the prediction score corresponding to the like task and the label corresponding to the like task. The second loss function can be calculated using the cross entropy method or the square loss function method. This application does not limit this. After obtaining the second loss function, the gradient descent method can be used to update the parameters in the expert network so that the parameters in the expert network conform to the distribution of the task. The expert network can then output a higher prediction score for the tasks preferred by the target object.

[0045] In one embodiment, the method 200 further includes: setting a reward function for the task probability space, and updating parameters in the probability space output unit using a policy gradient according to a reward value output by the reward function.

[0046] After multiple rounds of updating of the parameters in the fusion function unit and the expert network, the parameters in the probability space output unit can be updated next. First, the evaluation index (for example, AUC) of each task can be calculated using the recommendation model with updated parameters, and a reward function can be set for the task probability space. The setting rule of the reward function is based on the task probability space output in this round. The values ​​obtained by calculating the AUC of each task using the recommendation model with updated parameters are not lower than and at least one is higher than the value obtained by calculating the AUC of each task using the baseline model. Only then can a positive reward be set. The baseline model is an easy-to-implement benchmark recommendation model used before the recommendation model training is performed in the embodiment of the present application. For example, for the three tasks of clicks, likes, and viewing time, the AUCs calculated using the baseline model are 0.5, 0.6, and 0.75, respectively. The task probability space output in this round is {click: 0.1, like: 0.8, viewing time: 0.1}. The AUCs calculated using the recommendation model with updated parameters are 0.5, 0.6, and 0.8, respectively. The AUC of viewing time has increased, and the AUCs of clicks and likes have not decreased. A positive reward can be set for the task probability space {click: 0.1, like: 0.8, viewing time: 0.1}, and the reward value can be +1. For another example, for the three tasks of click, like, and watch time, the AUCs calculated using the baseline model are 0.5, 0.6, and 0.75 respectively. The task probability space output in this round is {click: 0.4, like: 0.4, watch time: 0.2}. The AUCs calculated using the recommendation model with updated parameters are 0.5, 0.5, and 0.6 respectively. The AUCs of like and watch time are reduced. A negative reward can be set for the task probability space {click: 0.4, like: 0.4, watch time: 0.2}, and the reward value can be -1. Next, according to the reward value output by the reward function, the policy gradient can be used to update the parameters in the probability space output unit. The idea of ​​the policy gradient is that if a task probability space gets a lot of reward values, its probability of occurrence is increased, and if a task probability space gets a little reward value, its probability of occurrence is reduced.

[0047] When all the training data (ie, samples) have been read, the training of the recommendation model can be terminated.

[0048] According to the embodiment of the present application, the idea of ​​reinforcement learning is adopted to bind the reward value output by the reward function with the measurement of each task, so that the probability space output unit can output a task probability space that is more in line with the characteristics of the target object, thereby realizing the binding of the target object and the task, and can recommend content that the target object is more interested in, thereby avoiding the averaging of the recommendation results.

[0049] An embodiment of the present application provides a recommendation method, which is applied to an electronic device, which may be an electronic device such as a server, a computer, a mobile phone, or a smart watch. The recommendation method comprises: outputting evaluation scores corresponding to objects to be sorted based on a recommendation model trained by the recommendation model training method; and sorting and displaying the objects to be sorted according to the evaluation scores corresponding to the objects to be sorted.

[0050] When the recommendation model training is completed, the evaluation score of the object to be sorted can be output based on the recommendation model. The input of the recommendation model can be the characteristics of the target object and the characteristics of the object to be sorted (i.e., item characteristics). Taking the short video scene as an example, the target object characteristics can be obtained when the target object requests a recommended scene through a terminal device (i.e., the target object uses a mobile terminal to access the recommendation interface of the short video application), and the object to be sorted can be a hot short video when the target object requests a recommended scene. The evaluation score can be the fusion score of the object to be sorted output by the target fusion function in the above recommendation model for a specific target object in a multi-task scenario. The evaluation score represents the degree of interest of the target object in the object.

[0051] After obtaining the evaluation scores of the objects to be sorted, the objects to be sorted can be sorted and displayed according to the evaluation scores corresponding to the objects to be sorted. For example, if there are 20 objects to be sorted and the homepage of the application can only display 5 objects to be sorted, then the 20 objects to be sorted are sorted in descending order according to the evaluation scores, and the top 5 objects to be sorted with the highest evaluation scores are selected and displayed on the homepage of the application.

[0052] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, an embodiment of the present application also provides a recommendation model training device, which is deployed in an electronic device, and the electronic device can be an electronic device such as a server, a computer, a mobile phone, or a smart watch. Figure 3 The recommendation model training device of the present application is described. Figure 3 1 is a structural block diagram showing a recommendation model training device according to an embodiment of the present application. Figure 3 As shown, the recommendation model training device 300 may include: an initialization module 301, a task probability space acquisition module 302, a prediction score acquisition module 303 and a parameter updating module 304.

[0053] The initialization module 301 is configured to initialize the parameters in the recommendation model, wherein the structure of the recommendation model includes a probability space output unit and an expert network, and the expert network is used to predict at least one task.

[0054] The initialization module 301 may be a component in the electronic device, and the electronic device initializes the parameters in the recommendation model through the component.

[0055] The task probability space acquisition module 302 is configured to obtain samples for training the recommendation model, input the target object features in the samples into the probability space output unit to obtain the task probability space, and the samples are marked with labels corresponding to the at least one task.

[0056] The task probability space acquisition module 302 may be a component in the electronic device, and the electronic device obtains the task probability space through the component.

[0057] The prediction score obtaining module 303 is configured to input the target object features and the item features in the sample into the expert network to obtain the prediction score corresponding to the at least one task.

[0058] The prediction score obtaining module 303 may be a component in the electronic device, and the electronic device obtains the prediction score corresponding to at least one task through the component.

[0059] The parameter updating module 304 is configured to sample the at least one task based on the task probability space, and update the parameters in the recommendation model based on the prediction score corresponding to the sampled task and the label corresponding to the sampled task.

[0060] The parameter updating module 304 may be a component in the electronic device, and the electronic device updates the parameters in the recommendation model through the component.

[0061] In one embodiment, the structure of the recommendation model also includes an embedding layer, and the task probability space acquisition module 302 is further configured to convert the target object features into a low-dimensional feature vector through the embedding layer; and input the low-dimensional feature vector corresponding to the target object features into the probability space output unit to obtain the task probability space.

[0062] In one embodiment, the prediction score acquisition module 303 is further configured to convert the item features into a low-dimensional feature vector through the embedding layer, and concatenate the low-dimensional feature vector corresponding to the target object features and the low-dimensional feature vector corresponding to the item features; input the concatenated low-dimensional feature vector into the expert network to obtain the prediction score corresponding to the at least one task.

[0063] In one embodiment, the structure of the recommendation model also includes a fusion function unit, and the recommendation model training device 300 also includes a fusion function unit training module, which is configured to input the low-dimensional feature vector corresponding to the target object feature into the fusion function unit to construct a target fusion function; input the prediction score corresponding to the at least one task into the target fusion function to obtain a fusion score; calculate a first loss function based on the fusion score and the label corresponding to the at least one task, and update the parameters in the fusion function unit through the first loss function.

[0064] In one embodiment, the parameter updating module 304 is further configured to calculate a second loss function based on the predicted scores corresponding to the sampled tasks and the labels corresponding to the sampled tasks, and update the parameters in the expert network by using the second loss function.

[0065] In one embodiment, the recommendation model training device 300 also includes a probability space output unit training module, which is configured to set a reward function for the task probability space and use a policy gradient to update the parameters in the probability space output unit according to the reward value output by the reward function.

[0066] In one embodiment, the fusion function unit training module also includes a target fusion function construction submodule, which is configured to input the low-dimensional feature vector corresponding to the target object feature into the fusion function unit to obtain weight parameters and bias parameters; and construct the target fusion function through the weight parameters and the bias parameters.

[0067] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0068] According to the embodiment of the present application, the recommendation model outputs the corresponding task probability space for the target object based on the target object characteristics in the sample to characterize the personalized consumption needs of the target object. Using the task probability space to sample tasks and updating the parameters in the recommendation model based on the predicted scores and labels of the sampled tasks can enable the recommendation model to output higher predicted scores for the tasks preferred by the target object, thereby effectively recommending more suitable recommended objects to the target object.

[0069] According to the embodiment of the present application, the target object features are used as the input of the fusion function unit based on the idea of ​​meta-learning, and the weight parameters and bias parameters for the target object are output to construct a target fusion function, so that different target objects have different target fusion functions, and the recommendation effect for different target objects is better.

[0070] According to the embodiment of the present application, the idea of ​​reinforcement learning is adopted to bind the reward value output by the reward function with the measurement of each task, so that the probability space output unit can output a task probability space that is more in line with the characteristics of the target object, thereby realizing the binding of the target object and the task, and can recommend content that the target object is more interested in, thereby avoiding the averaging of the recommendation results.

[0071] Figure 4 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 4As shown, the electronic device includes: a memory 401 and a processor 402. The memory 401 stores a computer program that can be run on the processor 402. When the processor 402 executes the computer program, the method in the above embodiment is implemented. The number of the memory 401 and the processor 402 can be one or more.

[0072] The electronic device also includes:

[0073] The communication interface 403 is used to communicate with external devices and perform data exchange transmission.

[0074] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the memory 401, the processor 402 and the communication interface 403 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0075] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0076] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0077] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0078] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0079] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0080] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0081] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0082] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0083] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0084] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0085] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0086] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0087] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0088] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A recommendation model training method, comprising: Initializing parameters in the recommendation model, wherein the structure of the recommendation model includes a probability space output unit and an expert network, and the expert network is used to predict at least one task; Acquire samples for training the recommendation model, input the target object features in the samples into the probability space output unit to obtain a task probability space, wherein the samples are marked with labels corresponding to the at least one task; Inputting the target object features and the item features in the sample into the expert network to obtain a prediction score corresponding to the at least one task; The at least one task is sampled based on the task probability space, and parameters in the recommendation model are updated based on the prediction scores corresponding to the sampled tasks and the labels corresponding to the sampled tasks.

2. The method according to claim 1, wherein: The structure of the recommendation model further includes an embedding layer, and the step of inputting the target object features in the sample into the probability space output unit to obtain the task probability space includes: Converting the target object features into a low-dimensional feature vector through the embedding layer; The low-dimensional feature vector corresponding to the target object feature is input into the probability space output unit to obtain the task probability space.

3. The method according to claim 2, wherein: The step of inputting the target object features and the item features in the sample into the expert network to obtain a prediction score corresponding to the at least one task includes: The item features are converted into low-dimensional feature vectors through the embedding layer, and the low-dimensional feature vectors corresponding to the target object features and the low-dimensional feature vectors corresponding to the item features are concatenated; The concatenated low-dimensional feature vector is input into the expert network to obtain a prediction score corresponding to the at least one task.

4. The method according to claim 3, wherein: The structure of the recommendation model further includes a fusion function unit, and the method further includes: Inputting the low-dimensional feature vector corresponding to the target object feature into the fusion function unit to construct a target fusion function; Inputting the prediction score corresponding to the at least one task into the target fusion function to obtain a fusion score; A first loss function is calculated based on the fusion score and the label corresponding to the at least one task, and the parameters in the fusion function unit are updated by the first loss function.

5. According to the method of claim 2 or 3, the updating of the parameters in the recommendation model based on the prediction scores corresponding to the sampled tasks and the labels corresponding to the sampled tasks comprises: A second loss function is calculated based on the predicted score corresponding to the sampled task and the label corresponding to the sampled task, and the parameters in the expert network are updated by the second loss function.

6. The method according to claim 4, further comprising: A reward function is set for the task probability space, and a policy gradient is used to update parameters in an output unit of the probability space according to a reward value output by the reward function.

7. The method according to claim 4, wherein: The step of inputting the low-dimensional feature vector corresponding to the target object feature into the fusion function unit to construct a target fusion function comprises: Inputting the low-dimensional feature vector corresponding to the target object feature into the fusion function unit to obtain weight parameters and bias parameters; The target fusion function is constructed by using the weight parameters and the bias parameters.

8. The method according to claim 2 or 3, wherein: The target object feature includes at least one of the following features: target object attribute feature, target object behavior sequence feature, and context feature; the item feature includes at least one of the following features: item attribute feature, item statistical feature, and item pre-training feature.

9. A recommendation method comprising: Based on the recommendation model trained by the recommendation model training method according to any one of claims 1 to 8, outputting the evaluation score corresponding to the object to be sorted; The objects to be sorted are sorted and displayed according to the evaluation scores corresponding to the objects to be sorted.

10. A recommendation model training device, comprising: An initialization module configured to initialize parameters in the recommendation model, wherein the structure of the recommendation model includes a probability space output unit and an expert network, wherein the expert network is used to predict at least one task; A task probability space acquisition module is configured to obtain samples for training the recommendation model, input the target object features in the samples into the probability space output unit to obtain a task probability space, wherein the samples are marked with a label corresponding to the at least one task; A prediction score obtaining module, configured to input the target object features and the item features in the sample into the expert network to obtain a prediction score corresponding to the at least one task; A parameter updating module is configured to sample the at least one task based on the task probability space, and update the parameters in the recommendation model based on the prediction score corresponding to the sampled task and the label corresponding to the sampled task.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images