Training method and device of recommendation system

Abstract

According to the training method of the recommendation system provided by the embodiment of the invention, in the recommendation system comprising the sorting model and the recommendation model, the output of the sorting model can accurately reflect the preference of the user, and accurate preference degree sorting is carried out on each sample object. And then sorting information output by the sorting model is used as a supervision signal to train the recommendation model, preference alignment is realized between the sorting model and the recommendation model, and popularity deviation in the recommendation model is reduced and eliminated. In this way, the recommendation performance of the recommendation system is improved, and more high-quality and more accurate personalized recommendation content can be provided for the user.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of large language model training, and in particular, to a training method and device for a recommendation system. Background Art

[0002] With the rapid development of Internet technology, recommendation systems have been widely used in various related scenarios, such as e-commerce shopping, audio and video playback, news information, etc. Generally speaking, the essence of the recommendation system is to find and match the content of interest to users from the massive Internet content (also referred to as objects in the embodiments of this specification) when the user's needs are unclear, and then improve the user experience by providing users with accurate push content.

[0003] However, the recommendation system still faces many challenges in its application, one of which is popularity bias. Popularity bias refers to the fact that popular (e.g., widely popular, highly clicked, etc.) content is recommended more frequently than its own popularity. This is because the training data is often constructed based on the historical interaction between users and content. A small number of highly popular content accounts for most of the user interactions in the training data, making it easier for the recommendation system to give a higher prediction score for highly popular content (head data) after training based on this biased data, while it usually only gives a lower prediction score for unpopular content (tail data). The overall goal of the recommendation system is to provide users with personalized suggestions, rather than simply recommending popular Internet content. Popularity bias makes the recommendation system unable to accurately reflect the true preferences of users.

[0004] Therefore, popularity bias will cause the recommendation system to be more inclined to recommend Internet content that has been widely accepted, which will not only affect the accuracy of the recommendation system, but also affect the user experience, causing users to miss some content that they may be interested in or has a smaller audience. In the long run, it will cause the problem of a single Internet scene ecology, over-strengthening the existing definition of popularity and failing to provide diverse choices.

[0005] Therefore, we hope to have a solution that can reduce the popularity bias in the recommendation system through technical means, so that it can effectively predict the next Internet content that the user is interested in, thereby providing users with richer and more diverse choices. Summary of the invention

[0006] One or more embodiments of this specification describe a method and device for training a recommendation system. In a recommendation system that includes a ranking model and a recommendation model, the output of the ranking model can accurately reflect the user's preferences and accurately rank the preferences of each sample object. The ranking information output by the ranking model is then used as a supervisory signal to train the recommendation model, achieve preference alignment between the ranking model and the recommendation model, and reduce or eliminate popularity deviations in the recommendation model. In this way, the recommendation performance of the recommendation system is improved, and users can be provided with more high-quality, accurate, and personalized recommendation content.

[0007] According to a first aspect, a training method for a recommendation system is provided, the recommendation system comprising a ranking model and a recommendation model, the method comprising:

[0008] A user description of a target user and an object description of each object in a sampled object set are obtained, wherein the user description is determined according to a sequence of objects with which the target user has interacted, and the object set includes a first object with which the target user has interacted, and several negative objects with which the target user has not interacted.

[0009] The user description and the object description of each object are respectively input into a pre-trained ranking model and a recommendation model to be trained to obtain a first score determined by the ranking model reflecting the target user's preference for each object, and a second score determined by the recommendation model reflecting the recommendation degree of each object to the target user, wherein the data set used for pre-training includes objects sampled with equal probability from a sample object sequence.

[0010] According to a first score of the first object and first scores of a plurality of negative objects, ranking information is determined; and according to the ranking information and second scores of each object, a distillation loss is determined.

[0011] The recommendation model is updated according to at least the distillation loss.

[0012] In one implementation, the object is an Internet product, and the interaction includes one or more of the following: clicking, browsing, collecting, and purchasing.

[0013] In one implementation, the ranking model and the recommendation model are both constructed based on a pre-trained text encoder.

[0014] In one implementation, the objects in the object sequence are arranged in order of interaction time.

[0015] In one implementation, the object set is obtained by the following steps:

[0016] Based on a first sampling probability, a first object is sampled from the object sequence, wherein the first sampling probability is positively correlated with the popularity of each object in the object sequence. Based on a random sampling probability, a number of negative objects are sampled.

[0017] In one scenario of the above implementation, the user description is obtained by following the steps below:

[0018] Based on a pre-built description template and description texts of each object in the object sequence, a description text of the target user is generated as the user description, wherein the description template includes text for prompting to recommend the target object to the user.

[0019] In one scenario of the above implementation, the object description is obtained by following the steps below:

[0020] The text corpus contained in any object is concatenated to obtain an object description corresponding to the object, wherein the text corpus includes at least one of the following: a text attribute name, a text value, and an object ID.

[0021] In one implementation, determining the distillation loss includes:

[0022] For any target negative object, if the ranking information indicates that its first score is less than the first score of the first object, the sample loss is determined as a first difference between a second score negatively related to the first object and a second score of the target negative object.

[0023] If the ranking information indicates that the first score of the target negative object is greater than the first score of the first object, the sample loss is determined as a second difference between a second score negatively related to the target negative object and a second score of the first object.

[0024] The distillation loss is obtained by averaging the sample losses of each negative object.

[0025] In one implementation, determining the distillation loss includes:

[0026] According to the ranking information, an object with the highest first score is determined as a calculation object.

[0027] The distillation loss is determined, the distillation loss being negatively correlated to a second fraction of the computational object.

[0028] In one implementation, determining the ranking information includes: determining a first probability distribution reflecting the target user's preference ranking of each object according to the first score of each object in the object set. Determining the distillation loss includes:

[0029] A second probability distribution reflecting the ranking of the recommendation degree of each object with respect to the target user is determined according to the second score of each object in the object set.

[0030] The distillation loss is calculated, where the distillation loss is negatively correlated to a similarity between the first probability distribution and the second probability distribution.

[0031] In one implementation, updating the recommendation model at least according to the distillation loss further includes:

[0032] A sampling loss is determined, the sampling loss being negatively correlated to a first ratio, the first ratio being a calculated value of the second score of the first object relative to a sum of calculated values ​​of the second scores of the objects in the set of objects, and the recommendation model is updated according to a weighted sum of the sampling loss and the distillation loss.

[0033] In one implementation, the ranking model is pre-trained by the following steps:

[0034] The user description of the sample user and the object description of each sample object in the sample object set are input into the ranking model to obtain a third score for each sample object; wherein the sample object set includes positive sample objects that the sample user has interacted with, and a number of negative sample objects that the sample user has not interacted with.

[0035] A pre-training loss is determined, which is negatively correlated to a second ratio, wherein the second ratio is a calculated value of the third score of the positive sample object relative to a sum of calculated values ​​of the third scores of each sample object in the sample object set.

[0036] The ranking model is updated with the goal of minimizing the pre-training loss.

[0037] In one scenario of the above implementation, the sorting model includes a text encoder and a multi-layer perceptron; the pre-training includes a first stage and a second stage. Updating the sorting model includes:

[0038] In the first stage, the text encoder and the multi-layer perceptron are updated. In the second stage, only the multi-layer perceptron is updated.

[0039] In one scenario of the above implementation, the first sample object set used in the first stage is obtained by the following steps:

[0040] Based on the second sampling probability, a first positive sample object is sampled from the first historical object sequence that the first sample user has interacted with, and the second sampling probability is positively correlated with the popularity of each object in the first historical object sequence. Based on the random sampling probability, a number of negative sample objects are sampled.

[0041] In one scenario of the above implementation, the second sample object set used in the second stage is obtained by the following steps:

[0042] Based on the third sampling probability, a second positive sample object is sampled from the second historical object sequence that the second sample user has interacted with, and the third sampling probability is equal probability. Based on the random sampling probability, a number of negative sample objects are sampled.

[0043] According to a second aspect, a training device for a recommendation system is provided, the recommendation system comprising a ranking model and a recommendation model, the device comprising:

[0044] Acquisition module: configured to acquire a user description of a target user and an object description of each object in a sampled object set, wherein the user description is determined based on a sequence of objects with which the target user has interacted, and the object set includes a first object with which the target user has interacted, and a number of negative objects with which the target user has not interacted.

[0045] Evaluation module: configured to input the user description and the object description of each object into a pre-trained ranking model and a recommendation model to be trained respectively, to obtain a first score determined by the ranking model reflecting the target user's preference for each object, and a second score determined by the recommendation model reflecting the recommendation degree of each object to the target user, wherein the data set used for pre-training includes objects sampled with equal probability from a sample object sequence.

[0046] The loss determination module is configured to determine the ranking information according to the first score of the first object and the first scores of several negative objects; and determine the distillation loss according to the ranking information and the second scores of each object.

[0047] Model updating module: configured to update the recommendation model at least according to the distillation loss.

[0048] According to a third aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0049] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method described in the first aspect is implemented.

[0050] In the method and device provided in the embodiments of this specification, a ranking model and a recommendation model are designed, wherein the ranking model is used to distinguish the preference of different objects relative to the selected object. The ranking model does not predict the next object, but gives the relative ranking between different objects, especially objects that the user has not interacted with. The ranking model samples in the object set based on equal probability, so that each object can be selected as the selected object with equal probability, and pre-training is performed accordingly, so that the pre-trained ranking model can correctly identify the relative preference between different objects. The recommendation model is used to predict the next object that the user may be interested in. It is based on the relative ranking of candidate objects given by the ranking model as an additional supervisory signal, and the model training is performed. At this time, the popularity deviation in the relative ranking has been corrected, and the recommendation model can be trained to correctly reflect the user's preferences, so that the recommendation system can accurately recommend target objects to users on a large number of Internet content objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 An exemplary sequence recommendation diagram provided for the embodiments of this specification;

[0053] Figure 2A A schematic diagram of a recommendation model training scenario disclosed in an embodiment of this specification;

[0054] Figure 2B A schematic diagram of a recommendation model training scenario disclosed in an embodiment of this specification;

[0055] Figure 2C A schematic diagram of a training framework provided in an embodiment of this specification;

[0056] Figure 3 A flow chart of a training method for a recommendation system provided according to an embodiment of this specification;

[0057] Figure 4 A flow chart of a sorting model pre-training method provided according to an embodiment of this specification;

[0058] Figure 5 A schematic diagram of a sorting model pre-training framework provided according to an embodiment of this specification;

[0059] Figure 6 A schematic diagram of a training device for a recommendation system provided according to an embodiment of the present specification. DETAILED DESCRIPTION

[0060] The solution provided by the embodiments of this specification is described below in conjunction with the accompanying drawings.

[0061] As mentioned above, recommendation systems are widely used in various scenarios and are used to recommend the next object of interest (Internet content) to users based on model prediction.

[0062] In a typical application scenario, taking sequence recommendation as an example, the various objects that the target user has interacted with historically can form an object sequence. Figure 1 , showing an exemplary sequence recommendation, the target user will have a variety of interactions with numerous Internet contents during the process of accessing the Internet. In one example, the application scenario may be online shopping, the object is an Internet commodity, and the interaction may be click, browse, collect, purchase, and the like. In another example, the application scenario may be audio and video playback, the object is an Internet audiovisual work, and the interaction may be playback, collection, comment, and the like. The above examples do not represent a limitation on the application scenarios of the recommendation system. In other examples, the objects and the interactions with the objects have different meanings in different application scenarios, and the embodiments of this specification do not give examples one by one.

[0063] Back to Figure 1 , the object sequence includes objects A, B, C, D, and E, which are arranged in sequence. The target user interacts with these objects in sequence on the Internet platform in the application scenario. It can be seen that if a boundary is drawn between object D and object E (as shown by the dotted line in the figure), the object before the boundary is regarded as the historical interaction object of the target user, and the object after the boundary is regarded as the recommended target object. In the next visit of the target user to the Internet platform (or in the process of continuing to browse), if object E is recommended to him, the user's interaction can be obtained. In other words, by analyzing the object sequence, Internet content related to the historical interaction object (i.e., the recommended target object) can be recommended to the target user. This target object will be more likely to obtain positive feedback behaviors of the target user, such as clicks, favorites, and other behaviors that have a positive effect on the conversion rate.

[0064] Through the above Figure 1The description of the examples can help us understand the sequence recommendation scenario used by the recommendation system, that is, recommending objects that may be of interest to the target user in the future based on the object sequence composed of the target user's historical interaction behaviors. In some examples, the objects in the object sequence can be arranged in chronological order according to the time sequence in which the target user interacts with them. In other examples, the objects in the object sequence can also be arranged according to the behavior category in which the target user interacts with them. In other examples, objects can also be arranged in the object sequence based on other dimensions according to business characteristics, and this specification does not specifically limit this.

[0065] In practical applications, recommendation systems rely on the target user's interaction history object dataset for training, and these data often have certain tendencies. In other words, these datasets are long-tail datasets, and the content in them shows obvious uneven distribution. Popular content dominates the dataset due to its high exposure and extensive user interaction, while a large amount of unpopular content is located in the long tail part, with relatively low frequency of occurrence and user interaction. Biased datasets will affect the training effect of the recommendation system and produce popularity bias.

[0066] In some cases where popularity bias occurs, since a few popular objects account for most of the user interactions, training a recommendation system on such a long-tail dataset often learns the popularity information of the objects, resulting in the recommendation of popular objects being ranked high and being recommended to the target users first, while unpopular objects do not receive fair exposure. However, unpopular objects do not necessarily mean that they will not receive positive feedback from the target users, and vice versa. Popular items do not necessarily mean that they will be favored by the target users. This leads to errors in the recommendation system.

[0067] The inventors have found that when training a recommendation system on such a long-tail data set, the sampling method will affect the popularity deviation learned by the recommendation system. In the embodiments of this specification, by introducing sample objects based on equal probability sampling, the recommendation system is trained so that the recommendation system can pay attention to some unpopular objects, so that when recommending to target users, it can be inclined to unpopular objects to a certain extent, thereby avoiding the popularity deviation described in this example.

[0068] In other examples of popularity bias, the target user's interests usually change over time. For example, during a certain period of time, the target user is more interested in digital products, so in the object sequence constructed based on user interaction, a large number of interactions are concentrated on digital product category objects, and other category objects with which the target user has not interacted are not included. As a result, the recommendation system is trained to be more inclined to recommend digital products or similar objects to the target user; and the user's interests are usually variable. The target user may be interested in sports equipment when visiting the Internet platform next time. However, since the recommendation system has been trained to have a tendency, it may cause the recommendation system to fail to recommend sports equipment category objects to the target user and fail to obtain positive feedback from the target user.

[0069] The inventors have found that when training a recommendation system on such a data set, it is necessary to consider objects that have not interacted with the target user (i.e., negative sample objects). In some embodiments of the present specification, the negative sample objects are relatively ranked as additional supervisory signals for training the recommendation system, so that the recommendation system can learn the target user's preference for negative sample objects during the training process, so that in the process of making recommendations, the target user's next object of interest can be predicted in a diversified manner, providing the target user with richer and more diverse choices.

[0070] One or more embodiments of the present specification propose corresponding solutions to the popularity deviation generated by the recommendation system in practical applications.

[0071] Figure 2A A recommendation model training scenario in the related art is shown, which is a recommendation model training method commonly used in the prior art. As shown in the accompanying figure, in this scenario, the recommendation model is trained based on the interaction object sequence formed by the user's real interaction history. Specifically, the recommendation model determines the one with the highest user preference from several candidate objects (including negative objects that the user has not interacted with) based on the n objects in the interaction object sequence as the recommended object; the training goal is to make the recommended object close to or equal to the n+1th object (label object) that the user actually interacts with. However, as mentioned above, the interaction object sequences of a large number of users constitute a long-tail data set, in which the interaction objects, even the candidate objects that have not been interacted with, may be selected from popular objects. Training a recommendation system based on such a long-tail data set often introduces popularity bias.

[0072] In addition, the inventors further discovered that the preference comparison determined in the above scenario is sparse and binary, which only includes the binary relationship between the interacting positive sample objects and each negative sample object (i.e., the comparison of the size of the preference), and it is difficult to judge the relative preference between the negative sample objects. Based on this binary relationship, the training of the recommendation model is supervised and iteratively updated, which can gradually enhance the distinction between positive sample objects and negative sample objects. As the training progresses, the recommendation model will be more inclined to predict recommended objects in positive samples, and the attention to negative sample objects may gradually decrease, resulting in a single ecology of the recommendation system and a lack of diversity.

[0073] Under the influence of the above two factors, Figure 2A The recommendation model shown will be trained to tend to predict recommended objects among popular objects in positive samples, while paying little attention to unpopular objects and negative objects, resulting in recommendation bias, that is, popularity bias.

[0074] In order to solve the popularity deviation caused by the recommendation model training process, Figure 2B In the training scenario of a recommendation model (debiasing) provided by the embodiment of this specification shown, a ranking model is introduced. The model can be based on the records of the target user's past interaction objects in the object sequence, as well as each negative sample object. In addition to giving the comparison of the preference of the target object for each negative sample object, it also gives the relative preference between the negative sample objects. In addition, during the pre-training process, the ranking model uses an equal sampling probability to sample label objects from the objects that the sample user has interacted with in the past (as label data, also referred to as positive sample objects), and uses the label objects to train the ranking model so that it can learn the user's true preferences and output a fair preference ranking. The recommendation model can be updated based on the preference comparison and the relative preference of negative samples output by the pre-training ranking model, so that the recommendation model can not only pay attention to the target user's preference for the positive sample object, but also take into account the relative preference of each negative sample object in the candidate object when predicting the target user's next recommended target object, thereby reducing and removing the popularity bias in the recommendation model.

[0075] In some embodiments, the ranking model and recommendation model can be constructed based on a text encoder. Due to the universality and high expressiveness of text representation, a language model is used to encode sample objects, and text representation is used as a feature description of sample objects, which can fully utilize the semantics of sample objects. Typically, in some scenarios, the text encoder is a pre-trained text encoder, which can fully utilize the training accumulation of historical text corpus for the text encoder, obtain a more expressive encoding representation, and avoid training the text encoder from scratch.

[0076] Based on the above technical concept, the inventor proposes a training method and device for a recommendation system in the embodiments of this specification. The training framework of the recommendation system is as follows: Figure 2C As shown, the user description of the target user and the descriptions of each object in the object set including positive objects and several negative objects are input into the sorting model and the recommendation model respectively. In a specific example, the sorting model and the recommendation model are both constructed based on a text encoder, and the user description and each object description are text descriptions. After obtaining the user description and each object description, the sorting model and the recommendation model encode the user representation and each object representation respectively.

[0077] Based on the user representation and each object representation, the ranking model can output the relative ranking of positive objects and several negative objects in terms of preference scores (shown as preference ranking in the figure), and the recommendation model can output the recommendation degree of each positive object and several negative objects. Next, the distillation loss can be obtained based on the characteristics of each object that can be reflected in the preference ranking (for example, the target user's preference for any object, the distribution characteristics of the object's preference score, etc.), combined with the recommendation degree of each object output by the recommendation model; the sampling loss can also be determined based on the recommendation degree of each object output by the recommendation model. Based on the distillation loss and the sampling loss, the model is updated. Typically, in some implementations, the ranking model is pre-trained, and its model parameters can be frozen in this embodiment, and only the parameters of the recommendation model are updated.

[0078] The above is an overview of the technical ideas of the exemplary embodiments of this specification. Figure 3 In FIG. 1 , a flow chart of a training method for a recommendation system provided according to an embodiment of the present specification is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 3In this embodiment, the recommendation system includes a ranking model and a recommendation model, and the method includes at least the following steps: Step S301: Obtain a user description of a target user and an object description of each object in a sampled object set, wherein the user description is determined according to a sequence of objects with which the target user has interacted, and the object set includes a first object with which the target user has interacted, and several negative objects with which the target user has not interacted. Step S303: Input the user description and the object description of each object into a pre-trained ranking model and a recommendation model to be trained, respectively, to obtain a first score reflecting the target user's preference for each object determined by the ranking model, and a second score reflecting the recommendation degree of each object with respect to the target user determined by the recommendation model, wherein the data set used for the pre-training includes objects sampled with equal probability from a sequence of sample objects. Step S305: Determine ranking information according to the first score of the first object and the first scores of several negative objects; determine distillation loss according to the ranking information and the second scores of each object. Step S307: Update the recommendation model at least according to the distillation loss.

[0079] The specific implementation methods of the above steps will be described in detail below with reference to the accompanying drawings.

[0080] As mentioned above, in some implementations, the ranking model and recommendation model included in the recommendation system are both built based on a pre-trained text encoder to fully utilize the semantic features of sample objects for ranking and recommendation.

[0081] In step S301, a user description of a target user and an object description of each object in a sampled object set are obtained, wherein the user description is determined according to a sequence of objects with which the target user has interacted, and the object set includes a first object with which the target user has interacted, and several negative objects with which the target user has not interacted.

[0082] In this embodiment, given a group of users, labeled And several objects, marked as I. For any target user The objects that have interacted with it can form an object sequence, represented by s u =(i1,i2,…,i N ), where i t ∈I is the tth object in the sequence that interacts with the target user, and N is the maximum capacity of the object sequence. Therefore, the recommendation model is defined as a function f, which can be the object sequence s of the target user u The combination with any object i is assigned a recommendation score, denoted as f(i,s u ). In sequential recommendation, the main goal is to build a top-K ranked list based on the function f to predict the next object of interest to the target user.

[0083] In a specific scenario, each object i∈I has some text attributes, such as title, category, content, etc. In some examples, the title of each object can be used as the object description of each object. In other examples, the text corpus contained in any object can also be spliced ​​to obtain the object description corresponding to the object. The text corpus can be any text attribute information contained in the object, such as text attribute name, text value, object ID, etc. In a specific example, the object description of each object can be recorded in the form of a key-value pair, for example, as:

[0084]

[0085] Where M is the number of text attribute information in the object; k is the text attribute name, such as title, category, etc.; is the text value of the jth text attribute of the ith object, represented by any set of key-value pairs 〈k j :v j 〉For example, it can be 〈Title: Data Cable〉, or 〈Category: Electronic Accessories〉, and so on.

[0086] The object descriptions obtained in the above steps can be spliced ​​together to construct a comprehensive and personalized user description. For example, typical objects can be selected from the objects with which the target user has interacted historically according to certain rules, and the user description of the target user can be determined based on the object descriptions of these typical objects. It is also possible to splice the corresponding object descriptions of the various objects included in the object sequence of the target object and use them as the user description of the target user.

[0087] According to one implementation, a description text may be generated as the user description based on a pre-built description template and description texts of each object in the object sequence, wherein the description template includes text for prompting to recommend the target object to the user. That is, assuming that the target user's object sequence s u =(i1,i2,…,i N ), the object description of the corresponding object is spliced ​​based on the template, and its user description is expressed as:

[0088]

[0089] The prompt text included in Prompt1 indicates to the recommendation system that the user description is used as background knowledge. In a specific example, Prompt1 may be "The browsing history of the target user is". The prompt text included in Prompt2 is used to instruct the recommendation system to recommend the target object to the target user. In a specific example, Prompt2 may be "Please recommend an object that the target user is interested in".

[0090] It should be noted that the description templates above are only examples, and the specific template content can be adjusted according to needs and scenarios.

[0091] In this embodiment, an object set is also prepared, which includes the first object that the target user has interacted with, and several negative objects that the target user has not interacted with. In actual applications, a variety of sampling strategies can be used to sample the first object from the objects that the target user has interacted with, and sample several negative objects from a large number of objects that the target user has not interacted with.

[0092] According to one implementation, based on a first sampling probability, a first object is sampled from the object sequence, and the first sampling probability is positively correlated with the popularity of each object in the object sequence; based on a random sampling probability, a number of negative objects are sampled to form the object set. That is, in a long-tail data set, since the distribution of user interaction behaviors is highly skewed, most user interactions are concentrated on the head object, and when the first object is sampled with the first sampling probability, the probability of the head object being selected is much greater than the probability of the tail object being selected.

[0093] In the sampled object set, according to the object description generation method described above, respective object descriptions can be generated for respective objects in the object set.

[0094] Next, in step S303, the user description and the object description of each object are respectively input into a pre-trained sorting model and a recommendation model to be trained to obtain a first score determined by the sorting model reflecting the target user's preference for each object, and a second score determined by the recommendation model reflecting the recommendation degree of each object to the target user, wherein the data set used for pre-training includes objects sampled with equal probability from a sample object sequence.

[0095] As mentioned above, the pre-trained ranking model can accurately identify the relative preference rankings between negative objects that the target user has not interacted with, which is also closely related to the popularity correction of the recommendation system. In addition, in pre-training, in order to remove the popularity bias, the sample objects used in pre-training will be selected based on equal probability and sampled from the sample object sequence, where any object has an equal probability of being selected as a sample object.

[0096] The specific process of the pre-trained ranking model will be explained in detail below. Here we only briefly introduce its sampling strategy. It should be known that the pre-trained ranking model can accurately output the relative ranking of negative objects based on the real preferences of the target users.

[0097] The recommendation model can score the recommendation degree (second score) of each input object. In downstream tasks, based on the second score of each object, suitable objects can be selected as target objects for recommendation to target users.

[0098] Next, the pre-trained ranking model can fairly determine the relative preference for negative objects that the target user has not interacted with. This type of information can be used as supervisory information outside the target user's interactive behavior during the training of the recommendation model to enhance the training effect and avoid the problem of insufficient training of the recommendation model for low-popularity objects. Specifically, in step S305, the ranking information is determined based on the first score of the first object and the first scores of several negative objects; the distillation loss is determined based on the ranking information and the second scores of each object; in step S307, the recommendation model is updated at least based on the distillation loss.

[0099] In this step, the ranking between objects is first determined based on the first scores of each object (including the first object and several negative objects) output by the ranking model. This ranking information can reflect the target user's true preference for each object, and thus serve as the basis for distillation learning of the recommendation model.

[0100] According to one implementation, the distillation learning strategy adopts the "paired distillation" method, that is, a negative object and a first object are randomly selected to form a sample pair, and the recommendation model is trained to learn the relative preference between the two objects in this sample pair. That is, for any negative object, the sample category is determined by its ranking with the first object on the first score, so as to determine the distillation loss function. Specifically:

[0101] If the first score of the negative object is less than the first score of the first object, the negative object is marked as a relative negative sample object. The set of these relative negative sample objects is recorded as The sample loss of any relative negative sample object in the set is negatively correlated to the difference between the second score of the first object and the second score of the relative negative sample object. pair-nega It can be expressed as:

[0102]

[0103] in, and They are the second score of the first object g and the second score of the negative object m output by the recommendation model for the target user u. Minimizing the loss function value (i.e., sample loss) will make the recommendation degree of the first object g much greater than the recommendation degree of the relative negative sample object m, so that the output of the recommendation model can clearly reflect the difference between the first object and each relative negative sample object.

[0104] Similarly, if the first score of the negative object is greater than the first score of the first object, the negative object is marked as a relative positive sample object. The set of these relative positive sample objects is recorded as The sample loss of any relative positive sample object in the set is negatively correlated to the difference between the second score of the relative positive sample object and the second score of the first object. pair-pos It can be expressed as:

[0105]

[0106] in, and They are the second score of the first object g and the second score of the negative object m output by the recommendation model for the target user u. Minimizing the loss function value (i.e., sample loss) will make the recommendation degree of the relative positive sample object much greater than the recommendation degree of the first object, so that the output of the recommendation model can clearly reflect the difference between each relative positive sample object and the first object.

[0107] In a specific scenario of this implementation, the distillation loss l dis-pair is the average of the sample losses of each negative object in the sampled object set:

[0108] l dis-pair =(l pair-pos +l pair-nega ) / 2 (5)

[0109] According to another implementation, the distillation learning strategy adopts a "hard distillation" method, using the highest ranked object output by the ranking model to supervise the training of the recommendation model. That is, based on the ranking information, the object with the highest first score is determined as the calculation object for determining the distillation loss. Calculation object i h The determination can be expressed as the formula:

[0110]

[0111] in, is the first score of the first object output by the ranking model for user u, is a set of first scores of several negative objects in the object set output by the ranking model for user u, S - A collection of negative objects.

[0112] In a specific scenario, the distillation loss is negatively correlated to the second score of the computational object. dis-hard It can be expressed as:

[0113]

[0114] in, and They are the second score of the first object and the second score of the negative object output by the recommendation model for the target user u. Minimizing the loss function value (i.e., distillation loss) will make the recommendation degree of the calculated object much greater than that of other objects, so that the output of the recommendation model can clearly distinguish the object with the highest preference ranking from other objects.

[0115] According to another implementation method, the distillation learning strategy adopts a "soft distillation" approach, that is, during the recommendation model training process, the recommendation degree probability distribution of each object output by the recommendation model is continuously brought closer to the preference degree probability distribution of each object output by the ranking model.

[0116] Specifically, first, according to the first scores of each object in the object set, a first probability distribution P that reflects the preference ranking of each object by the target user is determined. t :

[0117]

[0118] Then, according to the second scores of each object in the object set, a second probability distribution P reflecting the recommendation ranking of each object with respect to the target user is determined. s :

[0119]

[0120] Finally, the distillation loss is determined based on the two probability distributions obtained. The distillation loss l dis-soft Negatively correlated with the first probability distribution P t With the second probability distribution P s In a specific scenario, KL divergence is used as the loss function of distillation loss, expressed as:

[0121]

[0122] Minimizing the loss function value (i.e., distillation loss) will make the probability distribution of recommendation degrees of each object output by the recommendation model closer to the probability distribution of preference degrees of each object output by the ranking model, so that the recommendation model can quickly and effectively learn the capabilities of the ranking model.

[0123] The process of training the recommendation model based on the distillation loss is described above. In other embodiments, the training of the recommendation model not only relies on the above-mentioned distillation loss, but also refers to the sampling loss of the recommendation model.

[0124] According to one implementation, the sampling loss is negatively correlated to a first ratio, where the first ratio is a calculated value of the second score of the first object relative to a sum of calculated values ​​of the second scores of the objects in the object set. The recommendation model is updated according to a weighted sum of the sampling loss and the distillation loss.

[0125] In a specific scenario, the cross entropy function is used as the loss function to calculate the sampling loss l rec , as follows:

[0126]

[0127] Among them, B represents the batch size of the recommendation model training, g j and u j denote the first object and target user in the jth training instance in a batch, respectively, and S - Represents several negative sample objects in the batch, represents the second score of the first object output by the recommendation model for the target user, Represents the second score of the negative object m output by the recommendation model for the target user.

[0128] During the training of the recommendation model, the referenced prediction loss l is the sampling loss l rec Distillation loss dis The weighted sum of is expressed as:

[0129] l=l rec +λ*l dis (12)

[0130] Among them, the distillation loss is l dis It can be any of the distillation loss calculation methods mentioned above. λ is a hyperparameter that controls the speed of preference alignment between the recommendation model and the sorting model, and can be defined according to the needs of specific application scenarios.

[0131] It should be understood that the above steps S301-S307 can be repeatedly executed, and through multiple rounds of updating the recommendation model, the recommendation model and the pre-trained ranking model will continue to converge, thereby reducing and removing the popularity deviation in the recommendation model and improving the prediction effect of the recommendation system.

[0132] In some embodiments of the present specification, a pre-training method for a sorting model is also provided. Figure 4 A process of a sorting model pre-training method provided according to an embodiment of the specification is shown.

[0133] In step S401, the user description of the sample user and the object description of each sample object in the sample object set are input into the ranking model to obtain a third score for each sample object; wherein the sample object set includes positive sample objects that the sample user has interacted with, and a number of negative sample objects that the sample user has not interacted with.

[0134] In order to reduce or eliminate popularity bias, the training data used in the pre-training process at least includes positive sample objects sampled by equal probability sampling.

[0135] The text description of object i and sample user u and By inputting the sorting model, we can get their respective representation vectors:

[0136]

[0137] Wherein, PLM(·) represents the text encoder for constructing the ranking model, which can be any text encoder that can encode natural language text and output a series of context representation vectors, such as BERT, OPT, T5, etc. The embodiment of this specification does not impose any restrictions on the selection of the text encoder. AGG(·) represents aggregating the representation vectors output by the text encoder into a single, compact vector, and the final output h i With h u Represent any object i and user u respectively.

[0138] Based on this, we can calculate the vector h i With h u The similarity between any object i and sample user u is obtained. That is, the third score representing the preference of sample user u for any object i.

[0139] In one implementation, the third score is determined by calculating the inner product of the vectors, which is expressed as follows:

[0140]

[0141] According to the above method, the third score of the sample user for each sample object in the sample object set can be calculated, including positive sample objects and negative sample objects.

[0142] Next, in step S403, a pre-training loss is determined, which is negatively correlated to a second ratio, where the second ratio is a calculated value of the third score of the positive sample object relative to a sum of calculated values ​​of the third scores of the respective sample objects in the sample object set.

[0143] The loss function used in this step represents the proportional relationship between the third score calculation value of the positive sample object and the sum of the third score calculation values ​​of all sample objects. The training goal is to maximize this proportional relationship (i.e., the function value that is negatively correlated to the loss function), increase the separation between the positive samples and other samples, so that the output of the sorting model can clearly distinguish the positive sample objects from other sample objects.

[0144] According to one implementation, an exponential operation (e.g., exp) is performed on the third score of each sample object. The exponential function is monotonically increasing, and as the variable value increases, the growth rate of the function value will also accelerate. In other words, if the differences between the outputs of the ranking model are small, taking the exponent can amplify these differences, making the model prediction more discriminative.

[0145] In a specific scenario, the cross entropy function is used as the loss function to calculate the pre-training loss l rec , as follows:

[0146]

[0147] Where B represents the batch size of pre-training, g j and u j Respectively represent the positive sample objects and sample users in the jth training instance in a batch, S - Represents several negative sample objects in the batch.

[0148] Finally, in step S405, the sorting model is updated with the goal of minimizing the pre-training loss.

[0149] In one embodiment, considering that the data set used for pre-training has a long-tail characteristic, in order to solve the problem of under-sampling in data sampling that leads to insufficient model training for certain specific objects, in some embodiments, the pre-training of the ranking model is divided into two stages. Figure 5 A schematic diagram of a pre-training framework for a sorting model provided according to an embodiment is shown, see the accompanying drawings:

[0150] In the first stage, positive sample objects are sampled according to the popularity of each object in the object sequence, aiming to capture the general pattern of sample user preferences through the object sequence and initialize the sorting model. Then, in the second stage, positive sample objects are sampled from the object sequence based on equal probability to fine-tune the sorting model so that the sorting model can give a certain degree of attention to the tail objects.

[0151] In addition, during the fine-tuning process in the second stage, a large number of parameters in the text encoder of the sorting model are easily overfitted to the tail object, which destroys the training results of the first stage. To solve this problem, an additional multi-layer perceptron (MLP) is added to the sorting model, which is decoupled from the text encoder, which means that they can be trained and optimized independently. Therefore, the representation vector output by the sorting model can be expressed as:

[0152]

[0153] The above two-stage pre-training framework is specifically expanded as follows:

[0154] In the first stage, in order to enable the ranking model to better perceive the user's preference through pre-training, when sampling positive sample objects, sampling is performed based on the popularity of each object in the object sequence, where the probability of any object being selected as a positive sample object is proportional to its popularity level. For negative sample objects, sampling is performed based on random sampling probability. After the object description of each sample object in the sample object set and the user description of the sample user are input into the ranking model, the encoding calculation obtains the preference score of each sample object, and then the pre-training loss is determined, and the ranking model is updated. In the first stage, the text encoder and multi-layer perceptron of the ranking model are updated.

[0155] In the second stage, in order to allow the ranking model to pay attention to the tail objects, when sampling positive sample objects, sampling is performed based on equal probability in the object sequence, where the probability of any object being selected as a positive sample object is equal. For negative sample objects, sampling is performed based on random sampling probability. Similar to the first stage, after the object description of each sample object in the sample object set and the user description of the sample user are input into the ranking model, the encoding calculation obtains the preference score of each sample object, and then determines the pre-training loss and updates the ranking model. In the second stage, only the multi-layer perceptron of the ranking model is updated.

[0156] The calculation of the preference score involved in the description of the pre-training framework and the determination method of the pre-training loss have been described in detail in the previous article and will not be repeated here. It should be understood that the above steps S401-S405 can be performed repeatedly, and the sorting model can be repeatedly updated through multiple rounds to continuously improve the accuracy of the sorting model in expressing user preferences.

[0157] In this specification, the word "first" in terms such as the first object and the first score, and the corresponding "second", "third" (if any), etc. in the text are merely for the convenience of distinction and description and do not have any limiting meaning.

[0158] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] Figure 6 Schematic diagram of a training device for a recommendation system according to an embodiment of this specification. The device 600 is deployed in a computing device, which can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. Figure 3 The recommendation system includes a ranking model and a recommendation model, and the device 600 includes:

[0160] Acquisition module 601: configured to acquire a user description of a target user and an object description of each object in a sampled object set, wherein the user description is determined based on a sequence of objects with which the target user has interacted, and the object set includes a first object with which the target user has interacted, and a number of negative objects with which the target user has not interacted.

[0161] Evaluation module 602: configured to input the user description and the object description of each object into a pre-trained ranking model and a recommendation model to be trained respectively, to obtain a first score determined by the ranking model reflecting the target user's preference for each object, and a second score determined by the recommendation model reflecting the recommendation degree of each object to the target user, wherein the data set used for pre-training includes objects sampled with equal probability from a sample object sequence.

[0162] The loss determination module 603 is configured to determine the ranking information according to the first score of the first object and the first scores of several negative objects; and determine the distillation loss according to the ranking information and the second scores of each object.

[0163] Model updating module 604: configured to update the recommendation model at least according to the distillation loss.

[0164] According to another embodiment of the present specification, there is also provided a computer program product, including a computer program / instruction, which implements the aforementioned combination when executed by a processor. Figure 3 The steps of the method.

[0165] According to another embodiment, the present specification further provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the aforementioned combination is implemented. Figure 3 The steps of the method.

[0166] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0167] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A training method for a recommendation system, wherein the recommendation system includes a ranking model and a recommendation model, and the method includes: Obtaining a user description of a target user and an object description of each object in a sampled object set, wherein the user description is determined according to a sequence of objects that the target user has interacted with, and the object set includes a first object that the target user has interacted with, and a plurality of negative objects that the target user has not interacted with; Inputting the user description and the object description of each object into a pre-trained ranking model and a recommendation model to be trained respectively, obtaining a first score determined by the ranking model reflecting the preference of the target user for each object, and a second score determined by the recommendation model reflecting the recommendation degree of each object to the target user, wherein the data set used for the pre-training includes objects sampled with equal probability from a sample object sequence; Determine ranking information according to a first score of the first object and first scores of a plurality of negative objects; determine a distillation loss according to the ranking information and second scores of each object; The recommendation model is updated according to at least the distillation loss.

2. The method according to claim 1, wherein: The object is an Internet product, and the interaction includes one or more of the following: clicking, browsing, collecting, and purchasing.

3. The method according to claim 1, wherein: The ranking model and recommendation model are both built based on the pre-trained text encoder.

4. The method according to claim 1, wherein: The objects in the object sequence are arranged in the order of interaction time.

5. The method according to claim 1, wherein: The object set is obtained by following the steps: sampling a first object from the object sequence based on a first sampling probability, wherein the first sampling probability is positively correlated with the popularity of each object in the object sequence; Based on random sampling probability, a number of negative objects are sampled.

6. The method according to claim 1 or 3, wherein: The user description is obtained by following the steps below: Based on a pre-built description template and description texts of each object in the object sequence, a description text of the target user is generated as the user description, wherein the description template includes text for prompting to recommend the target object to the user.

7. The method according to claim 1 or 3, wherein the object description is obtained by: The text corpus contained in any object is concatenated to obtain an object description corresponding to the object, wherein the text corpus includes at least one of the following: a text attribute name, a text value, and an object ID.

8. The method according to claim 1, wherein: The determination of distillation loss comprises: For any target negative object, if the ranking information indicates that its first score is less than the first score of the first object, determining the sample loss as a first difference between a second score negatively related to the first object and a second score of the target negative object; If the ranking information indicates that the first score of the target negative object is greater than the first score of the first object, determining the sample loss as a second difference between a second score negatively related to the target negative object and a second score of the first object; The distillation loss is obtained by averaging the sample losses of each negative object.

9. The method according to claim 1, wherein: The determination of distillation loss comprises: According to the ranking information, determine the object with the highest first score as the calculation object; The distillation loss is determined, the distillation loss being negatively correlated to a second fraction of the computational object.

10. The method according to claim 1, wherein: The determining of the ranking information includes: determining a first probability distribution reflecting the target user's preference ranking of each object according to the first score of each object in the object set; The determination of distillation loss comprises: Determining, according to the second scores of the respective objects in the object set, a second probability distribution reflecting the ranking of the recommendation degrees of the respective objects with respect to the target user; The distillation loss is calculated, where the distillation loss is negatively correlated to a similarity between the first probability distribution and the second probability distribution.

11. The method according to claim 1, wherein: The updating of the recommendation model at least according to the distillation loss further includes: determining a sampling loss that is negatively correlated to a first ratio, the first ratio being a calculated value of the second score of the first object relative to a sum of calculated values ​​of the second scores of the objects in the set of objects; The recommendation model is updated according to a weighted sum of the sampling loss and the distillation loss.

12. The method according to claim 1, wherein: The sorting model is pre-trained through the following steps: Inputting the user description of the sample user and the object description of each sample object in the sample object set into the ranking model to obtain a third score for each sample object; wherein the sample object set includes positive sample objects that the sample user has interacted with, and a number of negative sample objects that the sample user has not interacted with; determining a pre-training loss that is negatively correlated to a second ratio, the second ratio being a calculated value of the third score of the positive sample object relative to a sum of calculated values ​​of the third scores of each sample object in the sample object set; The ranking model is updated with the goal of minimizing the pre-training loss.

13. The method according to claim 12, wherein: The sorting model includes a text encoder and a multi-layer perceptron; the pre-training includes a first stage and a second stage; Updating the sorting model, including: In the first stage, the text encoder and the multi-layer perceptron are updated; In the second stage, only the multilayer perceptron is updated.

14. The method according to claim 13, wherein: The first sample object set used in the first stage is obtained by the following steps: Based on a second sampling probability, sampling a first positive sample object from a first historical object sequence interacted with by a first sample user, wherein the second sampling probability is positively correlated with the popularity of each object in the first historical object sequence; Based on the random sampling probability, a number of negative sample objects are sampled.

15. The method according to claim 13, wherein: The second sample object set used in the second stage is obtained by the following steps: Based on a third sampling probability, sampling a second positive sample object from a second historical object sequence interacted with by the second sample user, wherein the third sampling probability is equal probability; Based on the random sampling probability, a number of negative sample objects are sampled.

16. A training device for a recommendation system, the recommendation system comprising a ranking model and a recommendation model, the device comprising: An acquisition module: configured to acquire a user description of a target user and an object description of each object in a sampled object set, wherein the user description is determined according to a sequence of objects with which the target user has interacted, and the object set includes a first object with which the target user has interacted, and a plurality of negative objects with which the target user has not interacted; Evaluation module: configured to input the user description and the object description of each object into a pre-trained ranking model and a recommendation model to be trained respectively, to obtain a first score determined by the ranking model reflecting the preference of the target user for each object, and a second score determined by the recommendation model reflecting the recommendation degree of each object to the target user, wherein the data set used for pre-training includes objects sampled with equal probability from a sample object sequence; A loss determination module: configured to determine ranking information according to a first score of the first object and first scores of a plurality of negative objects; and determine a distillation loss according to the ranking information and second scores of each object; Model updating module: configured to update the recommendation model at least according to the distillation loss.

17. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15.

18. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 15 is implemented.

Images