An optimized recommendation method

Abstract

This paper discloses an optimized recommendation method that balances model accuracy and efficiency by partially decoupling different feature types and interacting with low-dimensional features in the upper layers of the model. This algorithm's model does not require negative sampling during training, effectively learning all content feature information and improving model accuracy.

Description

Technical Field

[0001] The invention relates to an optimized recommendation method and belongs to the technical field of recommendation methods. Background Art

[0002] With the rapid development of Internet technology, the amount of media and information content on the Internet has exploded. Traditional active search models can no longer meet users' needs for content screening. Content supply platforms need to provide more personalized content display services for each user based on factors such as user preferences and scenarios, helping users locate content of interest more quickly. Recommendation systems have emerged as a result.

[0003] The core of recommendation systems lies in precise recommendation models. Over the past decade, deep learning technology has been widely used in recommendation systems due to its powerful representation learning and function fitting capabilities. Deep learning models extract feature samples based on historical user behavior, content attributes, and other data. By building complex neural network models to fit user interests and preferences, they predict the content they are likely to interact with next and recommend it to them, significantly saving users time and improving their user experience.

[0004] Users' historical behavior fully reflects their preferences for content, while the attribute information of the content itself can explicitly reflect the user's interest characteristics. Currently, most of the recommendation methods commonly used in the industry are based on extracting features from these two types of data, and then constructing different algorithm models to fit the relationship between feature data and user behavior.

[0005] The classic DIN (Deep Interest Network for Click-Through Rate Prediction) model first calculates the weights of a user's historical behavior sequence and the target content, abstracting the user's interest representation. This model then combines this with other features, such as basic user attributes, content attributes, and contextual context, to calculate the probability of the user's interest in the content through a multi-layer fully connected neural network. This model effectively fits user behavior patterns and achieves excellent recommendation results. However, its drawback is its high computational complexity. The model calculation requires sequentially inputting the characteristics of each user and each content item to calculate the user's estimated score for the input content. The user's scores for all content items are then sorted from highest to lowest to produce a list of content items that the user is most likely to click on next. Faced with a massive pool of candidate content, the time cost of the model calculation becomes unacceptable.

[0006] To reduce model computational costs, the industry has widely adopted the twin-tower model, which sacrifices accuracy to improve computational speed. Specifically, user and content features are fed into two multi-layer neural networks, generating abstract user and content representations, respectively. The cosine similarity between these two representations is then used as the probability of a user's interest in the item. In the twin-tower model, since the computation of user and content representations is completely decoupled and content representations do not change in real time, pre-computing and storing all content representations significantly reduces the computational effort during model inference and improves inference speed. However, since this model structure has almost no overlap between user and content features, it is unable to fully exploit the potential information of the features, resulting in low model accuracy.

[0007] To balance recommendation accuracy and inference speed, the most common approach in the industry is to combine the two aforementioned models: first, using the dual-tower model to generate a pool of hundreds of candidate content, then using a more complex model to sequentially score and rank these smaller pieces of content to arrive at a final recommendation ranking. However, this two-stage recommendation model requires independent training of two different models, resulting in high application costs. Furthermore, using these two different models in tandem can easily lead to bias, making optimal recommendation results unreliable.

[0008] Common recommendation models currently used in the industry often struggle to balance accuracy and inference speed, while combining multiple models for inference increases the difficulty of building recommendation systems. Furthermore, low-precision first-stage models struggle to accurately recall all content a user might like, hindering overall accuracy improvements in the recommendation system. Summary of the Invention

[0009] The purpose of the present invention is to provide an optimized recommendation method that can take into account both the accuracy and efficiency of the model.

[0010] The technical solution of the present invention is an optimized recommendation method, which is characterized by comprising:

[0011] Input feature processing: user feature set f user and content feature set F item As the model input, it is converted into a D-dimensional embedded vector through the encoding layer, and then passes through two independent network structures for feature cross-calculation and representation abstraction calculation, and is converted into a K-dimensional vector, which is recorded as c user and C item ;

[0012] Simulation training method: user and C item Perform cross fusion to obtain the final score as the final predicted score of each content, and then process it through the softmax function to obtain the user's preference probability for each content predicted by the model, which is recorded as P i, complete the model establishment; use the positive samples of the user's actual interactive behavior to train the model, through multiple rounds of iterative training, until the predetermined number of iterations is reached or the P in the model is reached. i and item user The cross entropy Loss of item no longer decreases; user The actual interaction content of the user is known;

[0013] Model inference method: pre-calculate the representation vector C of all content item And store it. When used for online reasoning, it only needs to be based on f user Calculate c user , then take out the cached C item , and perform subsequent calculations together.

[0014] In the above-mentioned optimized recommendation method, the user feature set f user It includes basic user attributes, such as age, gender, and region, as well as user interest and preference characteristics based on historical user interaction behavior statistics, such as a list of content the user has recently interacted with and the user's most preferred content tags in the past 30 days.

[0015] The content feature set F item Contains the attribute features of the content, such as classification, creator, release time, subject matter, tags, etc. It should be noted that, unlike the traditional two-tower model which only inputs the features of one content at a time, this model inputs the features of all content into the model for calculation each time. Therefore, the content feature set here includes all candidate content and their features, i.e. N represents the total number of contents.

[0016] In the aforementioned optimization recommendation method, the c user and C item The specific methods of cross-fusion are:

[0017] Through a network structure, the K-dimensional user representation vector c is directly user Converted into an N-dimensional vector Score_1, where the value of each dimension score_1 i represents the user's preference score for each content i inferred from user characteristics,

[0018] c user and Perform vector dot multiplication calculations respectively to obtain the K-dimensional cross vector Then another network structure is used to calculate the unique output value score_2 i , represents the user's preference score for the content inferred based on the cross-information of the user and the content;

[0019] Use the fusion function to score_1 i and score_2 i Perform fusion calculation to get the final score i , score i This is the user's final predicted score for each content; the user's predicted scores for all content are processed by the softmax function to obtain the user's preference probability for each content predicted by the model, which is recorded as P i .

[0020] In the aforementioned optimization recommendation method, preferably, the Adam algorithm is used to optimize the model parameters before performing multiple rounds of iterative training of the model.

[0021] In the aforementioned optimized recommendation method, preferably, the network structure is a deep neural network.

[0022] In the aforementioned optimization recommendation method, as a preference, the fusion function can select the MLP network in the deep neural network, score_final i =MLP(<score_1 i ,score_2 i >), where

[0023] Compared with existing technologies, this invention reduces a significant amount of repetitive computation during training and inference by partially decoupling the processing of different types of features, significantly improving inference speed. By interacting with low-dimensional features in the upper layers of the model, it achieves better feature cross-pollination with less computation, enabling the simultaneous learning of feature information for all content, thereby improving the overall accuracy of the model. Furthermore, this invention can directly find a user's favorite content from the entire content pool, shortening the recommendation process and reducing the complexity of building a recommendation system.

[0024] This paper addresses the problem that current commonly used recommendation models in the industry cannot balance accuracy and inference speed. By optimizing the model structure, we have designed a method that does not require negative samples during training and can directly extract information from the entire content within an acceptable computational time during both training and inference, thereby improving recommendation effectiveness and simplifying the recommendation process. This method has achieved excellent recommendation results in actual use. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the algorithm model structure of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below with reference to the accompanying drawings and examples, but they are not intended to limit the present invention.

[0027] Example. An optimized recommendation method, whose model is as follows Figure 1 shown.

[0028] 1. Input feature processing

[0029] The input of the model consists of two parts: user feature set f user and content feature set F item .

[0030] f user It includes basic user attributes, such as age, gender, and region, as well as user interest and preference characteristics based on historical user interaction behavior statistics, such as a list of content the user has recently interacted with and the user's most preferred content tags in the past 30 days.

[0031] F item Contains the attribute features of the content (item), such as classification, creator, release time, subject matter, tags, etc. It should be noted that, unlike the traditional two-tower model which only inputs the features of one content at a time, this model inputs the features of all content into the model for calculation each time. Therefore, the content feature set here includes all candidate content and their features, i.e. N represents the total number of contents.

[0032] The original feature values ​​generally include two types: floating-point numerical features and string-type categorical features. In order to uniformly convert them into floating-point value vectors that the model can process, each original feature will be converted into a D-dimensional embedded vector through the encoding layer. user and Represent the encoded user embedding feature set and the embedding feature set of all contents respectively, Where uc represents the total number of features of each user, ic represents the total number of features of each content, and D is an integer. Represents a domain.

[0033] e user and E item After two independent deep neural networks (DNNs), feature cross-talk and representation abstraction calculations are performed between user features and content features respectively, and converted into K-dimensional vectors, where K is an integer and is denoted as c user and C item , where c user ∈R uc*K , C item ∈R N*ic*K .

[0034] In this phase, user and content features are calculated independently and decoupled from each other. This design not only facilitates parallel computing to shorten training time, but also reduces computing costs by merging calculations - multiple sample data in the same training batch can share C item value.

[0035] 2. Model training method

[0036] After calculating user features and content features separately, the model begins to integrate and cross-reference the two in order to better explore the potential correlation between users and content, and thus better simulate user preferences. The specific method is as follows:

[0037] First, a deep neural network (DNN) is used to directly transform the K-dimensional user representation vector c user Converted into an N-dimensional vector Score_1, where the value of each dimension score_1 i represents the user's preference score for each content i inferred from user characteristics,

[0038] Next, c user and Perform vector dot multiplication calculations respectively to obtain the K-dimensional cross vector Then another multi-layer deep neural network DNN is used to calculate the unique output value score_2 i , represents the user's preference score for the content inferred based on the cross-information of the user and the content.

[0039] Compared with traditional models that cross-calculate all user and content features at the original input layer, this model extracts abstract representations of both user and content features before cross-calculating them. This can reduce the dimension of the cross-features from (uc+ic)*D to K with limited precision loss, greatly reducing the model's computational complexity for each content. This allows the model to simultaneously calculate the feature information of all content each time, thereby improving the model's learning ability.

[0040] Finally, the two scores are fused to obtain the final score. In this solution, an MLP in DNN can be used as the fusion function

[0041] score i =MLP(<score_1 i ,score_2 i >), where

[0042] score iThis is the user's final predicted score for each content. The user's predicted scores for all content are processed by the softmax function to obtain the user's preference probability for each content predicted by the model, which is recorded as P i .

[0043] This model only needs to be trained with positive samples of actual user interaction behaviors. user , the model uses P i and item user The cross entropy of is used as Loss. Loss = CrossEntroyLoss(P,item user )=-logP(item i =item user )

[0044] P(item i =item user ) means that each item _i There is a corresponding probability value P i . Take out item from all P _user The P value of the corresponding item.

[0045] Next, we can use the Adam algorithm to optimize the model parameters through multiple rounds of iterative training until the predetermined number of iterations is reached or the model loss no longer decreases.

[0046] 3 Model Inference Method

[0047] When performing inference prediction based on a trained model, theoretically, each time, the user and all content features must be fully calculated using the methods described in 2.2.1 and 2.2.2. The top N content with the highest probability values, calculated from the final output of the user's preference probability P for all content, are then recommended to the user. However, this computational cost is clearly unacceptable for online inference.

[0048] Considering that in real-world scenarios, user characteristics will constantly change with new interactions, while content characteristics generally remain stable over time, combined with the model features described in this article, the following optimizations can be performed during inference:

[0049] Since the input F item unchanged, so after the model training is completed, the C item Therefore, after the model training is completed, the representation vector C of all content can be pre-calculated. item And store it. When used for online reasoning, only need to be based on f user Calculate cuser , then take out the cached C item , and subsequent calculations are performed together, which greatly reduces the computational overhead and shortens the inference time.

[0050] Among them, the multi-layer neural network DNN in 1 and 2 can be replaced by other network structures, such as sum_pooling, transformer, etc.

[0051] The fusion function in 3 can also be replaced by other rules, such as direct addition or averaging, etc. The model optimization method in 3 can be replaced by other optimization algorithms, such as SGD, RMSProp, etc.

Claims

1. An optimized recommendation method, characterized in that: include: Input feature processing: user feature set f user and content feature set F item As the model input, it is converted into a D-dimensional embedded vector through the encoding layer, and then passes through two independent network structures for feature cross-calculation and representation abstraction calculation, and is converted into a K-dimensional vector, which is recorded as c user and C item ; Simulation training method: user and C item Perform cross fusion to obtain the final score as the final predicted score of each content, and then process it through the softmax function to obtain the user's preference probability for each content predicted by the model, which is recorded as P i , complete the model establishment; use the positive samples of the user's actual interactive behavior to train the model, through multiple rounds of iterative training, until the predetermined number of iterations is reached or the P in the model is reached. i and item user The cross entropy Loss of item no longer decreases; user The actual interaction content of the user is known; Model inference method: pre-calculate the representation vector C of all content item And store it. When used for online reasoning, it only needs to be based on f user Calculate c user , then take out the cached C item , and perform subsequent calculations together; The pair c user and C item The specific methods of cross-fusion are: Through a network structure, the K-dimensional user representation vector c is directly user Converted into an N-dimensional vector Score_1, where the value of each dimension score_1 i represents the user's preference score for each content i inferred from user characteristics, c user and Perform vector dot multiplication calculations respectively to obtain the K-dimensional cross vector Then another network structure is used to calculate the unique output value score_2 i , represents the user's preference score for the content inferred based on the cross-information of the user and the content; Use the fusion function to score_1 i and score_2 i Perform fusion calculation to get the final score i , score i This is the user's final predicted score for each content; the user's predicted scores for all content are processed by the softmax function to obtain the user's preference probability for each content predicted by the model, which is recorded as P i .

2. The optimized recommendation method according to claim 1, characterized in that: The user feature set f user Contains the user's basic attributes, the content feature set F item Contains attribute characteristics of the content.

3. The optimized recommendation method according to claim 1, characterized in that: The Adam algorithm is used to optimize the model parameters before multiple rounds of iterative training of the model.

4. The optimized recommendation method according to claim 1, characterized in that: The network structure is a deep neural network.

5. The optimized recommendation method according to claim 1, characterized in that: The fusion function is an MLP network, score_final i =MLP( <score_1 i ,score_2 i >), where

Images