Recommendation system based on cross-model denoising
By combining neural collaborative filtering and matrix factorization models, and utilizing KL divergence calculation and reliability filtering, noisy samples are eliminated and backfilled, overcoming the limitations of single-model denoising recommendation systems and improving the accuracy and reliability of recommendation systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2024-07-12
- Publication Date
- 2026-07-31
AI Technical Summary
Existing single-model denoising recommendation systems have limitations in processing implicit feedback data, leading to suboptimal recommendations, failing to effectively filter noise, and affecting the accuracy and reliability of recommendations.
A cross-model denoising method is adopted, which combines the Neural Collaborative Filtering (NCF) model and the Matrix Factorization (MF) model. The distribution consistency of the prediction results of the master and auxiliary models is calculated by KL divergence. High KL value noise samples are removed and then backfilled into the positive samples after reliability filtering, thereby improving the accuracy and reliability of the recommendation system.
By working together with the primary and secondary models, noisy samples are effectively filtered, improving the accuracy and reliability of the recommendation system and generating more accurate personalized recommendation lists.
Smart Images
Figure CN118779524B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of recommendation technology, and in particular to a recommendation system based on cross-model denoising. Background Technology
[0002] In the era of big data, information overload has become a serious problem. Recommender systems, as one of the most well-known and effective information filtering techniques, can uncover user preferences by analyzing past interactions. Implicit feedback learning is the most common approach in recommender system applications. Typically, items that a user has interacted with are considered positive (i.e., items the user prefers), while uninterrupted items are considered negative (i.e., items the user dislikes).
[0003] However, these implicit data often contain noise. For example, user interactions unintentionally may not fully reflect their true preferences; or users may not interact with certain items due to unfamiliarity, even though these items could represent potential preferences. Therefore, denoising of the implicit data is necessary to improve the system's recommendation accuracy. Current denoising recommendation systems typically process implicit feedback data using a single model, which has limitations and can produce suboptimal recommendations. Therefore, combining a main model and an auxiliary model, and using the consistency of the two models' results and the accuracy of the main model's recommendations for denoising, not only provides better interpretability but also improves the accuracy and reliability of recommended items. This dual-model consistency approach can more effectively filter out noise, more accurately capture users' true preferences, and thus improve the overall performance of the recommendation system. Summary of the Invention
[0004] To address the aforementioned issues and ensure the consistency between the primary and secondary models and the accuracy and reliability of the recommendation results, this invention proposes a recommendation system based on cross-model denoising, starting from implicit feedback learning, comprising the following modules:
[0005] The data acquisition and preparation module is used to acquire and clean the data collected from the database to obtain the required raw interaction matrix containing only 0s and 1s.
[0006] The negative sample generation module is used to generate corresponding negative samples based on the positive samples in the original interaction data.
[0007] The KL divergence calculation module is used to calculate the KL divergence values of the distribution of the prediction results of the main model and the auxiliary model for positive and negative samples using the KL divergence calculation formula.
[0008] The positive and negative sample denoising module is used to sort the KL values from high to low based on the distribution of the predicted values of the main model and the auxiliary model, and remove noise samples with relatively high KL values from the positive and negative samples.
[0009] The reliability filtering module is used to calculate the difference between the prediction results of the main model and the auxiliary model for the removed negative samples, compare the normalized difference with the reliability threshold, and select the negative samples with high reliability.
[0010] The negative sample backfilling module is used to fill the denoised positive samples with samples that are below a set threshold from the removed negative samples.
[0011] The recommendation result acquisition module is used to obtain the main model's predicted values and push the top k items with the highest predicted values to the target user to form their personalized recommendation list.
[0012] The negative sample generation module of this invention, based on the original interaction matrix, treats the interacting user-item pairs as positive samples, denoted as... At the same time, randomly sample the corresponding negative samples, i.e., user-item pairs that have not interacted, denoted as . The sampling method is as follows:
[0013]
[0014] in This indicates that the user and the project have not interacted.
[0015] The KL divergence calculation module of this invention uses the Neural Collaborative Filtering (NCF) model as the main model to obtain the final predicted value and generate a recommendation list. Simultaneously, it uses the Matrix Factorization (MF) model as an auxiliary model to assist in the training of the NCF model. Specifically, it includes the following steps:
[0016] 1) The NCF model training process integrates the Generalized Matrix Factorization (GMF) model and the Multilayer Perceptron (MLP) model, with the MLP and GMF models learning their respective user and item embeddings. Therefore, the model prediction is expressed as:
[0017] ,
[0018] ,
[0019] ,
[0020] in , These represent the user and item embedding vectors in the GMF model, respectively. , These represent the user and item embedding vectors of the MLP model, respectively.
[0021] 2) During the training process of the MF model, the user-item interaction matrix is mapped into two low-dimensional matrices: a user embedding vector and an item embedding vector. The predicted value of user u for item i is approximated by calculating the inner product of the user vector and the item vector. Therefore, the model's prediction can be expressed as:
[0022]
[0023] in , These represent the user and project feature embedding vectors, respectively.
[0024] 3) Calculate the KL divergence values for the NCF and MF model predictions for both positive and negative samples. Since KL divergence is asymmetric, this may affect model performance. Therefore, considering symmetry, the formula for calculating KL divergence is transformed into:
[0025] ,
[0026] , ,
[0027] in and These are the predicted probability values for the NCF and MF models, respectively.
[0028] The positive and negative sample denoising module of this invention sorts the KL values from high to low and removes noise samples with relatively high KL values from the positive and negative samples. Simultaneously, the number of noise samples removed increases linearly. Therefore, the removal ratio expression is:
[0029]
[0030] in To increase the coefficient, This represents the maximum percentage to be eliminated.
[0031] The reliability filtering module of this invention calculates and normalizes the prediction results of the NCF model and MF model for the removed negative samples, compares the difference between the prediction results with a reliability threshold, and filters out negative samples that are higher than the reliability threshold. The specific calculation formula is as follows:
[0032] ,
[0033] ,
[0034] ,
[0035] in These are the predicted values for negative samples by the NCF model and the MF model, respectively. The difference after normalization. These are the filtered, reliable negative samples.
[0036] The negative sample backfilling module of this invention fills positive samples with user-item pairs that are below the reliability threshold from the removed negative samples.
[0037] The recommendation result acquisition module of this invention is used to obtain a recommendation list for a target user u. The predicted values of items not interacted with by the user are sorted, and the top k items are selected to form a personalized recommendation list for the target user u. To ensure the accuracy and reliability of the model's recommendations, a loss function is constructed based on cross-entropy loss. Simultaneously, to ensure the consistency of predictions between the primary and secondary models, the KL divergence value is included as part of the loss function. Furthermore, to prevent overfitting, L2 regularization is used. The Adam optimizer is employed to optimize the model parameters. The loss function is as follows:
[0038] ,
[0039] in and These represent the sets of positive and negative samples used in model training. and These represent the actual value and the predicted value, respectively. These represent the predicted values of the NCF and MF models for positive and negative samples, respectively. The coefficient of the regularization term, These are the weight parameters of the multilayer perceptron.
[0040] This invention proposes a recommendation system based on cross-model denoising. It utilizes the NCF model as the primary prediction model and the MF model as an auxiliary model. Then, it employs KL divergence to measure the consistency of the prediction distributions between the NCF and MF models, eliminating noisy samples with high KL divergence values from both positive and negative samples to improve recommendation accuracy. Simultaneously, samples removed from the negative samples are reinstated into the positive samples, with full consideration given to the reliability of the reinstated samples to ensure both recommendation diversity and accuracy. Furthermore, within the NCF model framework, the GMF and MLP models are integrated to learn user preferences from historical behavior, further enhancing recommendation quality. Attached Figure Description
[0041] Figure 1 This is a schematic diagram of the recommendation system process of the present invention.
[0042] Figure 2 This is a framework diagram of the recommendation system of the present invention.
[0043] Figure 3 This is a comparison of the Recall@5, RecallL@10, NDCG@5, and NDCG@10 values of this invention with other methods on the MovieLens dataset. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of this invention clearer, a recommendation system based on cross-model denoising proposed in this invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific implementation methods described herein are merely illustrative of the invention and are not intended to limit the invention. Any changes, modifications, additions, alterations, or substitutions made by those skilled in the art within the scope of this invention should be covered by the claims of this invention.
[0045] Figure 1 This is a schematic diagram illustrating the process of a recommendation system based on cross-model denoising according to the present invention. From Figure 1 As can be seen, the proposed recommendation system based on cross-model denoising includes acquiring and cleaning data information from a database to obtain the required original user-item interaction matrix. Then, positive samples (interacting user-item pairs) and negative samples (non-interacting user-item pairs) are generated based on the user-item interaction matrix. Next, the KL divergence of the predicted values of the NCF model and MF model for the positive and negative samples is calculated, and the KL divergence values are sorted from high to low, removing noisy positive and negative samples with high KL divergence values. The difference between the predictions of the NCF model and MF model is then calculated for the removed negative samples and normalized. Simultaneously, the normalized difference is compared with a set reliability threshold, filtering unreliable negative samples and filling the positive samples with reliable negative samples. A neural collaborative filtering model is used to learn the user's global preferences, using the user's historical interaction information with items as input to generate predicted values and sort them to generate a list of recommended items.
[0046] Figure 2 This is a framework diagram of a recommendation system based on cross-model denoising according to the present invention. From Figure 2As can be seen, the recommendation system based on cross-model denoising proposed in this invention includes a data acquisition and preparation module, a negative sample generation module, a KL divergence calculation module, a positive and negative sample denoising template, a reliability filtering module, a negative sample backfilling module, and a recommendation result acquisition module. The data acquisition and preparation module extracts data from the database and performs cleaning and preprocessing operations to obtain the required original interaction matrix. The negative sample generation module, based on the original interaction matrix, uses interacting user-item pairs as positive samples and randomly samples non-interacting user-item pairs as negative samples. The KL divergence calculation module uses the KL divergence calculation formula to calculate the KL values of the main model and auxiliary model prediction results distributions for positive and negative samples, respectively. The positive and negative sample denoising module sorts the KL values of the main and auxiliary model prediction distributions from high to low and removes noise samples with relatively high KL values from the positive and negative samples. The reliability filtering module calculates the difference between the main model and auxiliary model prediction results after the negative samples have been removed, compares the normalized difference with a reliability threshold, and selects negative samples with high reliability. The negative sample backfilling module fills in positive samples with samples below a set threshold from the removed negative samples, meaning the prediction results of the two models are close. Filling in qualified negative samples with positive samples improves the diversity and reliability of recommendations. The recommendation result acquisition module takes the user's historical behavior information as input, learns the user's preferences through a neural collaborative filtering model, sorts the predicted values of items the user has not interacted with, and selects the top k items to form a personalized recommendation list for the target user u.
[0047] Furthermore, the following example illustrates this further:
[0048] The MovieLens dataset contains M users. and N projects The rating range is {1, 2, 3, 4, 5}. Using... Represents the true preference matrix, This represents user u's true preference for item i. If there is interaction between user u and item i, then... ,otherwise .
[0049] First, the system data is acquired and cleaned through the data acquisition and preparation module to obtain the variables in the hypothesis. The specific implementation steps of the proposed recommendation system based on cross-model denoising are as follows:
[0050] S1: The data acquisition and preparation module retrieves the required data information from the database, including user ID, project ID, and interaction matrix R;
[0051] S2: Negative sample generation module, which uses interacted user-item pairs as positive samples and randomly generates non-interactive user-item pairs as negative samples. It uses a batch training model, therefore, in each training batch... Select a set of positive samples For each pair in this group ,from A set of negative samples was randomly selected from the middle. .
[0052] S3: The KL divergence calculation module uses the NCF model as the main model to train the model's parameters and obtain the model's predicted values to generate a recommendation list. Simultaneously, the MF model is used as an auxiliary model to assist in the training of the NCF model. Both models use the Adam optimizer to optimize their parameters. Specifically, the steps are as follows:
[0053] 1) The NCF model training process involves fusing the GMF and MLP models, with each model learning its own user and item embeddings. Therefore, the model prediction is expressed as:
[0054] ,
[0055] ,
[0056] ,
[0057] in , These represent the user and item embedding vectors in the GMF model, respectively. , These represent the user and item embedding vectors of the MLP model, respectively.
[0058] 2) During the training process of the MF model, the user-item interaction matrix is mapped into two low-dimensional matrices: the user embedding vector and the item embedding vector. The prediction of user u for item i is approximated by calculating the inner product of the user vector and the item vector. Therefore, the prediction of this model can be expressed as:
[0059] ,
[0060] in , These represent the user and project feature embedding vectors, respectively.
[0061] 3) Calculate the KL divergence of the NCF and MF model predictions for both positive and negative samples, converting the asymmetric KL calculation to a symmetric one. This includes the symmetric calculation expression:
[0062] ,
[0063] , ,
[0064] in and These are the predicted probability values for the NCF model and the MF model, respectively.
[0065] S4: Positive and negative sample denoising module. It sorts KL values from high to low and removes noise samples with relatively high KL values from both positive and negative samples. Removal ratio expression:
[0066] ,
[0067] in To increase the coefficient, This represents the maximum percentage to be eliminated.
[0068] S5: Reliability filtering module. This module calculates the prediction results of the NCF and MF models for the removed negative samples, normalizes the difference between these predictions, and compares it with a reliability threshold. Negative samples exceeding the reliability threshold are filtered out. The calculation process is as follows:
[0069] ,
[0070] ,
[0071] ,
[0072] in These are the predicted values for negative samples by the NCF model and the MF model, respectively. The difference after normalization. These are the filtered, reliable negative samples.
[0073] S6: Negative sample backfilling module, which fills the user-item pairs below the reliability threshold from the removed negative samples into the denoised positive samples.
[0074] S7: Recommendation Result Acquisition Module. This module retrieves the recommendation list for the target user u. It sorts the predicted values of items not interacted with by the user and selects the top k items to form a personalized recommendation list for user u. To ensure the accuracy and reliability of the model's recommendations, a loss function is constructed based on cross-entropy loss. Simultaneously, to ensure consistency between the predictions of the primary and secondary models, the KL divergence value is included as part of the loss function. Furthermore, L2 regularization is used to prevent overfitting. The Adam optimizer is employed to optimize the model parameters. The loss function is as follows:
[0075] ,
[0076] in and These represent the sets of positive and negative samples after model training and denoising, respectively. and These represent the actual value and the predicted value, respectively. These represent the predicted values of the NCF and MF models for positive and negative samples, respectively. The coefficient of the regularization term, These are the weight parameters of the multilayer perceptron.
[0077] Figure 3 The proposed method was measured on two datasets and two metrics, and compared with other methods. The two metrics are: Recall and Standardized Discount Cumulative Return (NDCG). Their calculation methods and measurement details are shown below:
[0078] Recall: The ratio of the number of items a user actually purchases from the list of recommended items to the number of items the user actually buys. The percentage is calculated as follows:
[0079] ,
[0080] in The top K items in the recommended list for user u. This represents the projects that user u has actually interacted with.
[0081] Standardized Discount Cumulative Return (NDCG): This indicates the quality of the model's item ranking recommendations by assigning higher scores to higher-ranking clicks. The calculation formula is as follows:
[0082] ,
[0083] ,
[0084] in, This represents the cumulative gain from recommending K items to user u. This indicates the relevance of the recommendation results based on location idx. If the item is in the user's actual purchase item set, then... The value is 1 if it is 1, otherwise it is 0. Is it pressed The ideal calculated on the descending recommended list K represents the length of the recommendation list.
Claims
1. A recommendation system based on cross-model denoising, characterized in that, This module is used to denoise and fill in implicit data to improve recommendation quality, and includes the following modules: The data acquisition and preparation module is used to acquire and clean the data collected from the database to obtain the required raw interaction matrix containing only 0s and 1s; The negative sample generation module is used to generate corresponding negative samples based on the positive samples in the original interaction data. The KL divergence calculation module is used to calculate the KL divergence values of the distributions of the prediction results of the main model and the auxiliary model for positive and negative samples using the KL divergence calculation formula. The Neural Co-processing Model (NCF) is used as the main model to obtain the final predicted values and generate a recommendation list. Simultaneously, the Matrix Factorization Model (MF) is used as the auxiliary model to assist in the training of the NCF model. This training includes the following steps: S1: The NCF model training process integrates the Generalized Matrix Factorization (GMF) model and the Multilayer Perceptron (MLP) model, with the MLP and GMF models learning their respective user and item embeddings; therefore, the model prediction is expressed as: ， ， ， in , These represent the user and item embedding vectors in the GMF model, respectively. , These represent the user and item embedding vectors of the MLP model, respectively. S2: The MF model training process maps the user-item interaction matrix into two low-dimensional matrices: user embedding vectors and item embedding vectors. The predicted value of user u for item i is approximated by calculating the inner product of the user vectors and item vectors. Therefore, the prediction is expressed as: ， in , These represent the user and project feature embedding vectors, respectively. S3: Calculate the KL divergence values of the NCF and MF model prediction results for positive and negative samples respectively; since the KL divergence is asymmetric, this may affect the performance of the recommendation; therefore, considering symmetry, the formula for calculating KL is transformed into: ， ， ， in and These are the predicted probability values for the NCF and MF models, respectively. The positive and negative sample denoising module is used to sort the KL values from high to low based on the distribution of the predicted values of the main model and the auxiliary model, and remove noise samples with relatively high KL values from the positive and negative samples. The reliability filtering module is used to calculate the difference between the prediction results of the main model and the auxiliary model for the removed negative samples, compare the normalized difference with the reliability threshold, and select negative samples with high reliability. The negative sample backfilling module is used to fill the denoised positive samples with samples that are below a set threshold from the removed negative samples. The recommendation result acquisition module is used to obtain the main model's predicted values and push the top k items with the highest predicted values to the target user to form their personalized recommendation list.
2. The recommendation system based on cross-model denoising as described in claim 1, characterized in that, The negative sample generation module, based on the original interaction matrix, treats the interacting user-item pairs as positive samples, denoted as... Simultaneously, a corresponding negative sample is randomly selected and denoted as... That is, user-item pairs that have not interacted; the sampling method is: ， in This indicates that the user and the project have not interacted.
3. The recommendation system based on cross-model denoising as described in claim 1, characterized in that, The positive and negative sample denoising module sorts the KL values from high to low and removes noise samples with relatively high KL values from the positive and negative samples; at the same time, it increases the number of noise samples to be removed in a linear manner. Therefore, the elimination ratio expression is: ， in To increase the coefficient, This represents the maximum percentage to be eliminated.
4. The recommendation system based on cross-model denoising as described in claim 1, characterized in that, The reliability filtering module calculates and normalizes the prediction results of the NCF model and MF model for the removed negative samples, compares the difference between the prediction results with a reliability threshold, and filters out negative samples that are higher than the reliability threshold; the specific calculation formula is as follows: ， ， ， in and These are the predicted values for negative samples by the NCF model and the MF model, respectively. The difference after normalization. These are the filtered, reliable negative samples.
5. In the recommendation system based on cross-model denoising as described in claim 1, the negative sample backfilling module fills the positive samples with user-item pairs that are below the reliability threshold from the removed negative samples.
6. A recommendation system based on cross-model denoising as described in claim 1, characterized in that, The recommendation result acquisition module is used to acquire the recommendation list for the target user u; The predicted values of items not interacted with by the user are sorted, and the top k items are selected to form a personalized recommendation list for the target user u. To ensure the accuracy and reliability of the model's recommendations, a loss function is constructed based on cross-entropy loss. Simultaneously, to ensure the consistency of predictions between the master and slave models, the KL divergence value is included as part of the loss function. Furthermore, to prevent overfitting, L2 regularization is used, and the Adam optimizer is employed to optimize the model parameters. The loss function is as follows: ， in and These represent the sets of positive and negative samples used in model training. and These represent the actual value and the predicted value, respectively. These represent the predicted values of the NCF and MF models for positive and negative samples, respectively. The coefficient of the regularization term, These are the weight parameters of the multilayer perceptron.