Personalized robust recommendation defense method based on dynamic reward mechanism
By employing a dynamic reward mechanism and personalized adversarial perturbation strategies, the robustness and fairness issues of recommendation systems under data poisoning attacks are addressed, improving the dynamic balance between the defense effectiveness and recommendation quality of recommendation systems. This approach is applicable to various models and scenarios.
Patent Information
- Application Number
- CN202511476692.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-16
- Publication Date
- 2026-01-09
AI Technical Summary
Existing recommendation systems suffer from insufficient robustness, lack of fairness, and a lack of personalized and dynamic adaptability in adversarial training strategies when facing data poisoning attacks, leading to decreased recommendation accuracy and unfair treatment of users.
A personalized robust recommendation defense method based on a dynamic reward mechanism is adopted. By constructing a dual-objective impact assessment mechanism of robustness and fairness, the impact value of each user-item interaction sample pair is quantified. The adversarial perturbation amplitude is dynamically adjusted according to user behavior characteristics, and a multi-dimensional reward function is designed to guide the model to balance defense and recommendation quality during adversarial training.
It improves the model's recommendation performance in clean data scenarios, reduces the success rate of attacks, enhances the system's resistance to attacks, and ensures fairness and recommendation accuracy for the user group. It is applicable to a variety of basic models and attack scenarios.
Smart Images
Figure CN121301926A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a personalized robust recommendation defense method based on a dynamic reward mechanism. Background Technology
[0002] Recommendation systems, as an important bridge connecting users and information, have been widely used in many fields such as e-commerce, social media, music services, and news push, effectively alleviating the problem of information overload, accurately pushing the content needed by users, and promoting the development of related industries.
[0003] However, due to the open and collaborative nature of recommender systems, their security has long faced challenges. Some attackers manipulate recommendation results by injecting malicious data (i.e., poisoning attacks) to gain illicit profits. For example, in e-commerce scenarios, attackers might create false associations between popular and low-quality products, inducing the recommender system to prioritize low-quality items. Such malicious behavior not only degrades user experience but also undermines the accuracy and fairness of recommendation results, hindering the healthy development of recommender systems.
[0004] To address these security threats, the industry has conducted extensive research on poisoning attack defense strategies. Existing defense methods can be broadly categorized into two types: passive attack detection and proactive robust model construction. Passive attack detection primarily identifies malicious users by analyzing the potential characteristics of data such as user-rating matrices, using statistical methods or machine learning models.
[0005] However, these methods often rely on mining data features, which may lead to problems such as delayed identification or insufficient accuracy when facing complex and ever-changing attack methods. Proactive robust model building focuses on enhancing the system's resistance to attacks during the model design and training stages, such as through adversarial training, optimizing the training process, or improving the model architecture.
[0006] However, existing active defense methods still have many problems to be solved: First, most methods focus too much on improving the robustness of the model (such as reducing the success rate of attacks and restoring the accuracy of the model), while ignoring the need for fairness in recommendation systems, which may lead to unfair treatment of different groups of users in the recommendation results. Secondly, existing adversarial training methods mostly adopt a globally uniform perturbation strategy, which fails to fully consider the differences in behavior patterns, activity levels, and item characteristics among different users. This makes it difficult to adapt to such heterogeneity and may cause a decrease in recommendation accuracy for some users. Third, while a few personalized adversarial training methods attempt to adjust the perturbation amplitude based on user characteristics, they cannot dynamically optimize the perturbation strategy according to the real-time recommendation effect, thus limiting their robustness in complex attack scenarios.
[0007] Therefore, in order to address the problems of insufficient robustness, lack of fairness, and lack of personalized and dynamic adaptability of adversarial training strategies in existing recommendation systems when facing data poisoning attacks, it is an urgent technical problem for those skilled in the art to provide a method that can improve the system's anti-attack capability while taking into account recommendation quality and group fairness, and achieve a dynamic balance between defense effect and recommendation performance. Summary of the Invention
[0008] In view of this, the present invention provides a personalized robust recommendation defense method based on a dynamic reward mechanism.
[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A personalized robust recommendation defense method based on a dynamic reward mechanism includes the following steps: S1. Data Acquisition and Preprocessing: Obtain real-world datasets related to the recommendation system, clean the datasets, filter out invalid interaction data, retain valid user-item interaction pairs, normalize the rating data, and divide them into training, validation and test sets. S2. Sample selection based on influence function: A robustness and fairness dual-objective impact assessment mechanism is constructed to quantify the impact of each user-item interaction sample on the model; S3. Personalized Counter-Perturbation Strategy Design: Dynamically adjust the magnitude of counter-disturbance based on different user behavior characteristics; S4. Construction of a multi-dimensional reward function: The design incorporates a reward mechanism that integrates embedded interaction, behavioral similarity, and batch optimization to guide the model to dynamically balance defense and recommendation quality. S5. Model Training and Optimization: The model is trained based on the filtered dataset, combining personalized adversarial perturbations and multi-dimensional reward functions.
[0010] Preferably, step S1 specifically includes the following steps: S1.1. Data Acquisition: Collect real-world datasets related to the recommendation system, including user-item interaction records, covering 796-2848 users, 1682-39586 items, and 30,000-890,000 interaction data points; S1.2. Data cleaning: Filter out invalid samples, including abnormal ratings, duplicate interaction records and cold start users, and retain valid user-item interaction pairs (u,i); S1.3. Data Normalization: The minimum-maximum normalization method is used to map the scoring data to the [0,1] interval, as shown in the following formula:
[0011] in, The original rating given by user u to item i. These are the minimum and maximum scores in the dataset, respectively. The score is the normalized score; S1.4. Dataset Partitioning: The processed dataset is divided into training, validation, and test sets in an 8:1:1 ratio.
[0012] Preferably, step S2 specifically includes the following steps: S2.1. Calculation of Robustness Impact Value: For a training sample (u,i), its robustness impact value is measured by the change in model loss under adversarial perturbation, using the following formula:
[0013] Where V' is the adversarial verification set. Let be the gradient of the loss function with respect to the model parameters θ. Let be the Hessian matrix of the loss function; To simplify the calculation, the Hessian vector product (HVP) approximation is used. : ; S2.2. Calculation of Fairness Impact Value: Based on the demographic parity (DP) principle, the impact of the sample on the prediction bias of different sensitive attribute groups is quantified. The formula is as follows:
[0014] in, The fairness loss function measures the group prediction bias. S2.3. Comprehensive Impact Screening: A weighted fusion of robustness and fairness impact values is used to obtain the comprehensive impact score of the sample.
[0015] Where α and β are weight parameters, according to Sort the samples in ascending order and remove low-impact samples with a percentage of γ, where γ defaults to 3%, to obtain the filtered dataset D'.
[0016] Preferably, step S3 specifically includes the following steps: S3.1. Calculation of User Embedding Norm: Let the user embedding matrix be... Where n is the number of users, d is the embedding dimension, and the embedding vector of user u in the t-th training round is... Calculate the average of the embedding norms for all users: ; S3.2. Determining the perturbation amplitude: The user embedding norm is standardized to user-specific coefficients using the sigmoid function, and the maximum perturbation amplitude is determined by combining this with the baseline value μ.
[0017] in, These are the standardized coefficients; S3.3. Construction of Personalized Adversarial Loss: Based on the Bayesian Personalized Ranking (BPR) loss, an adversarial loss is introduced, with the following formula:
[0018] in, For the original BPR loss, For personalized perturbations, the amplitude shall not exceed λ is the adversarial loss weight, which defaults to 1.0.
[0019] Preferably, step S4 specifically includes the following steps: S4.1. Embedded Interaction Reward: Measures the matching degree between recommended items and user preferences, calculated through the inner product difference of positive and negative sample embeddings.
[0020] in, Embedded for users, Embedding of positive and negative sample items respectively. This is an inner product operation; S4.2. Behavioral Similarity Reward: Based on the user's historical interaction matrix, calculate the coverage of recommended items among similar users:
[0021] in, For a set of similar users, For similarity threshold, Indicates user With items It is interactive; S4.3. Maximum Reward within a Batch: The maximum reward for the same user sample within the mini-batch is taken to avoid suboptimal solutions.
[0022] in, Let be the sample set of user u in the current batch; S4.4. Total Reward Fusion: Weighted merging of the three reward categories, using the following formula:
[0023] Where α is the weight parameter for the maximum reward in a batch, which defaults to 0.5.
[0024] Preferably, step S5 specifically includes the following steps: S5.1. Definition of Maximum Reward Value Function: Based on the Extended Markov Decision Process (MDP), the maximum reward value function is defined as follows:
[0025] Where γ is the discount factor, which defaults to 0.9; The value function satisfies the Bellman equation:
[0026] Where s represents the state, containing user characteristics and historical best reward auxiliary variables. 'a' represents the recommended action, and 'V' represents the operation of taking the maximum value. S5.2. Construction of the total loss function: This function integrates the BPR loss, personalized adversarial loss, and maximum reward loss. The formula is as follows:
[0027] in, To maximize the reward loss; S5.3. Parameter Optimization: The model parameters are updated using the policy gradient method. The gradient formula is:
[0028] in, For the discounted steady-state distribution of the extended state, The action value function is trained iteratively until the loss converges.
[0029] The present invention achieves the following technical effects compared to the prior art: (1) The sample screening mechanism based on the influence function of this invention accurately removes low-value data by quantifying the impact of samples on robustness and fairness, effectively reducing the pollution of the model by poisoned samples. Experiments show that this mechanism improves the recommendation performance of the model by 19.92% in clean data scenarios; (2) The personalized adversarial perturbation strategy of this invention dynamically adjusts the perturbation amplitude according to the user's embedded features, avoiding the performance imbalance caused by the "one-size-fits-all" strategy. While ensuring the recommendation accuracy of highly active users, it strengthens the protection of low-active users and improves the model's adaptability to heterogeneous user behavior. (3) This invention integrates multi-dimensional reward functions with embedded matching degree, user behavior similarity and batch optimal information, guides the model to dynamically balance defense effect and recommendation quality in adversarial training, reduces the attack success rate by an average of 20.50%, and significantly enhances the system's anti-attack capability. (4) The method of the present invention takes into account both robustness and fairness, and achieves synergistic optimization of defense and recommendation performance through dynamic reward mechanism. It is applicable to different base models (such as MF, LightGCN) and various attack scenarios (such as RandomAttack, CleaR Attack, etc.), and has wide applicability and practicality. Attached Figure Description
[0030] Figure 1 This is a flowchart illustrating an embodiment of the present invention; Figure 2 This demonstrates the attack performance of this invention across various datasets and different base models. Figure 3 This invention demonstrates the performance of the recommendation system under various attack scenarios across multiple datasets. Figure 4 This demonstrates the attack performance of this invention across various datasets and different base models. Figure 5 This paper presents the performance of the recommendation system of this invention under different base models on various datasets. Detailed Implementation
[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0033] like Figure 1 As shown, this invention discloses a personalized robust recommendation defense method based on a dynamic reward mechanism, comprising the following steps: S1. Data Acquisition and Preprocessing S1.1. Data Acquisition: Collect real-world datasets related to the recommendation system, including user-item interaction records (such as ratings, clicks, purchases, etc.). Exemplary datasets include FilmTrust, ML-100K, and DouBan, covering 796-2848 users, 1682-39586 items, and 30,000-890,000 interaction data.
[0034] S1.2. Data Cleaning: Filter out invalid samples, including abnormal ratings (such as ratings that exceed a reasonable range), duplicate interaction records, and cold start users (thresholds can be set according to the actual scenario, such as users with fewer than 5 interactions), and retain valid user-item interaction pairs (u,i).
[0035] S1.3. Data Normalization: The minimum-maximum normalization method is used to map the scoring data to the [0,1] interval, as shown in the following formula: in, The original rating given by user u to item i. These are the minimum and maximum scores in the dataset, respectively. This is the normalized score.
[0036] S1.4. Dataset partitioning: The processed dataset is divided into training, validation and test sets in an 8:1:1 ratio for model training, parameter tuning and performance evaluation.
[0037] S2. Sample selection based on influence function By quantifying the impact of samples on model robustness and fairness, low-value data is eliminated. The specific process is as follows: S2.1. Calculation of Robustness Impact Value: For a training sample (u,i), its robustness impact value is measured by the change in model loss under adversarial perturbation, using the following formula:
[0038] Where V' is the adversarial validation set (generated by applying minimum perturbation to the original validation set samples). Let be the gradient of the loss function with respect to the model parameters θ. Let be the Hessian matrix of the loss function.
[0039] To simplify the calculation, the Hessian vector product (HVP) approximation is used. :
[0040] S2.2. Calculation of Fairness Impact Value: Based on the demographic parity (DP) principle, the prediction bias impact of the sample on groups with different sensitive attributes (e.g., g=0 and g=1) is quantified using the following formula:
[0041] in, This is the fairness loss function, which measures the group prediction bias.
[0042] S2.3. Comprehensive Impact Screening: A weighted fusion of robustness and fairness impact values is used to obtain the comprehensive impact score of the sample.
[0043] Where α and β are weight parameters (both default to 0.5), calculated according to... Sort the samples in ascending order, remove low-impact samples with a percentage of γ (default 3%), and obtain the filtered dataset D'.
[0044] S3. Personalized Counter-Disturbance Strategy The amplitude of the adversarial perturbation is dynamically adjusted based on user-embedded features, and the steps are as follows: S3.1. Calculation of User Embedding Norm: Let the user embedding matrix be... (where n is the number of users and d is the embedding dimension), the embedding vector of user u in the t-th training round is: Calculate the average of the embedding norms for all users:
[0045] S3.2. Determining the perturbation amplitude: The user embedding norm is standardized to user-specific coefficients using the sigmoid function, and the maximum perturbation amplitude is determined by combining this with the baseline value μ.
[0046] in, The coefficient is the standardized coefficient.
[0047] S3.3. Construction of Personalized Adversarial Loss: An adversarial loss is introduced based on the Bayesian Personalized Ranking (BPR) loss, with the following formula:
[0048] in, For the original BPR loss, For personalized perturbations (amplitude not exceeding) ), where λ is the adversarial loss weight (default 1.0).
[0049] S4. Construction of Multi-Dimensional Reward Function The steps to design a reward mechanism that integrates embedded interaction, behavioral similarity, and batch optimization are as follows: S4.1. Embedded Interaction Reward: Measures the matching degree between recommended items and user preferences, calculated through the inner product difference of positive and negative sample embeddings.
[0050] in, Embedded for users, Embedding of positive and negative sample items respectively. This is for inner product operations.
[0051] S4.2. Behavioral Similarity Reward: Based on the user's historical interaction matrix, calculate the coverage of recommended items among similar users:
[0052] in, For similar user sets ( (Similarity threshold) Indicates user With items It is interactive.
[0053] S4.3. Maximum Reward within a Batch: The maximum reward for the same user sample within the mini-batch is taken to avoid suboptimal solutions.
[0054] in, This is the sample set of user u in the current batch.
[0055] S4.4. Total Reward Fusion: Weighted merging of the three reward categories, using the following formula:
[0056] Where α is the weight parameter for the maximum reward in a batch (default 0.5).
[0057] S5. Model Training and Optimization The steps for training the model, combining sample selection, personalized perturbation, and dynamic reward, are as follows: S5.1. Definition of the Maximum Reward Function: Based on the Extended Markov Decision Process (MDP), the maximum reward value function is defined.
[0058] Where γ is the discount factor (default 0.9).
[0059] The value function satisfies the Bellman equation:
[0060] Where s represents the state, containing user characteristics and historical best reward auxiliary variables. 'a' represents the recommended action, and 'V' represents the operation of taking the maximum value.
[0061] S5.2. Construction of the total loss function: This function integrates the BPR loss, personalized adversarial loss, and maximum reward loss. The formula is as follows:
[0062] in, This represents the maximum reward loss.
[0063] S5.3. Parameter Optimization: The model parameters are updated using the policy gradient method. The gradient formula is:
[0064] in, For the discounted steady-state distribution of the extended state, The action value function is trained iteratively until the loss converges. Example 1:
[0065] In clean data scenarios, compared with basic models (such as MF and LightGCN) and existing defense methods (such as APR and PamaCF), the present invention improves the recommendation performance metrics (Recall@10, NDCG@10) by an average of 19.92%, demonstrating its optimization effect on recommendation quality. Example 2:
[0066] In various attack scenarios (such as Random Attack and CleaR Attack), this invention reduces the attack success rate (T-HR@50) by an average of 20.50%, with particularly significant defensive effects against less common items (T-HR@50 reduced to a minimum of 0.00424), verifying its effectiveness in enhancing system robustness.
[0067] The performance results of the present invention in recommender systems under different base models on multiple datasets, in attack performance under different base models on multiple datasets, in recommender systems under different attack scenarios on multiple datasets, and in attack performance under different base models on multiple datasets are shown in the following figures. Figure 2-5 As shown.
[0068] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.
Claims
1. A personalized robust recommendation defense method based on a dynamic reward mechanism, characterized in that, Includes the following steps: S1. Data Acquisition and Preprocessing: Obtain real-world datasets related to the recommendation system, clean the datasets, filter out invalid interaction data, retain valid user-item interaction pairs, normalize the rating data, and divide them into training, validation and test sets. S2. Sample selection based on influence function: A robustness and fairness dual-objective impact assessment mechanism is constructed to quantify the impact of each user-item interaction sample on the model; S3. Personalized Counter-Perturbation Strategy Design: Dynamically adjust the magnitude of counter-disturbance based on different user behavior characteristics; S4. Construction of a multi-dimensional reward function: The design incorporates a reward mechanism that integrates embedded interaction, behavioral similarity, and batch optimization to guide the model to dynamically balance defense and recommendation quality. S5. Model Training and Optimization: The model is trained based on the filtered dataset, combining personalized adversarial perturbations and multi-dimensional reward functions.
2. The personalized robust recommendation defense method based on a dynamic reward mechanism according to claim 1, characterized in that, Step S1 specifically includes the following steps: S1.
1. Data Acquisition: Collect real-world datasets related to the recommendation system, including user-item interaction records, covering 796-2848 users, 1682-39586 items, and 30,000-890,000 interaction data points; S1.
2. Data cleaning: Filter out invalid samples, including abnormal ratings, duplicate interaction records and cold start users, and retain valid user-item interaction pairs (u,i); S1.
3. Data Normalization: The minimum-maximum normalization method is used to map the scoring data to the [0,1] interval, as shown in the following formula: in, The original rating given by user u to item i. These are the minimum and maximum scores in the dataset, respectively. The score is the normalized score; S1.
4. Dataset Partitioning: The processed dataset is divided into training, validation, and test sets in an 8:1:1 ratio.
3. The personalized robust recommendation defense method based on a dynamic reward mechanism according to claim 1, characterized in that, Step S2 specifically includes the following steps: S2.
1. Calculation of Robustness Impact Value: For a training sample (u,i), its robustness impact value is measured by the change in model loss under adversarial perturbation, using the following formula: Where V' is the adversarial verification set. Let be the gradient of the loss function with respect to the model parameters θ. Let be the Hessian matrix of the loss function; To simplify the calculation, the Hessian vector product (HVP) approximation is used. : ； S2.
2. Calculation of Fairness Impact Value: Based on the demographic parity (DP) principle, the impact of the sample on the prediction bias of different sensitive attribute groups is quantified. The formula is as follows: in, The fairness loss function measures the group prediction bias. S2.
3. Comprehensive Impact Screening: A weighted fusion of robustness and fairness impact values is used to obtain the comprehensive impact score of the sample. Where α and β are weight parameters, according to Sort the samples in ascending order and remove low-impact samples with a percentage of γ, where γ defaults to 3%, to obtain the filtered dataset D'.
4. The personalized robust recommendation defense method based on a dynamic reward mechanism according to claim 1, characterized in that, Step S3 specifically includes the following steps: S3.
1. Calculation of User Embedding Norm: Let the user embedding matrix be... Where n is the number of users, d is the embedding dimension, and the embedding vector of user u in the t-th training round is... Calculate the average of the embedding norms for all users: ； S3.
2. Determining the perturbation amplitude: The user embedding norm is standardized to user-specific coefficients using the sigmoid function, and the maximum perturbation amplitude is determined by combining this with the baseline value μ. in, These are the standardized coefficients; S3.
3. Construction of Personalized Adversarial Loss: Based on the Bayesian Personalized Ranking (BPR) loss, an adversarial loss is introduced, with the following formula: in, For the original BPR loss, For personalized perturbations, the amplitude shall not exceed λ is the adversarial loss weight, which defaults to 1.
0.
5. A personalized robust recommendation defense method based on a dynamic reward mechanism according to claim 1, characterized in that, Step S4 specifically includes the following steps: S4.
1. Embedded Interaction Reward: Measures the matching degree between recommended items and user preferences, calculated through the inner product difference of positive and negative sample embeddings. in, Embedded for users, Embedding of positive and negative sample items respectively. This is an inner product operation; S4.
2. Behavioral Similarity Reward: Based on the user's historical interaction matrix, calculate the coverage of recommended items among similar users: in, For a set of similar users, For similarity threshold, Indicates user With items It is interactive; S4.
3. Maximum Reward within a Batch: The maximum reward for the same user sample within the mini-batch is taken to avoid suboptimal solutions. in, Let be the sample set of user u in the current batch; S4.
4. Total Reward Fusion: Weighted merging of the three reward categories, using the following formula: Where α is the weight parameter for the maximum reward in a batch, which defaults to 0.
5.
6. The personalized robust recommendation defense method based on a dynamic reward mechanism according to claim 1, characterized in that, Step S5 specifically includes the following steps: S5.
1. Definition of Maximum Reward Value Function: Based on the Extended Markov Decision Process (MDP), the maximum reward value function is defined as follows: Where γ is the discount factor, which defaults to 0.9; The value function satisfies the Bellman equation: Where s represents the state, containing user characteristics and historical best reward auxiliary variables. 'a' represents the recommended action, and 'ν' represents the maximum value operation. S5.
2. Construction of the total loss function: This function integrates the BPR loss, personalized adversarial loss, and maximum reward loss. The formula is as follows: in, To maximize the reward loss; S5.
3. Parameter Optimization: The model parameters are updated using the policy gradient method. The gradient formula is: in, For the discounted steady-state distribution of the extended state, The action value function is trained iteratively until the loss converges.