An anti-reward-hacker recommendation system optimization method and anti-reward-hacker recommendation system

Abstract

The application discloses an anti-reward hacker recommendation system optimization method and an anti-reward hacker recommendation system, and the method comprises the following steps: a dynamic interval mechanism is constructed, a dynamic interval item is calculated based on a user historical sequence and a positive and negative sample pair, and the dynamic interval item is used for quantifying the preference difference intensity between the positive and negative samples; the dynamic interval item is subjected to cross-sample statistical normalization to obtain a normalized interval item; the dynamic interval is decoupled into an optimization constraint item through a stop gradient operation, model training is carried out in combination with a preference optimization loss; a negative log-likelihood loss is introduced, and the generation probability of the positive sample is directly maximized; the preference optimization loss and the negative log-likelihood loss are jointly optimized to obtain a total loss function, which is used for inhibiting the reward hacker behavior in the recommendation system. The application dynamically adjusts the interval threshold based on the user behavior intensity, ensures that the model differentiates modeling of the fine-grained preference and the coarse-grained demand of the final positive and negative samples in the recommendation sequence, and alleviates the recommendation deviation caused by data sparseness and diversity.

Description

Technical Field

[0001] The present application relates to the technical field of recommendation systems, and more specifically to an optimization method for an anti-reward hacker recommendation system and an anti-reward hacker recommendation system. Background Art

[0002] In recommendation systems based on large language models, reward-driven preference optimization is a new paradigm for improving recommendation effectiveness. This approach drives model iteration by maximizing the difference in predicted scores between explicit user feedback (such as likes and favorites) and unfeeded user behavior, theoretically achieving precise alignment with user preferences. However, the reward hacking problem exposed by this paradigm in practice has become a core bottleneck restricting its application: due to the inevitable modeling deviation between reward function design and actual user intent, the model may improve metrics by systematically exploiting vulnerabilities in the reward mechanism rather than deeply understanding user needs, ultimately causing recommendation results to deviate from the true distribution of user interests.

[0003] Current mainstream preference optimization methods can be divided into two categories: reference model-based methods and no-reference model methods. Reference model-based methods rely on a pre-trained reference model and a policy model to jointly assist in predicting the value of the reward function. The reference model constrains the stability of the optimization process through the Kullback-Leibler divergence. Essentially, they transform the human preference optimization problem from a reinforcement learning paradigm to a supervised learning paradigm. No-reference model methods, on the other hand, abandon the reference model design and directly optimize using the likelihood difference between positive and negative samples. They also introduce a fixed boundary constant in the loss function to force the model to distinguish between positive and negative samples.

[0004] Although the above methods perform well in the basic preference alignment task, their inherent defects still lead to two typical reward hacking patterns:

[0005] 1. Inadequate adaptation to user data diversity. Existing methods struggle to dynamically distinguish between preferences of varying importance in user behavior: models overly focus on easily confounded, fine-grained preferences in the training data (e.g., the choice of "milk" flavor), while failing to distinguish between significantly different scenarios (e.g., movie recommendations across different categories). This bias causes the model's recommendations for significantly different scenarios to deviate from actual needs.

[0006] 2. Optimization mechanisms that over-rely on negative sample comparisons. Existing methods improve metrics by forcibly increasing the difference between the prediction scores of positive and negative samples, while neglecting to effectively learn the characteristics of positive samples. Models may artificially inflate click-through rates by indiscriminately lowering the prediction scores of unclicked items (negative samples), rather than deeply exploring the core characteristics of clicked items (positive samples), such as quality and category. This strategy can lead models into a "metric trap"—ostensibly improving item discrimination, but actual recommendations deviate from the true preference distribution. Summary of the Invention

[0007] In this embodiment, a method, system, electronic device, and storage medium for optimizing a recommendation system against reward hacking are provided to address the problems in related technologies of insufficient adaptation to the diversity of user data and over-reliance on negative sample comparison optimization mechanisms.

[0008] In a first aspect, an embodiment of the present invention provides a method for optimizing a reward-resistant hacker recommendation system, the method comprising:

[0009] Build a dynamic interval mechanism based on user history sequence and positive and negative sample pairs Calculate dynamic interval terms , used to quantify the strength of the preference difference between positive and negative samples;

[0010] Perform cross-sample statistical normalization on the dynamic interval term to obtain the normalized interval term ; Decouple the dynamic interval into an optimization constraint by stopping the gradient operation and combine it with the preference optimization loss Conduct model training;

[0011] Introducing negative log-likelihood loss , directly maximize the generation probability of positive samples;

[0012] Joint optimization preference optimization loss and negative log-likelihood loss , and get the total loss function , for suppressing reward hacking in recommender systems.

[0013] In an optional embodiment, the calculation formula of the dynamic interval term is:

[0014] ;

[0015] in, The reward function for direct preference optimization is defined as:

[0016] ;

[0017] in, and Represents the policy model and reference model given user history sequence and any item The probability of represents an unknown partition function, is a hyperparameter.

[0018] In an optional embodiment, the calculation formula of the normalized interval term is:

[0019] ;

[0020] in and They are the dynamic interval items on the full training data The mean and standard deviation of .

[0021] In an optional embodiment, the expression of the preference optimization loss is:

[0022] ;

[0023] in, Indicates in the dataset expectations, is the Sigmoid function, and is a hyperparameter, Indicates stopping the gradient operation.

[0024] In an optional embodiment, the expression of the negative log-likelihood loss is:

[0025] ;

[0026] in, Indicates in the dataset expectations on.

[0027] In an optional embodiment, the expression of the total loss function is:

[0028] ;

[0029] in, is the weight coefficient for balancing the two losses.

[0030] In an optional embodiment, the method further includes initializing using a pre-trained universal language model as a base model;

[0031] Based on the processed preference data pairs, the model parameters are iteratively optimized through dynamic interval constraints and a dual-objective loss function.

[0032] Evaluate the optimized model and adjust the hyperparameters based on the evaluation results to further optimize the model performance.

[0033] Compared with the prior art, the beneficial effects of the anti-reward hacker recommendation system optimization method of the present invention are as follows:

[0034] This invention dynamically adjusts the interval threshold based on the intensity of user behavior, ensuring that the model differentially models the fine-grained preferences (such as brand differences) and coarse-grained demands (such as category selection) of the final positive and negative samples in the recommendation sequence, mitigating recommendation biases caused by data sparsity and diversity. This mechanism achieves parameter adaptation through end-to-end gradient backpropagation, eliminating the need for manual static threshold setting. By jointly optimizing the negative log-likelihood loss and contrastive loss, the model is forced to simultaneously capture core attributes of positive samples, such as product quality and user reviews, as well as the differential expression of negative samples, avoiding the "metric trap" caused by relying solely on negative sample comparisons.

[0035] In a second aspect, an embodiment of the present invention provides an anti-reward hacker recommendation system, comprising:

[0036] Data preprocessing module, used to collect user historical interaction sequences and sample them into preference data pairs;

[0037] A dynamic interval calculation module, configured to calculate the dynamic interval item according to the method described in the first aspect;

[0038] Loss function optimization module, used to jointly optimize preference optimization loss and negative log-likelihood loss;

[0039] The model deployment module is used to deploy the optimized model to the recommendation system server and adjust the recommendation results in real time.

[0040] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor, a communication interface, a memory and a bus, wherein the processor, the communication interface and the memory communicate with each other through the bus, and the processor can call logic instructions in the memory to execute the steps of the method provided in the first aspect.

[0041] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the anti-reward hacker recommendation system optimization method as described in the first aspect are implemented.

[0042] Compared with the prior art, the beneficial effects of the anti-reward hacker recommendation system, electronic device, and storage medium of the present invention are the same as those of the anti-reward hacker recommendation system optimization method described in the first aspect, so they will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 This is a flow chart of a method for optimizing a reward-resistant hacker recommendation system according to an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of the EqPO method in an embodiment of the present invention;

[0046] Figure 3 This is a structural block diagram of the anti-reward hacker recommendation system in an embodiment of the present invention;

[0047] Figure 4 2 is a structural block diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of this application, this application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] In sequence recommendations based on large language models, the reward hacking problem urgently needs to solve the challenges of difficulty bias optimization and negative sample utilization. Although the introduction of auxiliary discriminant models can alleviate some problems, its additional training cost and multi-model coupling risks lead to low efficiency in actual deployment. Therefore, this paper proposes a new balanced preference optimization paradigm (which can be understood as the EqPO method) that can uniformly address the above challenges under a single model framework. We further designed a dynamic margin calibration and dual-loss collaborative mechanism to overcome the trade-off dilemma of traditional methods between stability and generalization. The core innovation lies in modeling sample differences as learnable dynamic margin constraints and incorporating explicit likelihood reinforcement into the comparative optimization objective, such as Figure 2 As shown in the figure, the technical framework of EqPO will be elaborated in detail below.

[0051] In an embodiment of the present invention, an anti-reward hacker recommendation system optimization method (EqPO) is provided. Figure 1 This is a flowchart of the anti-reward hacker recommendation system optimization method of the present invention, combined with Figure 1 As shown, the process includes the following steps:

[0052] S100, build a dynamic interval mechanism based on user history sequence and positive and negative sample pairs Calculate dynamic interval terms , used to quantify the strength of the preference difference between positive and negative samples;

[0053] It should be noted that in sequential recommendation based on large language models, traditional fixed interval constraints cannot adapt to the intensity of preference differences between different user-item interaction pairs. Such differences can be naturally modeled as the implicit reward space offset between the policy model and the reference model. Specifically, for the user history sequence Positive and negative sample pairs , the dynamic interval term is defined as the implicit reward difference between the strategy model and the reference model in the direct preference optimization method DPO. The calculation formula of the dynamic interval term is:

[0054] ;

[0055] in, The reward function for direct preference optimization is defined as:

[0056] ;

[0057] in, and Represents the policy model and reference model given user history sequence and any item The probability of represents an unknown partition function, is a hyperparameter.

[0058] S200, perform cross-sample statistical normalization on the dynamic interval term to obtain the normalized interval term ; Decouple the dynamic interval into an optimization constraint by stopping the gradient operation and combine it with the preference optimization loss Conduct model training;

[0059] In order to eliminate the difference in interval dimensions and enhance training stability, statistical normalization across samples is implemented. The calculation formula for the normalized interval term is:

[0060] ;

[0061] in and They are the dynamic interval items on the full training data The mean and standard deviation of .

[0062] To prevent the policy model from overfitting the reward difference calculation, a stop gradient operation is used Decouple the dynamic interval into an optimization constraint. The expression of the preference optimization loss is:

[0063] ;

[0064] in, Indicates in the dataset expectations, is the Sigmoid function, and is a hyperparameter, Indicates stopping the gradient operation.

[0065] The above dynamic interval term quantifies the inherent differences in preference pairs during policy iteration: when When it is large (such as "down jacket vs. short-sleeved T-shirt"), it indicates that there are significant differences between samples; conversely, a small value (such as "latte vs. cappuccino") corresponds to a fine-grained preference scenario.

[0066] S300, introduce negative log-likelihood loss , directly maximize the generation probability of positive samples;

[0067] Traditional preference alignment methods overly rely on implicit contrast signals between positive and negative samples, leading the model to fall into the trap of "reward difference shortcuts"—improving metrics by systematically suppressing the prediction scores of negative samples while neglecting to learn the core features of positive samples. To address this problem, an explicit negative likelihood loss function is introduced to directly maximize the probability of generating positive samples: It should be noted that the expression of negative log-likelihood loss is:

[0068] ;

[0069] This loss term forces the model to independently optimize the absolute probability distribution of positive samples in addition to implicit contrastive learning.

[0070] S400, Joint Optimization Preference Optimization Loss and negative log-likelihood loss , and get the total loss function , for suppressing reward hacking in recommender systems.

[0071] Finally, the expression of the total loss function is:

[0072] ;

[0073] in, Indicates in the dataset expectations on.

[0074] in, is the weight coefficient for balancing the two losses.

[0075] Overall, the EqPO method effectively suppresses reward hacking in recommendation systems through dynamic interval calibration and negative likelihood loss reinforcement mechanism without relying on auxiliary discriminant models.

[0076] It also includes,initializing using a pre-trained general language model as the base model;

[0077] Based on the processed preference data pairs, the model parameters are iteratively optimized through dynamic interval constraints and a dual-objective loss function.

[0078] Evaluate the optimized model and adjust the hyperparameters based on the evaluation results to further optimize the model performance.

[0079] Specifically, historical interaction sequences of users on online platforms are collected and prepared, and records containing obvious errors and logical inconsistencies are removed to ensure data quality. The data are then sampled into preference data pairs. A pre-trained general language model is used as the base model for initialization. Based on the processed preference data pairs, the pre-trained model parameters are iteratively optimized using dynamic interval constraints and a dual-objective loss function. The updated model is evaluated to check the prediction accuracy and model adaptability. Hyperparameters are adjusted based on the evaluation results to further optimize model performance. The optimized EqPO model is deployed to the recommendation system server, and the dynamic interval threshold and loss weight coefficient are dynamically adjusted by collecting user behavior data in real time. The model can adaptively distinguish fine-grained preference differences based on the latest interaction sequences, while simultaneously suppressing reward hacking behavior, significantly improving the real-time performance of recommendation results and user satisfaction.

[0080] The EqPO method proposed in this paper can be widely applied to language model-driven recommendation system optimization tasks across multiple domains. In practical applications, EqPO effectively suppresses reward hacking and improves recommendation quality through a dynamic spacing mechanism and a dual-objective joint optimization strategy. For example, on e-commerce platforms, merchants can use EqPO to dynamically adjust the optimization boundaries between positive and negative samples, ensuring that the model differentially models fine-grained preferences (such as brand differences) and coarse-grained requirements (such as category selection) for the final positive and negative samples in the recommendation sequence, mitigating recommendation bias caused by data sparsity and diversity. In the short video recommendation scenario, EqPO explicitly emphasizes the core features of content with high completion rates (such as thematic depth and creator credibility) through a negative likelihood loss, preventing the model from cheating by suppressing irrelevant content. These application examples demonstrate EqPO's core advantage in balancing recommendation accuracy and combating reward hacking, providing a scalable technical path for real-time optimization of recommendation systems.

[0081] The EqPO method proposed in this paper demonstrates significant advantages in addressing the reward hacking problem in recommendation systems. First, through a dynamic spacing mechanism and a dual-objective joint optimization strategy, EqPO achieves a balanced modeling of fine-grained user preferences and significantly different scenarios. This effectively suppresses the optimization shortcuts that suppress negative examples without introducing additional discriminant models. Compared to traditional methods, EqPO not only significantly improves the authenticity of recommendation results, but also maintains the same level of training efficiency as mainstream single-model methods.

[0082] Experimental results (as shown in Table 1 below) show that EqPO outperforms existing state-of-the-art methods when processing the three datasets of LastFM, Goodreads, and Steam. Specifically, on the LastFM dataset, EqPO showed significant improvement in the HitRatio@1 metric compared to the baseline model. When the base model was Llama2-7b, EqPO's HitRatio@1 metric reached 0.6669, an increase of 4.32% over the DPO baseline and 11.37% over SimPO; on the Llama3-8B-Instruct model, EqPO's HitRatio@1 metric reached 0.7130, an increase of 3.18% over the DPO baseline and 7.82% over SimPO. This improvement verifies EqPO's adaptability to diverse user preferences while also achieving better accuracy.

[0083] Further experimental results show that IF-DF significantly outperforms other control models on the Goodreads and Steam datasets, with improvements in the HitRatio@1 metric. Compared to S-DPO, which requires additional negative samples and incurs twice the computational overhead, EqPO uses only a single negative sample, effectively mitigating the reward hacking problem and improving prediction performance.

[0084] EqPO demonstrates greater robustness in suppressing reward hacking. Regarding the second type of reward hacking (where the model inflates metrics by indiscriminately suppressing negative sample scores), EqPO reduced the incidence of this type of cheating on the LastFM dataset from 26.17% in S-DPO to 15.51%, a 41% reduction. This result demonstrates the negative likelihood loss's ability to enhance the core features of positive samples and the collaborative error-correcting capabilities of dual-objective optimization.

[0085] In summary, EqPO innovatively integrates dynamic margin constraints and explicit likelihood reinforcement to construct an efficient and robust recommendation optimization paradigm that is resistant to reward hacking. Its combined advantages in improving recommendation quality, inhibiting cheating, and reducing computational costs provide a practical solution for large-scale language model-driven real-time recommendation systems, with broad prospects for industrial application.

[0086] Table 1: HitRatio@1 results of different methods on the Llama-2-7b base on three datasets

[0087]

[0088] The present invention also provides an anti-reward hacker recommendation system for implementing the aforementioned method embodiments. Details already described are omitted for clarity. The terms "module," "unit," "subunit," and the like used below may refer to a combination of software and / or hardware that implements a predetermined function. While the systems described in the following embodiments are preferably implemented in software, implementation in hardware or a combination of software and hardware is also possible and contemplated.

[0089] like Figure 3 As shown, Figure 3 This is a structural block diagram of the anti-reward hacker recommendation system of the present invention, which includes:

[0090] Data preprocessing module 101, used to collect user historical interaction sequences and sample them into preference data pairs;

[0091] A dynamic interval calculation module 102 is used to calculate the dynamic interval term according to the above-mentioned anti-reward hacker recommendation system optimization method;

[0092] a loss function optimization module 103 for jointly optimizing the preference optimization loss and the negative log-likelihood loss;

[0093] The model deployment module 104 is used to deploy the optimized model to the recommendation system server and adjust the recommendation results in real time.

[0094] The application of the reward-hacking-resistant recommendation system is largely similar to the above method. Specifically, historical user interaction sequences on an online platform are collected and prepared, and records containing obvious errors and logical inconsistencies are removed to ensure data quality. The data is then sampled into preference data pairs. A pretrained general language model is used as the base model for initialization. Based on the processed preference data pairs, the pretrained model parameters are iteratively optimized using dynamic interval constraints and a dual-objective loss function. The updated model is evaluated to check prediction accuracy and model adaptability. Hyperparameters are adjusted based on the evaluation results to further optimize model performance. The optimized EqPO model is deployed to the recommendation system server, and the dynamic interval threshold and loss weight coefficient are dynamically adjusted by collecting real-time user behavior data. The model can adaptively distinguish fine-grained preference differences based on the latest interaction sequences, while simultaneously suppressing reward-hacking behavior, significantly improving the real-time performance of recommendation results and user satisfaction.

[0095] Figure 4 A structural block diagram of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the following method:

[0096] Build a dynamic interval mechanism based on user history sequence and positive and negative sample pairs Calculate dynamic interval terms , used to quantify the strength of the preference difference between positive and negative samples;

[0097] Perform cross-sample statistical normalization on the dynamic interval term to obtain the normalized interval term ; Decouple the dynamic interval into an optimization constraint by stopping the gradient operation and combine it with the preference optimization loss Conduct model training;

[0098] Introducing negative log-likelihood loss , directly maximize the generation probability of positive samples;

[0099] Joint optimization preference optimization loss and negative log-likelihood loss , and get the total loss function , for suppressing reward hacking in recommender systems.

[0100] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0101] An embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in the above embodiments is implemented.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for optimizing a reward-resistant hacker recommendation system, characterized in that: include: Build a dynamic interval mechanism based on user history sequence and positive and negative sample pairs Calculate dynamic interval terms , used to quantify the strength of the preference difference between positive and negative samples; Perform cross-sample statistical normalization on the dynamic interval term to obtain the normalized interval term ; Decouple the dynamic interval into an optimization constraint by stopping the gradient operation and combine it with the preference optimization loss Conduct model training; Introducing negative log-likelihood loss , directly maximize the generation probability of positive samples; Joint optimization preference optimization loss and negative log-likelihood loss , and get the total loss function , used to curb reward hacking in recommender systems; The expression of the preference optimization loss is: ； in, Indicates in the dataset expectations, is the Sigmoid function, and is a hyperparameter, Indicates stopping the gradient operation; The expression of the negative log-likelihood loss is: ； in, Indicates in the dataset expectations on The expression of the total loss function is: ； in, is the weight coefficient for balancing the two losses.

2. The method for optimizing the anti-reward hacker recommendation system according to claim 1, characterized in that: The calculation formula of the dynamic interval term is: ； in, The reward function for direct preference optimization is defined as: ； in, and Represents the policy model and reference model given user history sequence and any item The probability of represents an unknown partition function, is a hyperparameter.

3. The method for optimizing the anti-reward hacker recommendation system according to claim 2, characterized in that: The calculation formula of the normalized interval term is: ； in and They are the dynamic interval items on the full training data The mean and standard deviation of .

4. The method for optimizing the anti-reward hacker recommendation system according to claim 1, characterized in that: It also includes,initializing using a pre-trained general language model as the base model; Based on the processed preference data pairs, the model parameters are iteratively optimized through dynamic interval constraints and a dual-objective loss function. Evaluate the optimized model and adjust the hyperparameters based on the evaluation results to further optimize the model performance.

5. An anti-reward hacker recommendation system, characterized in that: include: Data preprocessing module, used to collect user historical interaction sequences and sample them into preference data pairs; A dynamic interval calculation module, configured to calculate a dynamic interval item according to any one of the methods of claims 1 to 4; Loss function optimization module, used to jointly optimize preference optimization loss and negative log-likelihood loss; The model deployment module is used to deploy the optimized model to the recommendation system server and adjust the recommendation results in real time.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the anti-reward hacker recommendation system optimization method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the anti-reward hacker recommendation system optimization method according to any one of claims 1 to 4 are implemented.

Images