A data-free recommendation modeling method and device for recommendation system evaluation

Abstract

This invention discloses a method and apparatus for modeling data-free recommendation models for recommendation system evaluation. The method includes: S1: constructing a generator G to generate virtual recommendation data; S2: constructing a clone model C to learn the recommendation mechanism of the target recommendation system; S3: performing collaborative optimization to adjust the parameters of the generator G and the clone model C; S4: evaluating the surrogate model obtained in step S3 according to the evaluation scheme. This invention effectively solves the challenges of data-free recommendation system modeling by utilizing a generator-surrogate model-based collaborative optimization framework.

Description

Technical Field

[0001] This invention relates to the field of machine learning and recommender system technology, and mainly to a data-free recommender modeling method and apparatus for recommender system evaluation. Background Technology

[0002] In today's information-saturated world, recommender systems serve as a bridge between users and massive amounts of information. However, the security issues of recommender systems are becoming increasingly prominent. Attacks such as model theft, data injection, and collaborative filtering threaten system security. These attacks can not only leak core model information but also manipulate recommendation results, harming user interests. Therefore, conducting comprehensive security assessments of recommender systems is crucial. This not only helps verify the effectiveness of recommender systems in fields such as marketing and e-commerce but also ensures the system's adaptability and reliability in responding to changing user needs and privacy protection challenges.

[0003] When evaluating the security of recommender systems, the model under evaluation often cannot be used directly; instead, a proxy model needs to be constructed. However, considering issues such as data privacy and intellectual property rights, the proxy model cannot access the target model's architecture and training data, significantly limiting its learning and evaluation of the target model's recommendation mechanism. Therefore, exploring modeling methods for recommender systems in data-free scenarios is urgently needed. However, existing modeling methods typically focus on sequence recommendations in data-free scenarios or are limited to modeling scenarios where some data is available, resulting in problems such as a single research object, strong data dependence, and insufficient feasibility. The generator-proxy model joint modeling framework has been widely applied in fields such as computer vision, demonstrating excellent modeling capabilities and possessing the potential to solve this problem. Summary of the Invention

[0004] This invention aims to address the shortcomings of existing technologies by proposing a data-free recommendation modeling method and apparatus for evaluating recommendation systems.

[0005] To address the problem of traditional proxy models' over-reliance on specific datasets, this invention proposes an innovative method. This method integrates out-of-distribution data generated by a generator and introduces a query mechanism, thereby overcoming the limitation of dependence on specific datasets. To address the issue of traditional proxy models being limited to specific models, this invention employs graph neural network technology, effectively reducing the complexity required to improve the model's generalization ability. Furthermore, to address the problem of insufficient accuracy in proxy models, this invention utilizes numerical optimization techniques and alternating network optimization to ultimately construct a high-precision proxy model.

[0006] The first aspect of this invention relates to a data-free recommendation model modeling method for recommendation system evaluation, comprising the following steps:

[0007] S1: Construct generator G to generate virtual recommendation data.

[0008] S2: Construct a clone model C to learn the recommendation mechanism of the target recommendation system.

[0009] S3: Perform co-optimization, adjusting the parameters of the generator G and the clone model C.

[0010] S4: Evaluate the proxy model obtained in step S3 according to the evaluation scheme.

[0011] In step S1, a generator G is constructed to generate virtual recommendation data. Specifically, the generator is a key tool for building the proxy model. After training, it can approximate the user configuration data distribution of the target model, thereby enabling the construction of the proxy model in a data-free environment and playing a crucial role in the alternating optimization network. Specifically, the generator G receives a random noise vector without actual data input and generates an output sample through a series of nonlinear transformations:

[0012]

[0013] in, θ is a low-dimensional noise vector randomly sampled from a standard normal distribution. G The parameters represent the generator model. This indicates that the generator G transforms a random noise vector through a series of nonlinear transformations. The process of converting data into user-project interaction data, therefore, This can be viewed as a user-project bipartite graph:

[0014]

[0015] Where m represents the number of users in the bipartite graph, and n represents the number of items in the bipartite graph, these are the data generated by the generator. Input the data into model T and query it; the corresponding response will be returned.

[0016]

[0017] Where, θ T These represent the parameters of the target model. yes The result of querying the target model T is the corresponding list of recommended users.

[0018] In step S2, a clone model C is constructed to learn the recommendation mechanism of the target recommendation system T. Specifically: Because... This can be viewed as a user-item bipartite graph. Therefore, in the absence of data access, this invention uses a graph neural network (LightGCN) to construct a clone model C. The clone model C learns from the response data obtained after querying the target model T, analyzing the embedding representations of user and item nodes to capture the complex relationships between users and items. Then, the clone model uses these embedding representations to predict user interests in items, thereby generating a recommendation list.

[0019]

[0020] Where, θ C The parameters representing the clone model, yes The input to the clone model C yields the corresponding recommendation list for the generated user.

[0021] In step S3, the present invention achieves collaborative optimization of the generator model G and the clone model C through an alternating optimization network training mechanism, specifically including the following steps:

[0022] S3.1: Determine the co-optimization objective; during training, first optimize the generator. The generator's objective is to generate samples that maximize the difference from the target model's output, in order to better train the clone model C. Therefore, the output of the target model T is used. Output of clone model C The KL divergence between the two is used as the loss function of the generator, and the loss function formula is as follows:

[0023]

[0024] Here, the KL divergence measures the difference between two probability distributions. By maximizing... The KL divergence between the target and target models is used to improve the generalization ability of generator G on unseen data, enabling it to generate more realistic samples. Then, the clone model C is optimized. The goal of clone model C is to minimize the difference between its output and that of the target model, thereby improving its accuracy. To achieve the ultimate goal of clone model C matching the target model T, the loss function of clone model C in this invention is as follows:

[0025]

[0026] The loss function aims to minimize the target model output. Output of the clone model The parameters of the clone model C are optimized using the KL divergence between the generator G and the target model T, making its predictions closer to the real situation or the output of the target model T, thereby improving the accuracy and reliability of the clone. By alternately optimizing the network training mechanism, it is ensured that the generator G and the clone model C mutually promote each other during training, thus achieving higher overall performance.

[0027] S3.2: Employ numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as shown in the following formula:

[0028]

[0029] in, This represents the parameters of the generator network at time t+1. This represents the parameter value that needs to be updated, and η is the learning rate, which controls the step size of the parameter updates. This is the gradient of the generator loss function with respect to the generator parameters. To estimate the gradient in black-box attack scenarios, this invention proposes a zero-order gradient estimation method. This method approximates the gradient using the forward difference method, as shown in the following formula:

[0030]

[0031] Among them, u i Let be a random variable drawn with uniform probability from a d-dimensional unit sphere, and ∈ be a smoothing factor. Meanwhile, to reduce estimation errors, this invention employs the average stochastic gradient estimation method, calculating forward differences through m random directions and taking the average as the gradient estimate:

[0032]

[0033] in, This represents a gradient estimate or approximation of the variable x. This represents the generator loss function within m mini-batches of samples. The average of the gradient estimates with respect to parameter x. With more accurate gradient estimates, the generator parameters are also effectively updated.

[0034] In step S4, the proxy model obtained in step S3 is evaluated, specifically including the following steps:

[0035] S4.1: Fine-tune the clone model C using evaluation data; by fine-tuning the target model based on user interaction history, catastrophic forgetting during training can be effectively reduced, ensuring the proxy model can continuously and correctly classify old examples from the early training stages. Subsequently, the Bayesian loss function is used to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows:

[0036]

[0037] Where M represents the total number of users, N u It is a collection of items that user u has interacted with. and Let be the ratings of user u for items i and j, respectively. σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2 This is a regularization term used to control the model's complexity and prevent overfitting. During fine-tuning, the cloned model C is continuously updated based on new evaluation data and user feedback to ensure that the model can accurately simulate the recommendation logic of the target model.

[0038] S4.2: Evaluate the performance of clone model C and use metrics such as hit rate, fidelity, and F1 score to measure the performance of clone model C in order to evaluate the performance of clone model C; further, conduct offline evaluation and online A / B testing on the clone model obtained from modeling.

[0039] The main innovative points of this invention are as follows:

[0040] (1) By introducing off-distribution data generated by the generator and the query mechanism, the dependence on specific datasets is effectively reduced, enabling the recommender system to model in a data-free environment, thereby enhancing the applicability and flexibility of the model.

[0041] (2) By using numerical optimization techniques and alternating optimization of network structure, a high-precision surrogate modeling framework was constructed, which significantly improved the accuracy and reliability of model prediction.

[0042] The beneficial effects of this invention are as follows: Addressing the various difficulties in constructing proxy models when direct user behavior data is unavailable or sensitive data is scarce, such as data sparsity, model generalization ability, and low accuracy of the proxy model, this invention proposes a data-free recommendation model modeling method for recommendation system evaluation. This method not only provides a new approach to recommendation system modeling in data-free environments but also enhances the overall performance of recommendation systems through its innovative proxy model construction method, making it widely applicable and highly competitive in diverse application scenarios. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of the method of the present invention.

[0045] Figure 2 This is a schematic diagram of the device of the present invention. Specific implementation methods

[0046] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0047] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0048] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. While only preferred methods and materials have been described herein, any methods and materials similar or equivalent to those described herein may be used in the implementation or testing of this invention. All references to this specification are incorporated by way of citation to disclose and describe methods and / or materials associated with those references. In the event of any conflict with any incorporated reference, the content of this specification shall prevail.

[0049] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be readily apparent to those skilled in the art. This application specification and embodiments are merely exemplary.

[0050] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0051] Unless otherwise specified, the term "parts" in this invention refers to parts by weight.

[0052] Example 1

[0053] This embodiment relates to a modeling method for evaluating recommender systems when the internal workings of the target model cannot be directly accessed. Applied to a scenario using the Amazon e-commerce platform as an example, it evaluates the performance of a user-based collaborative filtering model (User CF), and includes the following steps:

[0054] S1: Construct generator G to generate virtual recommendation data.

[0055] S2: Construct a clone model C to learn the recommendation mechanism of the target recommendation system.

[0056] S3: Perform co-optimization, adjusting the parameters of the generator G and the clone model C.

[0057] S4: Evaluate the surrogate model obtained in step S3 according to the evaluation plan, and use the evaluation results of the surrogate model to replace the direct evaluation of the product model.

[0058] In step S1, the generator is a key tool for constructing the proxy model. After training, it can approximate the user configuration data distribution of the target model, including information such as user gender, age, geographical location, and purchasing power, thereby enabling the construction of the proxy model in a data-free environment. Specifically, the user feature generator G receives a random noise vector without actual data input and generates an output sample through a series of nonlinear transformations:

[0059]

[0060] in, It is a low-dimensional noise vector randomly sampled from the standard normal distribution, and θG represents the parameters of the generator model. This indicates that the generator G transforms a random noise vector through a series of nonlinear transformations. The process of converting data into user-product interaction data, therefore, This can be viewed as a user-product bipartite graph:

[0061]

[0062] Where m represents the number of users in the bipartite graph, and n represents the number of items in the bipartite graph, these are the data generated by the generator. Inputting data into model T for querying will return the corresponding response:

[0063]

[0064] Where, θ T These represent the parameters of the target model. yes The recommended list obtained after inputting into the target model T and querying it will give you the target model's predicted ranking of products that the generated user may be interested in.

[0065] In step S2, the cloned model C aims to learn and mimic the recommendation mechanism of the target recommendation system T. Because... This can be viewed as a user-item bipartite graph. Therefore, in the absence of data access, this invention uses a graph neural network (LightGCN) to construct the clone model C. The clone model C learns from the response data obtained after querying the target model T, analyzing the embedding representations of user and item nodes to capture the complex relationships between users and items. Then, the clone model uses these embedding representations to predict user interest in items, thereby generating a recommendation list.

[0066]

[0067] Where, θ C The parameters representing the clone model, Cloning model C will yield the corresponding output list. This will give you the clone model's predicted ranking of products that the user might be interested in.

[0068] In step S3, the present invention achieves collaborative optimization of the generator model G and the clone model C through an alternating optimization network training mechanism, specifically including the following steps:

[0069] S3.1: Determine the co-optimization objective; during training, first optimize the generator. The generator's objective is to generate samples that maximize the difference from the target model's output, in order to better train the clone model C. Therefore, the output of the target model T is used. Output of clone model C The KL divergence between the two is used as the loss function of the generator, and the loss function formula is as follows:

[0070]

[0071] Here, the KL divergence measures the difference between two probability distributions. By maximizing... The KL divergence between the target and target models is used to improve the generalization ability of generator G on unseen data, enabling it to generate more realistic samples. Then, the clone model C is optimized. The goal of clone model C is to minimize the difference between its output and that of the target model, thereby improving its accuracy. To achieve the ultimate goal of clone model C matching the target model T, the loss function of clone model C in this invention is as follows:

[0072]

[0073] The loss function aims to minimize the target model output. Output of the clone model The parameters of the clone model C are optimized using the KL divergence between the generator G and the target model T, making its predictions closer to the real situation or the output of the target model T, thereby improving the accuracy and reliability of the clone. By alternately optimizing the network training mechanism, it is ensured that the generator G and the clone model C mutually promote each other during training, thus achieving higher overall performance.

[0074] S3.2: Employ numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as shown in the following formula:

[0075]

[0076] in, This represents the parameters of the generator network at time t+1. This represents the parameter value that needs to be updated, and η is the learning rate, which controls the step size of the parameter updates. This is the gradient of the generator loss function with respect to the generator parameters. To estimate the gradient in black-box attack scenarios, this invention proposes a zero-order gradient estimation method. This method approximates the gradient using the forward difference method, as shown in the following formula:

[0077]

[0078] Among them, u i Let be a random variable drawn with uniform probability from a d-dimensional unit sphere, and ∈ be a smoothing factor. Meanwhile, to reduce estimation errors, this invention employs the average stochastic gradient estimation method, calculating forward differences through m random directions and taking the average as the gradient estimate:

[0079]

[0080] in, This represents a gradient estimate or approximation of the variable x. This represents the generator loss function within m mini-batches of samples. The average of the gradient estimates with respect to parameter x. With more accurate gradient estimates, the generator parameters are also effectively updated.

[0081] In step S4, the present invention needs to evaluate and optimize the proxy model obtained in step S3, design an evaluation scheme, and use the evaluation results of the proxy model to replace the direct evaluation of the product model. Specifically, it includes the following steps:

[0082] S4.1: Fine-tune the clone model C using evaluation data; by fine-tuning the target model based on user interaction history, catastrophic forgetting during training can be effectively reduced, ensuring the proxy model can continuously and correctly classify old examples from the early training stages. Subsequently, the Bayesian loss function is used to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows:

[0083]

[0084] Where M represents the total number of users, N u It is a collection of products that user u has interacted with. and Let be the ratings of user u for products i and j, respectively. σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2 This is a regularization term used to control the model's complexity and prevent overfitting. During fine-tuning, the cloned model C is continuously updated based on new evaluation data and user feedback to ensure that the model can accurately simulate the recommendation logic of the target model.

[0085] S4.2: Based on the evaluation scheme, perform performance evaluation on the cloned model C; in this embodiment, a batch of known datasets will be used. The results are input into the target model and the surrogate model respectively to obtain their respective recommendation results. and To verify the modeling effectiveness of the cloned model, hit rate and fidelity were used for evaluation. First, to verify the model's performance in recommending products of interest to users, a hit rate evaluation was conducted. Hit rate refers to the list of products that the user actually expressed interest in from the product list recommended by the model. The proportion, of which the hit rate of the target model is calculated using the following formula:

[0086]

[0087] in, This indicates the number of items the user is interested in from the list of recommended items in the target model. This represents the total number of products recommended by the target model. Similarly, the hit rate of the clone model is calculated using the following formula:

[0088]

[0089] in, This represents the number of items the user is interested in from the list of products recommended by the agent model. This represents the total number of items recommended by the proxy model. Experimental results show that the target recommendation model has a hit rate of 93%, while the proxy model has a hit rate of 90%, indicating that the target recommendation model is slightly better in terms of accuracy. However, the proxy model's performance is quite close, showing a high hit rate. To evaluate the fitting performance of the proxy model, fidelity was chosen as the evaluation metric. Fidelity measures the similarity between the item list generated by the proxy model and the item list generated by the target model, and its calculation formula is as follows:

[0090]

[0091] in, This represents the intersection of the recommendation lists from the target model and the proxy model, i.e., the number of items recommended by both models. This represents the total number of items recommended by the target model. Experimental deployment revealed that the proxy model achieved a fidelity of 91.5%, demonstrating its excellent ability to mimic the behavior and output of the target recommendation model. It can accurately replicate the target model's recommendation logic, thus enabling an indirect evaluation of the target model's performance.

[0092] S4.3: First, compare the evaluation results of the surrogate model with the actual product recommendation results to analyze its recommendation effect, accuracy, and degree of matching with real data. Next, further determine whether the surrogate model can continuously provide valuable feedback, thereby ensuring its effectiveness and reliability in practical applications. This process not only verifies the predictive ability of the surrogate model but also helps to identify potential problems in the product recommendation model in a timely manner, thus optimizing the overall performance of the recommendation system. In summary, this invention proposes a surrogate model evaluation method for recommendation systems suitable for scenarios where no data is available. Under conditions of data scarcity, it can construct and evaluate the target model with high accuracy. Example 2

[0093] This embodiment provides a data-free recommendation model modeling method for recommendation system evaluation, applied to scenarios involving the evaluation of product models and subsequent product recommendations. It includes the following steps:

[0094] S1: Construct generator G to generate virtual recommendation data.

[0095] S2: Construct a clone model C to learn the recommendation mechanism of the target recommendation system.

[0096] S3: Perform co-optimization, adjusting the parameters of the generator G and the clone model C.

[0097] S4: Evaluate the surrogate model obtained in step S3 according to the evaluation plan, and use the evaluation results of the surrogate model to replace the direct evaluation of the product model.

[0098] S5: Monitor the target recommendation model based on the evaluation results for use in actual product recommendations.

[0099] In step S1, the generator is a key tool for constructing the proxy model. After training, it can approximate the user configuration data distribution of the target model, including information such as user gender, age, geographical location, and purchasing power, thereby enabling the construction of the proxy model in a data-free environment. Specifically, the user feature generator G receives a random noise vector without actual data input and generates an output sample through a series of nonlinear transformations:

[0100]

[0101] in, θ is a low-dimensional noise vector randomly sampled from a standard normal distribution. G The parameters represent the generator model. This indicates that the generator G transforms a random noise vector through a series of nonlinear transformations. The process of converting data into user-product interaction data, therefore, This can be viewed as a user-product bipartite graph:

[0102]

[0103] Where m represents the number of users in the bipartite graph, and n represents the number of items in the bipartite graph, these are the data generated by the generator. Inputting data into model T for querying will return the corresponding response:

[0104]

[0105] Where, θ T These represent the parameters of the target model. yes The recommended list obtained after inputting into the target model T and querying it will give you the target model's predicted ranking of products that the generated user may be interested in.

[0106] In step S2, the cloned model C aims to learn and mimic the recommendation mechanism of the target recommendation system T. Because... This can be viewed as a user-item bipartite graph. Therefore, in the absence of data access, this invention uses a graph neural network (LightGCN) to construct the clone model C. The clone model C learns from the response data obtained after querying the target model T, analyzing the embedding representations of user and item nodes to capture the complex relationships between users and items. Then, the clone model uses these embedding representations to predict user interest in items, thereby generating a recommendation list.

[0107]

[0108] Where, θ C The parameters representing the clone model, Cloning model C will yield the corresponding output list. This will give you the clone model's predicted ranking of products that the user might be interested in.

[0109] In step S3, the present invention achieves collaborative optimization of the generator model G and the clone model C through an alternating optimization network training mechanism, specifically including the following steps:

[0110] S3.1: Determine the co-optimization objective; during training, first optimize the generator. The generator's objective is to generate samples that maximize the difference from the target model's output, in order to better train the clone model C. Therefore, the output of the target model T is used. Output of clone model C The KL divergence between the two is used as the loss function of the generator, and the loss function formula is as follows:

[0111]

[0112] Here, the KL divergence measures the difference between two probability distributions. By maximizing... The KL divergence between the target and target models is used to improve the generalization ability of generator G on unseen data, enabling it to generate more realistic samples. Then, the clone model C is optimized. The goal of clone model C is to minimize the difference between its output and that of the target model, thereby improving its accuracy. To achieve the ultimate goal of clone model C matching the target model T, the loss function of clone model C in this invention is as follows:

[0113]

[0114] The loss function aims to minimize the target model output. Output of the clone model The parameters of the clone model C are optimized using the KL divergence between the generator G and the target model T, making its predictions closer to the real situation or the output of the target model T, thereby improving the accuracy and reliability of the clone. By alternately optimizing the network training mechanism, it is ensured that the generator G and the clone model C mutually promote each other during training, thus achieving higher overall performance.

[0115] S3.2: Employ numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as shown in the following formula:

[0116]

[0117] in, This represents the parameters of the generator network at time t+1. This represents the parameter value that needs to be updated, and η is the learning rate, which controls the step size of the parameter updates. This is the gradient of the generator loss function with respect to the generator parameters. To estimate the gradient in black-box attack scenarios, this invention proposes a zero-order gradient estimation method. This method approximates the gradient using the forward difference method, as shown in the following formula:

[0118]

[0119] Among them, u i Let be a random variable drawn with uniform probability from a d-dimensional unit sphere, and ∈ be a smoothing factor. Meanwhile, to reduce estimation errors, this invention employs the average stochastic gradient estimation method, calculating forward differences through m random directions and taking the average as the gradient estimate:

[0120]

[0121] in, This represents a gradient estimate or approximation of the variable x. This represents the generator loss function within m mini-batches of samples. The average of the gradient estimates with respect to parameter x. With more accurate gradient estimates, the generator parameters are also effectively updated.

[0122] In step S4, the present invention needs to evaluate and optimize the proxy model obtained in step S3, design an evaluation scheme, and use the evaluation results of the proxy model to replace the direct evaluation of the product model. Specifically, it includes the following steps:

[0123] S4.1: Fine-tune the clone model C using evaluation data; by fine-tuning the target model based on user interaction history, catastrophic forgetting during training can be effectively reduced, ensuring the proxy model can continuously and correctly classify old examples from the early training stages. Subsequently, the Bayesian loss function is used to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows:

[0124]

[0125] Where M represents the total number of users, N u It is a collection of products that user u has interacted with. and Let be the ratings of user u for products i and j, respectively. σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2This is a regularization term used to control model complexity and prevent overfitting. During fine-tuning, the cloned model C is continuously updated based on new evaluation data and user feedback to ensure that the model accurately simulates the recommendation logic of the target model.

[0126] S4.2: Based on the evaluation scheme, perform performance evaluation on the cloned model C; in this embodiment, a batch of known datasets will be used. The results are input into the target model and the surrogate model respectively to obtain their respective recommendation results. and To verify the modeling effectiveness of the cloned model, hit rate and fidelity were used for evaluation. First, to verify the model's performance in recommending products of interest to users, a hit rate evaluation was conducted. Hit rate refers to the list of products that the user actually expressed interest in from the product list recommended by the model. The proportion, of which the hit rate of the target model is calculated using the following formula:

[0127]

[0128] in, This indicates the number of items the user is interested in from the list of recommended items in the target model. This represents the total number of products recommended by the target model. Similarly, the hit rate of the clone model is calculated using the following formula:

[0129]

[0130] in, This represents the number of items the user is interested in from the list of products recommended by the agent model. This represents the total number of items recommended by the proxy model. Experimental results show that the target recommendation model has a hit rate of 93%, while the proxy model has a hit rate of 90%, indicating that the target recommendation model is slightly better in terms of accuracy. However, the proxy model's performance is quite close, showing a high hit rate. To evaluate the fitting performance of the proxy model, fidelity was chosen as the evaluation metric. Fidelity measures the similarity between the item list generated by the proxy model and the item list generated by the target model, and its calculation formula is as follows:

[0131]

[0132] in, This represents the intersection of the recommendation lists from the target model and the proxy model, i.e., the number of items recommended by both models. This represents the total number of items recommended by the target model. Experimental deployment revealed that the proxy model achieved a fidelity of 91.5%, demonstrating its excellent ability to mimic the behavior and output of the target recommendation model. It can accurately replicate the target model's recommendation logic, thus enabling an indirect evaluation of the target model's performance.

[0133] S4.3: First, compare the evaluation results of the surrogate model with the actual product recommendation results to analyze its recommendation effect, accuracy, and degree of matching with real data. Next, further determine whether the surrogate model can continuously provide valuable feedback, thereby ensuring its effectiveness and reliability in practical applications. This process not only verifies the predictive ability of the surrogate model but also helps to identify potential problems in the product recommendation model in a timely manner, thus optimizing the overall performance of the recommendation system.

[0134] In step S5, the target recommendation model is monitored based on the evaluation results obtained in the previous step, and modeling and evaluation are performed regularly to ensure that its recommendation logic continuously meets the expected standards. Simultaneously, the recommendation effect is verified through regular online A / B testing to ensure that the recommendation results are consistent with user preferences, thereby optimizing the recommendation model and ultimately applying it to actual product recommendations.

[0135] In summary, this invention utilizes a proxy method for recommender system models in scenarios where no real-world data is available, enabling high-precision construction of the target model even without data constraints. Furthermore, a rigorous proxy model monitoring and evaluation process ensures the accuracy of the product recommendation model's results and user satisfaction, providing strong support for the practical application of recommender systems.

[0136] Example 3

[0137] This embodiment relates to a dataless recommendation agent device for recommendation system evaluation, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a dataless recommendation model modeling method for recommendation system evaluation according to Embodiment 1.

[0138] Example 4

[0139] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements a method for constructing a high-precision proxy model in a data-free scenario, as described in Embodiment 1.

[0140] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A data-free recommendation model modeling method for recommendation system evaluation, characterized in that, Includes the following steps: S1: Construct generator G to generate virtual recommendation data; S2: Construct a clone model C to learn the recommendation mechanism of the target recommendation system; S3: Perform collaborative optimization and adjust the parameters of the generator G and the clone model C; S4: Evaluate the proxy model obtained in step S3 according to the evaluation plan; Step S1 specifically includes: In the absence of actual data input, the generator G receives a random noise vector and generates an output sample through a series of nonlinear transformations: in, θ is a low-dimensional noise vector randomly sampled from a standard normal distribution. G The parameters represent the generator model. This indicates that the generator G transforms a random noise vector through a series of nonlinear transformations. The process of converting data into user-project interaction data, therefore, Considered as a user-project bipartite graph: Where M represents the number of users in the bipartite graph, and n represents the number of items in the bipartite graph, these are the data generated by the generator. Inputting data into model T for querying, and returning the corresponding response. Where, θ T These represent the parameters of the target model. yes The recommended list of corresponding users is obtained after inputting into the target model T and querying it. Step S2 specifically includes: constructing a clone model C using a graph neural network LightGCN; the clone model C learns from the response data obtained after querying the target model T, analyzes the embedding representations of user and item nodes to capture the complex relationship between users and items, and then uses these embedding representations to predict users' interests in items, thereby generating a recommendation list. Where, θ C The parameters representing the clone model, yes The input to the clone model C yields the corresponding recommendation list for the generated user.

2. The data-free recommendation model modeling method for recommendation system evaluation according to claim 1, characterized in that: In step S3, the generator model G and the clone model C are optimized in a coordinated manner through an alternating optimization network training mechanism. Specifically, this includes: S3.1: Determine the co-optimization objective; during training, first optimize the generator. The generator's objective is to generate samples that maximize the difference from the target model's output, in order to better train the clone model C. Therefore, the output of the target model T is used. Output of clone model C The KL divergence between the two is used as the loss function of the generator, and the loss function formula is as follows: Here, the KL divergence measures the difference between two probability distributions; by maximizing The KL divergence between the generator and the target model is used to improve the generalization ability of the generator G on unseen data. Then, the clone model C is optimized, with the goal of minimizing the difference between its output and that of the target model to improve its accuracy. The loss function of the clone model C is as follows: The loss function aims to minimize the target model output. Output of the clone model The parameters of the clone model C are optimized by using the KL divergence between the two models to make its predictions more closely resemble the real situation or the output of the target model T; the generator G and the clone model C promote each other during the training process by alternately optimizing the network training mechanism. S3.2: Employ numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as shown in the following formula: in, This represents the parameters of the generator network at time t+1. This represents the parameter value that needs to be updated, and η is the learning rate, which controls the step size of the parameter updates. This is the gradient of the generator loss function with respect to the generator parameters. To estimate the gradient in black-box attack scenarios, a zero-order gradient estimation method is proposed, which approximates the gradient using the forward difference method. The specific formula is as follows: Among them, u i Let be a random variable drawn with uniform probability from a d-dimensional unit sphere, and ∈ be a smoothing factor. Meanwhile, to reduce estimation error, the average stochastic gradient estimation method is used, which calculates the forward difference through m random directions and takes the average as the gradient estimate. in, This represents a gradient estimate or approximation of the variable x. This represents the generator loss function in m samples with random directions. The average value of the gradient estimate with respect to parameter x; with the accurate estimation of the gradient, the generator parameters are also effectively updated.

3. The data-free recommendation model modeling method for recommendation system evaluation according to claim 1, characterized in that: Step S4 specifically includes: S4.1: Fine-tune the clone model C using evaluation data; by fine-tuning the model based on the user history interaction records that appeared in the target model, reduce the catastrophic forgetting phenomenon during the training process, and ensure that the proxy model can continue to correctly classify old examples in the early training stage. Subsequently, calculate the training samples using the Bayesian loss function to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows: Where M represents the total number of users, N u It is a collection of items that user u has interacted with. and Let be the ratings of user u for items i and j, respectively. σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2 It is a regularization term used to control the complexity of the model and prevent overfitting. During the fine-tuning process, the cloned model C is continuously updated based on new evaluation data and user feedback to ensure that the model can continuously adapt to changes in user needs and optimize the recommendation strategy to improve the user experience. S4.2: Evaluate the performance of clone model C and use hit rate, fidelity, and F1 score as evaluation metrics to measure the performance of clone model C; perform offline evaluation and online A / B testing on the clone model obtained from modeling.

4. A data-free recommendation modeling device for recommendation system evaluation, characterized in that, The system includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a data-free recommendation model modeling method for recommendation system evaluation as described in any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a data-free recommendation model modeling method for recommendation system evaluation as described in any one of claims 1-3.

Images