Recommendation system evaluation-oriented data-free recommendation model modeling method and device

Abstract

The invention discloses a non-data recommendation model modeling method and device for recommendation system evaluation, and the method comprises the steps: S1, constructing a generator G for generating virtual recommendation data; s2, constructing a clone model C used for learning a recommendation mechanism of a target recommendation system; s3, collaborative optimization is carried out, and parameters of the generator G and the clone model C are adjusted; and S4, evaluating the proxy model obtained by optimization in the step S3 according to an evaluation scheme. The collaborative optimization framework based on the generator-agent model is used for solving the modeling problem of the data-free recommendation system, and the difficulties of no data and model black box are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning and recommendation systems, and mainly to a data-free recommendation model building method and device for recommendation system evaluation. Background Art

[0002] In today's information overload, recommendation systems are the link between users and massive amounts of information. However, the security issues of recommendation systems are becoming increasingly prominent. Model stealing attacks, data injection attacks, and collaborative filtering attacks threaten the security of the system. These attacks may not only leak core model information, but also manipulate recommendation results and harm user interests. Therefore, it is crucial to conduct a comprehensive security evaluation of the recommendation system. This not only helps to verify the effectiveness of the recommendation system in fields such as marketing and e-commerce, but also ensures the adaptability and reliability of the system in response to changes in user needs and privacy protection challenges.

[0003] When conducting security evaluation of recommendation systems, the model to be evaluated often cannot be used directly for evaluation, but a proxy model needs to be built to replace it. At the same time, considering issues such as data privacy and intellectual property rights, the proxy model cannot obtain the model architecture and training data of the target model, which greatly limits the proxy model's learning and evaluation of the target model's recommendation mechanism. Therefore, it is very urgent to explore the modeling methods of recommendation models for data-free scenarios. However, existing recommendation model modeling methods usually focus on sequence recommendations in data-free scenarios, or are limited to modeling scenarios where some data is known. There are problems such as single research objects, strong data dependence, and insufficient feasibility. The joint modeling framework based on the generator-proxy model has been widely used in fields such as computer vision, showing excellent modeling capabilities and has the potential to solve this problem. Summary of the invention

[0004] The present invention aims to solve the deficiencies of the prior art and proposes a data-free recommendation model building method and device for recommendation system evaluation.

[0005] In response to the problem of excessive reliance on data sets by traditional proxy models, the present invention proposes an innovative method. This method integrates the out-of-distribution data generated by the generator and introduces a query mechanism, thereby breaking through the reliance on specific data sets. In response to the problem that traditional proxy models are limited to specific models, the present invention uses graph neural network technology to effectively reduce the complexity required to improve the generalization ability of the model. At the same time, in response to the problem of insufficient accuracy of the proxy model, the present invention uses numerical optimization technology and alternating optimization networks to ultimately construct a high-precision proxy model.

[0006] A first aspect of the present invention relates to a data-free recommendation model building method for recommendation system evaluation, comprising the following steps:

[0007] S1: Build a generator G to generate virtual recommendation data.

[0008] S2: Build a clone model C to learn the recommendation mechanism of the target recommendation system.

[0009] S3: Perform collaborative optimization to adjust the parameters of the generator G and the clone model C.

[0010] S4: Evaluate the proxy model optimized in step S3 according to the evaluation plan.

[0011] In step S1, a generator G is constructed to generate virtual recommendation data. Specifically, the generator is a key tool for constructing a proxy model. After training, it can approximate the user configuration data distribution of the target model, thereby realizing the construction of a proxy model in a data-free environment and playing a vital role in the alternating optimization network. Specifically, the generator G receives a random noise vector without actual data input, and generates an output sample through a series of nonlinear transformations:

[0012]

[0013] in, is a low-dimensional noise vector randomly sampled from a standard normal distribution, θ G represents the parameters of the generator model, The generator G transforms the random noise vector The process of converting into user-item interaction data, therefore, It can be viewed as a user-item bipartite graph:

[0014]

[0015] Among them, m represents the number of users in the bipartite graph, n represents the number of items in the bipartite graph, and these data generated by the generator are Input into model T for query and return the corresponding response:

[0016]

[0017] Among them, θ T represents the parameters of the target model, yes The corresponding recommendation list generated by querying the target model T is generated.

[0018] In step S2, a clone model C is constructed to learn the recommendation mechanism of the target recommendation system T. The details are as follows: It can be regarded as a user-item bipartite graph. Therefore, under the condition of no data access, the present invention uses the graph neural network LightGCN to construct the clone model C. The clone model C learns the response data obtained after querying the target model T, analyzes the embedded representation of user and item nodes to capture the complex relationship between users and items. Then, the clone model will use these embedded representations to predict the user's interest in the item, thereby generating a recommendation list.

[0019]

[0020] Among them, θ C represents the parameters of the cloned model, yes Input into clone model C to generate the corresponding recommendation list for the user.

[0021] In step S3, the present invention realizes the collaborative optimization of the generator model G and the clone model C by alternately optimizing the network training mechanism, which specifically includes the following steps:

[0022] S3.1: Determine the collaborative optimization goal; during the training process, the generator is optimized first. The goal of the generator is to generate samples that maximize the difference with the output of the target model in order to better train the clone model C, thus using the output of the target model T Output with cloned model C The KL divergence between is used as the loss function of the generator, and the loss function formula is as follows:

[0023]

[0024] The KL divergence measures the difference between two probability distributions. The KL divergence between them improves the generalization ability of the generator G on unseen data, so that it can generate more realistic samples. Then the clone model is optimized, and the goal of the clone model C is to minimize the difference with the target model output to improve its accuracy. In order to achieve the ultimate goal of matching the clone model C with the target model T, the loss function of the clone model C of the present invention is as follows:

[0025]

[0026] Among them, the loss function aims to minimize the target model output Output with cloned model The KL divergence between them is used to optimize the parameters of the clone model C, making its prediction closer to the actual situation or the output of the target model T, thereby improving the accuracy and reliability of the clone. By alternately optimizing the network training mechanism, it is ensured that the generator G and the clone model C promote each other during the training process, thereby achieving higher overall performance.

[0027] S3.2: Use numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as follows:

[0028]

[0029] in, represents the parameters of the generator network at time t+1, represents the parameter value that needs to be updated currently, η is the learning rate, which is used to control the step size of parameter update, is the gradient of the generator loss function with respect to the generator parameters. In order to estimate the gradient in the black box attack scenario, this paper proposes a zero-order gradient estimation method. This method approximates the gradient by the forward difference method, and the specific formula is as follows:

[0030]

[0031] Among them, u i is a random variable drawn from the d-dimensional unit sphere with uniform probability, and ∈ is a smoothing factor. At the same time, in order to reduce the estimation error, the present invention adopts the average stochastic gradient estimation method, calculates the forward difference through m random directions, and takes the average value as the gradient estimate:

[0032]

[0033] in, represents the gradient estimate or approximation of the variable x, Indicates that the generator loss function is in the samples in m small batches The average of the gradient estimates with respect to the parameters x. With accurate estimates of the gradients, the parameters of the generator are also updated efficiently.

[0034] In step S4, the proxy model obtained in step S3 is evaluated, which specifically includes the following steps:

[0035] S4.1: Fine-tune the clone model C using the evaluation data; by fine-tuning the historical user interaction records that have appeared in the target model, the catastrophic forgetting phenomenon of the model during training can be effectively reduced, ensuring that the proxy model can continue to correctly classify old examples in the early stages of training. Subsequently, the Bayesian loss function is used to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows:

[0036]

[0037] Among them, M represents the total number of users, N u is the set of items that user u has interacted with, and are the ratings of user u on items i and j, σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2 is a regularization term used to control the complexity of the model and prevent overfitting. During the fine-tuning process, the clone model C is continuously updated based on new evaluation data and user feedback to ensure that the model can accurately simulate the recommendation logic of the target model.

[0038] S4.2: Evaluate the performance of clone model C and use evaluation indicators such as hit rate, fidelity, and F1 score to measure the performance of clone model C to evaluate the performance of clone model C; further conduct offline evaluation and online A / B testing on the clone model obtained by modeling.

[0039] The main innovative features of the present invention are as follows:

[0040] (1) By introducing out-of-distribution data and query mechanisms generated by the generator, the dependence on specific data sets is effectively reduced, enabling the recommendation system to be modeled in a data-free environment, thereby enhancing the applicability and flexibility of the model.

[0041] (2) Through numerical optimization technology and alternate optimization of network structure, a high-precision proxy model modeling framework was constructed, which significantly improved the accuracy and reliability of model prediction.

[0042] The beneficial effects of the present invention are as follows: in view of the various difficulties in building a proxy model without direct user behavior data or sensitive data being unavailable, such as data sparsity, model generalization ability, low proxy model accuracy, etc., the present invention proposes a data-free recommendation model modeling method for recommendation system evaluation. This method not only provides a new idea for recommendation system modeling in a data-free environment, but also enhances the overall performance of the recommendation system through its innovative proxy model construction method, making it widely applicable and highly competitive in a variety of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flow chart of the method of the present invention.

[0045] Figure 2 It is a schematic diagram of the device of the present invention. Specific implementation methods

[0047] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0048] It should be understood that the terms described in the present invention are only for describing special embodiments and are not intended to limit the present invention. In addition, for the numerical range in the present invention, it should be understood that each intermediate value between the upper and lower limits of the scope is also specifically disclosed. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the described range is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded in the scope.

[0049] Unless otherwise indicated, all technical and scientific terms used herein have the same meanings as those generally understood by those skilled in the art. Although the present invention describes only preferred methods and materials, any methods and materials similar or equivalent to those described herein may also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods and / or materials associated with the documents. In the event of a conflict with any incorporated document, the content of this specification shall prevail.

[0050] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0051] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0052] Unless otherwise specified, the "parts" described in the present invention are all based on mass parts.

[0053] Example 1

[0054] This embodiment relates to a model building method for recommendation system evaluation when the internal working mechanism of the target model cannot be directly accessed. It is applied to the scenario with the e-commerce platform Amazon as the background, and is used to evaluate the effect of the user-based collaborative filtering model User CF, including the following steps:

[0055] S1: Build a generator G to generate virtual recommendation data.

[0056] S2: Build a clone model C to learn the recommendation mechanism of the target recommendation system.

[0057] S3: Perform collaborative optimization to adjust the parameters of the generator G and the clone model C.

[0058] S4: Evaluate the proxy model optimized in step S3 according to the evaluation plan, and use the evaluation results of the proxy model to replace the direct evaluation of the product model.

[0059] In step S1, the generator is a key tool for constructing the proxy model. After training, it can approximate the user configuration data distribution of the target model, including information such as the user's gender, age, geographic location, and purchasing power, thereby realizing the construction of the proxy model in a data-free environment. Specifically, the user feature generator G receives a random noise vector without actual data input, and generates an output sample through a series of nonlinear transformations:

[0060]

[0061] in, is a low-dimensional noise vector randomly sampled from a standard normal distribution, θG represents the parameters of the generator model, The generator G transforms the random noise vector The process of converting into user-product interaction data, therefore, It can be regarded as a user-item bipartite graph:

[0062]

[0063] Among them, m represents the number of users in the bipartite graph, n represents the number of items in the bipartite graph, and these data generated by the generator are Input into model T for query, and the corresponding response will be returned:

[0064]

[0065] Among them, θ T represents the parameters of the target model, yes The recommendation list corresponding to the generated user is obtained after inputting it into the target model T for query, that is, the predicted ranking of the products that the generated user may be interested in by the target model.

[0066] In step S2, the clone model C aims to learn and imitate the recommendation mechanism of the target recommendation system T. It can be regarded as a user-item bipartite graph. Therefore, under the condition of no data access, the present invention uses the graph neural network LightGCN to construct the clone model C. The clone model C learns the response data obtained after querying the target model T, analyzes the embedded representation of user and item nodes to capture the complex relationship between users and items. Then, the clone model will use these embedded representations to predict the user's interest in the product, thereby generating a recommendation list.

[0067]

[0068] Among them, θ C represents the parameters of the cloned model, By cloning model C, we will get the corresponding output list That is, the clone model will generate a predicted ranking of the products that the user may be interested in.

[0069] In step S3, the present invention realizes the collaborative optimization of the generator model G and the clone model C by alternately optimizing the network training mechanism, which specifically includes the following steps:

[0070] S3.1: Determine the collaborative optimization goal; during the training process, the generator is optimized first. The goal of the generator is to generate samples that maximize the difference with the output of the target model in order to better train the clone model C, thus using the output of the target model T Output with cloned model C The KL divergence between is used as the loss function of the generator, and the loss function formula is as follows:

[0071]

[0072] The KL divergence measures the difference between two probability distributions. The KL divergence between them improves the generalization ability of the generator G on unseen data, so that it can generate more realistic samples. Then the clone model is optimized, and the goal of the clone model C is to minimize the difference with the target model output to improve its accuracy. In order to achieve the ultimate goal of matching the clone model C with the target model T, the loss function of the clone model C of the present invention is as follows:

[0073]

[0074] Among them, the loss function aims to minimize the target model output Output with cloned model The KL divergence between them is used to optimize the parameters of the clone model C, making its prediction closer to the actual situation or the output of the target model T, thereby improving the accuracy and reliability of the clone. By alternately optimizing the network training mechanism, it is ensured that the generator G and the clone model C promote each other during the training process, thereby achieving higher overall performance.

[0075] S3.2: Use numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as follows:

[0076]

[0077] in, represents the parameters of the generator network at time t+1, represents the parameter value that needs to be updated currently, η is the learning rate, which is used to control the step size of parameter update, is the gradient of the generator loss function with respect to the generator parameters. In order to estimate the gradient in the black box attack scenario, this paper proposes a zero-order gradient estimation method. This method approximates the gradient by the forward difference method, and the specific formula is as follows:

[0078]

[0079] Among them, u i is a random variable drawn from the d-dimensional unit sphere with uniform probability, and ∈ is a smoothing factor. At the same time, in order to reduce the estimation error, the present invention adopts the average stochastic gradient estimation method, calculates the forward difference through m random directions, and takes the average value as the gradient estimate:

[0080]

[0081] in, represents the gradient estimate or approximation of the variable x, Indicates that the generator loss function is in the samples in m small batches The average of the gradient estimates with respect to the parameters x. With accurate estimates of the gradients, the parameters of the generator are also updated efficiently.

[0082] In step S4, the present invention needs to evaluate and optimize the proxy model obtained in step S3, design an evaluation plan, and use the evaluation results of the proxy model to replace the direct evaluation of the product model, which specifically includes the following steps:

[0083] S4.1: Fine-tune the clone model C using the evaluation data; by fine-tuning the historical user interaction records that have appeared in the target model, the catastrophic forgetting phenomenon of the model during training can be effectively reduced, ensuring that the proxy model can continue to correctly classify old examples in the early stages of training. Subsequently, the Bayesian loss function is used to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows:

[0084]

[0085] Among them, M represents the total number of users, N u is the set of products that user u has interacted with, and are the ratings of user u on products i and j, σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2 is a regularization term used to control the complexity of the model and prevent overfitting. During the fine-tuning process, the clone model C is continuously updated based on new evaluation data and user feedback to ensure that the model can accurately simulate the recommendation logic of the target model.

[0086] S4.2: According to the evaluation scheme setting, the performance of the clone model C is evaluated; in this embodiment, a batch of known data sets Input into the target model and proxy model respectively to obtain their respective recommendation results and In order to verify the modeling effect of the clone model, the hit rate and fidelity were used for evaluation. First, the hit rate was evaluated to verify the performance of the model in recommending products that users are interested in. The hit rate refers to the list of products that the user actually shows interest in among the list of products recommended to the user by the recommendation model. The proportion of the target model, where the hit rate calculation formula is as follows:

[0087]

[0088] in, Indicates the number of items that the user is interested in in the list of items recommended by the target model. Represents the total number of products recommended by the target model. Similarly, the hit rate calculation formula of the clone model is as follows:

[0089]

[0090] in, Indicates the number of items that the user is interested in in the list of items recommended by the proxy model. Represents the total number of items recommended by the proxy model. The experimental results show that the target recommendation model has a hit rate of 93% and the proxy model has a hit rate of 90%, indicating that the target recommendation model is slightly better in terms of accuracy, but the proxy model's performance is also quite close, showing a higher hit rate. In order to evaluate the fitting performance of the proxy model, fidelity is chosen as the evaluation indicator. Fidelity is used to measure the similarity between the product list generated by the proxy model and the product list generated by the target model. The calculation formula is as follows:

[0091]

[0092] in, represents the number of intersections between the target model and the proxy model recommendation lists, that is, the number of items recommended by both models. Represents the total number of items recommended by the target model. After experimental deployment, it was found that the fidelity of the proxy model reached 91.5%. This result shows that it is very good at imitating the behavior and output of the target recommendation model, and can accurately replicate the recommendation logic of the target model, thereby achieving indirect evaluation of the target model performance.

[0093] S4.3: First, compare the evaluation results of the proxy model with the actual product recommendation results to analyze the recommendation effect, accuracy, and degree of match with the real data. Then, further determine whether the proxy model can continue to provide valuable feedback to ensure its effectiveness and reliability in practical applications. This process not only verifies the predictive ability of the proxy model, but also helps to promptly discover potential problems in the product recommendation model, thereby optimizing the overall performance of the recommendation system. In short, the present invention proposes a recommendation system model evaluation agent method suitable for scenarios where no data is available. In the absence of data, the target model can be constructed and evaluated with high precision. Example 2

[0094] This embodiment provides a data-free recommendation model modeling method for recommendation system evaluation, which is applied to the scenario of implementing supervision and evaluation on commodity models and making commodity recommendations based on this. It includes the following steps:

[0095] S1: Build a generator G to generate virtual recommendation data.

[0096] S2: Build a clone model C to learn the recommendation mechanism of the target recommendation system.

[0097] S3: Perform collaborative optimization to adjust the parameters of the generator G and the clone model C.

[0098] S4: Evaluate the proxy model optimized in step S3 according to the evaluation plan, and use the evaluation results of the proxy model to replace the direct evaluation of the product model.

[0099] S5: Supervise the target recommendation model based on the evaluation results for actual product recommendations.

[0100] In step S1, the generator is a key tool for constructing the proxy model. After training, it can approximate the user configuration data distribution of the target model, including information such as the user's gender, age, geographic location, and purchasing power, thereby realizing the construction of the proxy model in a data-free environment. Specifically, the user feature generator G receives a random noise vector without actual data input, and generates an output sample through a series of nonlinear transformations:

[0101]

[0102] in, is a low-dimensional noise vector randomly sampled from a standard normal distribution, θ G represents the parameters of the generator model, The generator G transforms the random noise vector The process of converting into user-product interaction data, therefore, It can be regarded as a user-item bipartite graph:

[0103]

[0104] Among them, m represents the number of users in the bipartite graph, n represents the number of items in the bipartite graph, and these data generated by the generator are Input into model T for query, and the corresponding response will be returned:

[0105]

[0106] Among them, θ T represents the parameters of the target model, yes The recommendation list corresponding to the generated user is obtained after inputting it into the target model T for query, that is, the predicted ranking of the products that the generated user may be interested in by the target model.

[0107] In step S2, the clone model C aims to learn and imitate the recommendation mechanism of the target recommendation system T. It can be regarded as a user-item bipartite graph. Therefore, under the condition of no data access, the present invention uses the graph neural network LightGCN to construct the clone model C. The clone model C learns the response data obtained after querying the target model T, analyzes the embedded representation of user and item nodes to capture the complex relationship between users and items. Then, the clone model will use these embedded representations to predict the user's interest in the product, thereby generating a recommendation list.

[0108]

[0109] Among them, θ C represents the parameters of the cloned model, By cloning model C, we will get the corresponding output list That is, the clone model will generate a predicted ranking of the products that the user may be interested in.

[0110] In step S3, the present invention realizes the collaborative optimization of the generator model G and the clone model C by alternately optimizing the network training mechanism, which specifically includes the following steps:

[0111] S3.1: Determine the collaborative optimization goal; during the training process, the generator is optimized first. The goal of the generator is to generate samples that maximize the difference with the output of the target model in order to better train the clone model C, thus using the output of the target model T Output with cloned model C The KL divergence between is used as the loss function of the generator, and the loss function formula is as follows:

[0112]

[0113] The KL divergence measures the difference between two probability distributions. The KL divergence between them improves the generalization ability of the generator G on unseen data, so that it can generate more realistic samples. Then the clone model is optimized, and the goal of the clone model C is to minimize the difference with the target model output to improve its accuracy. In order to achieve the ultimate goal of matching the clone model C with the target model T, the loss function of the clone model C of the present invention is as follows:

[0114]

[0115] Among them, the loss function aims to minimize the target model output Output with cloned model The KL divergence between them is used to optimize the parameters of the clone model C, making its prediction closer to the actual situation or the output of the target model T, thereby improving the accuracy and reliability of the clone. By alternately optimizing the network training mechanism, it is ensured that the generator G and the clone model C promote each other during the training process, thereby achieving higher overall performance.

[0116] S3.2: Use numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as follows:

[0117]

[0118] in, represents the parameters of the generator network at time t+1, represents the parameter value that needs to be updated currently, η is the learning rate, which is used to control the step size of parameter update, is the gradient of the generator loss function with respect to the generator parameters. In order to estimate the gradient in the black box attack scenario, this paper proposes a zero-order gradient estimation method. This method approximates the gradient by the forward difference method, and the specific formula is as follows:

[0119]

[0120] Among them, u i is a random variable drawn from the d-dimensional unit sphere with uniform probability, and ∈ is a smoothing factor. At the same time, in order to reduce the estimation error, the present invention adopts the average stochastic gradient estimation method, calculates the forward difference through m random directions, and takes the average value as the gradient estimate:

[0121]

[0122] in, represents the gradient estimate or approximation of the variable x, Indicates that the generator loss function is in the samples in m small batches The average of the gradient estimates with respect to the parameters x. With accurate estimates of the gradients, the parameters of the generator are also updated efficiently.

[0123] In step S4, the present invention needs to evaluate and optimize the proxy model obtained in step S3, design an evaluation plan, and use the evaluation results of the proxy model to replace the direct evaluation of the product model, which specifically includes the following steps:

[0124] S4.1: Fine-tune the clone model C using the evaluation data; by fine-tuning the historical user interaction records that have appeared in the target model, the catastrophic forgetting phenomenon of the model during training can be effectively reduced, ensuring that the proxy model can continue to correctly classify old examples in the early stages of training. Subsequently, the Bayesian loss function is used to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows:

[0125]

[0126] Among them, M represents the total number of users, N u is the set of products that user u has interacted with, and are the ratings of user u on products i and j, σ is the sigmoid function used to convert the ratings into probability values, and λ||E (0) || 2is a regularization term used to control the complexity of the model and prevent overfitting. During the fine-tuning process, the clone model C is continuously updated based on new evaluation data and user feedback to ensure that the model can accurately simulate the recommendation logic of the target model.

[0127] S4.2: According to the evaluation scheme setting, the performance of the clone model C is evaluated; in this embodiment, a batch of known data sets Input into the target model and proxy model respectively to obtain their respective recommendation results and In order to verify the modeling effect of the clone model, the hit rate and fidelity were used for evaluation. First, the hit rate was evaluated to verify the performance of the model in recommending products that users are interested in. The hit rate refers to the list of products that the user actually shows interest in among the list of products recommended to the user by the recommendation model. The proportion of the target model, where the hit rate calculation formula is as follows:

[0128]

[0129] in, Indicates the number of items that the user is interested in in the list of items recommended by the target model. Represents the total number of products recommended by the target model. Similarly, the hit rate calculation formula of the clone model is as follows:

[0130]

[0131] in, Indicates the number of items that the user is interested in in the list of items recommended by the proxy model. Represents the total number of items recommended by the proxy model. The experimental results show that the target recommendation model has a hit rate of 93% and the proxy model has a hit rate of 90%, indicating that the target recommendation model is slightly better in terms of accuracy, but the proxy model's performance is also quite close, showing a higher hit rate. In order to evaluate the fitting performance of the proxy model, fidelity is chosen as the evaluation indicator. Fidelity is used to measure the similarity between the product list generated by the proxy model and the product list generated by the target model. The calculation formula is as follows:

[0132]

[0133] in, represents the number of intersections between the target model and the proxy model recommendation lists, that is, the number of items recommended by both models. Represents the total number of items recommended by the target model. After experimental deployment, it was found that the fidelity of the proxy model reached 91.5%. This result shows that it is very good at imitating the behavior and output of the target recommendation model, and can accurately replicate the recommendation logic of the target model, thereby achieving indirect evaluation of the target model performance.

[0134] S4.3: First, compare the evaluation results of the proxy model with the actual product recommendation results to analyze its recommendation effect, accuracy, and degree of match with real data. Then, further determine whether the proxy model can continue to provide valuable feedback to ensure its effectiveness and reliability in practical applications. This process not only verifies the predictive ability of the proxy model, but also helps to promptly discover potential problems in the product recommendation model, thereby optimizing the overall performance of the recommendation system.

[0135] In step S5, the target recommendation model is supervised according to the evaluation results obtained in the previous step, and modeling and evaluation are performed regularly to ensure that its recommendation logic continues to meet the expected standards. At the same time, the recommendation effect is verified through regular online A / B testing to ensure that the recommendation results are consistent with user preferences, thereby optimizing the recommendation model and finally applying it to actual product recommendations.

[0136] In summary, the present invention uses a proxy method for the recommendation system model in the real-world scenario where no data is available, which can achieve high-precision construction of the target model under the restriction of no data. In addition, through the strict supervision and evaluation process of the proxy model, the accuracy of the recommendation results of the product recommendation model and user satisfaction are ensured, providing strong support for the practical application of the recommendation system.

[0137] Example 3

[0138] The present embodiment relates to a data-free recommendation agent device for recommendation system evaluation, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a data-free recommendation model modeling method for recommendation system evaluation in Example 1.

[0139] Example 4

[0140] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, a method of constructing a high-precision proxy model in a data-free scenario according to Embodiment 1 is implemented.

[0141] The embodiments described above are only descriptions of the preferred modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A data-free recommendation modeling method for recommendation system evaluation, characterized in that: The following steps are involved: S1: Build a generator G to generate virtual recommendation data; S2: Build a clone model C to learn the recommendation mechanism of the target recommendation system; S3: Perform collaborative optimization to adjust the parameters of the generator G and the clone model C; S4: Evaluate the proxy model optimized in step S3 according to the evaluation plan.

2. The method for building a data-free recommendation model for recommendation system evaluation according to claim 1, characterized in that: The construction of the generator G in step S1 is used to generate virtual recommendation data, specifically including: in the absence of actual data input, the generator G receives a random noise vector and generates an output sample through a series of nonlinear transformations: x=G(z:θ G ){z~N(0,Ι) (1) Among them, z is a low-dimensional noise vector randomly sampled from a standard normal distribution, θ G represents the parameters of the generator model, x = G(z; θ G ) represents the process of the generator G converting the random noise vector z into user-item interaction data through a series of nonlinear transformations. Therefore, x is regarded as a user-item bipartite graph: Among them, m represents the number of users in the bipartite graph, and n represents the number of items in the bipartite graph. The data x generated by the generator is input into the model T for query, and the corresponding response is returned. Among them, θ T represents the parameters of the target model, It is the recommendation list of the corresponding generated user obtained after x is input into the target model T for query.

3. The method for building a data-free recommendation model for recommendation system evaluation according to claim 1, characterized in that: The construction of the clone model C described in step S2 is used to learn the recommendation mechanism of the target recommendation system, specifically including: using the graph neural network LightGCN to construct the clone model C; the clone model C learns the response data obtained after querying the target model T, analyzes the embedded representation of the user and project nodes to capture the complex relationship between users and projects, and then the clone model uses these embedded representations to predict the user's interest in the project, thereby generating a recommendation list Among them, θ C represents the parameters of the cloned model, It is the recommendation list of the corresponding generated user obtained by inputting x into the clone model C.

4. The method for building a data-free recommendation model for recommendation system evaluation according to claim 1, characterized in that: In step S3, the network training mechanism is alternately optimized to achieve collaborative optimization of the generator model G and the clone model C, specifically including: S3.1: Determine the collaborative optimization goal; during the training process, the generator is optimized first. The goal of the generator is to generate samples that maximize the difference with the output of the target model in order to better train the clone model C, thus using the output of the target model T Output with clone model C The KL divergence between is used as the loss function of the generator, and the loss function formula is as follows: Among them, KL divergence measures the difference between two probability distributions; by maximizing The KL divergence between them is used to improve the generalization ability of the generator G on unseen data, so that it can generate more realistic samples. Then the clone model is optimized. The goal of the clone model C is to minimize the difference with the output of the target model to improve its accuracy. In order to achieve the ultimate goal of matching the clone model C with the target model T, the loss function of the clone model C is as follows: Among them, the loss function aims to minimize the target model output Output with cloned model The KL divergence between them is used to optimize the parameters of the clone model C, so that its prediction is closer to the actual situation or the output of the target model T, and the accuracy and reliability of the clone are improved; by alternately optimizing the network training mechanism, the generator G and the clone model C promote each other during the training process, achieving higher overall performance; S3.2: Use numerical optimization techniques to adjust model parameters; specifically, use gradient descent to minimize the loss function, as follows: in, represents the parameters of the generator network at time t+1, represents the parameter value that needs to be updated currently, η is the learning rate, which is used to control the step size of parameter update, is the gradient of the generator loss function with respect to the generator parameters. In order to estimate the gradient in the black-box attack scenario, a zero-order gradient estimation method is proposed to approximate the gradient through the forward difference method. The specific formula is as follows: Among them, u i is a random variable drawn from the d-dimensional unit sphere with uniform probability, and ∈ is a smoothing factor. At the same time, in order to reduce the estimation error, the average stochastic gradient estimation method is used to calculate the forward difference in m random directions and take the average value as the gradient estimate: in, represents the gradient estimate or approximation of the variable x, Indicates that the generator loss function is in the samples in m small batches The average of the gradient estimates with respect to the parameters x; with accurate estimates of the gradients, the parameters of the generator are efficiently updated.

5. The method for building a data-free recommendation model for recommendation system evaluation according to claim 1, characterized in that: The evaluation of the proxy model in step S4 specifically includes: S4.1: Fine-tune the clone model C using the evaluation data; by fine-tuning the historical user interaction records that have appeared in the target model, reduce the catastrophic forgetting phenomenon of the model during training, and ensure that the proxy model can continue to correctly classify old examples in the early stages of training. Subsequently, use the Bayesian loss function to calculate these training samples to enhance user engagement and satisfaction. The Bayesian loss function formula is as follows: Among them, M represents the total number of users, N u is the set of items that user u has interacted with, and are the ratings of user u on items i and j, σ is the sigmoid function used to convert the ratings into probability values, and λ‖E (0) ‖ 2 is a regularization term used to control the complexity of the model and prevent overfitting. During the fine-tuning process, the clone model C is continuously updated based on new evaluation data and user feedback to ensure that the model can continue to adapt to changes in user needs and optimize the recommendation strategy to improve user experience. S4.2: Evaluate the performance of clone model C and use evaluation indicators such as hit rate, fidelity, and F1 score to measure the performance of clone model C to evaluate the performance of clone model C; conduct offline evaluation and online A / B testing on the clone model obtained by modeling.

6. A data-free recommendation model building device for recommendation system evaluation, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a data-free recommendation model modeling method for recommendation system evaluation according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a data-free recommendation model building method for recommendation system evaluation according to any one of claims 1 to 5 is implemented.

Images