A Collaborative Filtering Recommendation Method Based on Aesthetic Features and Adversarial Learning

Abstract

A collaborative filtering recommendation method based on aesthetic features and adversarial learning integrates aesthetic features into the evaluation system using an aesthetic network and improves the recommendation accuracy and robustness through adversarial learning. This method can be divided into two stages: the feature acquisition stage and the model training stage. In the feature acquisition stage, the product dataset is input into the CNN and the aesthetic network to extract the CNN features and aesthetic features of the product respectively, and the two types of features are concatenated to obtain a new feature matrix, which is used to construct the CFA model. In the model training stage, the SGD learning algorithm is used to first learn the model parameters to optimize the model performance, and then adversarial perturbations are added at the model parameters to form the ACFA model, and adversarial learning is carried out to minimize the model performance as much as possible. The present invention can improve the recommendation accuracy and robustness of the promotion system and has certain application value in the field of recommendation systems.

Description

Technical Field

[0001] The present invention relates to the field of recommendation systems, and in particular to a collaborative filtering recommendation method based on aesthetic features and adversarial learning. Background Art

[0002] Recommendation systems are widely used in various online services such as e-commerce, online news, and social media. Among them, the collaborative filtering recommendation method has achieved great success in many applications due to its characteristic of modeling based on the historical interactions between users and items.

[0003] The visual appearance of products has an important impact on consumers' decisions, so product images are increasingly valued in recommendation systems. Most existing methods rely on traditional features to represent images, such as visual features extracted by convolutional neural networks (CNN features), features extracted by scale-invariant feature transform algorithms (SIFT features), and color histograms. Aesthetic features, as an important type of feature, are rarely considered. Although they play a crucial role in product recommendation because users' decisions largely depend on whether the product meets aesthetics, traditional image features cannot directly describe this. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art and improve the recommendation accuracy of the collaborative filtering recommendation algorithm and enhance its anti-interference ability, the present invention proposes a collaborative filtering recommendation method based on aesthetic features and adversarial learning, integrates aesthetic features into the evaluation system, as an important factor affecting users' ratings of products to improve recommendation accuracy, and applies the adversarial learning method to enhance the robustness of the recommendation model.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A collaborative filtering recommendation method based on aesthetic features and adversarial learning, comprising the following steps:

[0007] Step 1: Extract CNN feature f CNN and AES feature f AES from the product dataset respectively, where the AES feature is the aesthetic feature;

[0008] Step 2: Concatenate the CNN feature and the AES feature to obtain a new feature matrix F q ;

[0009] Step 3: Add aesthetic features on the basis of collaborative filtering, define the proposed original CFA model, and use to represent;

[0010] Step 4: Determine the target and addition location of the adversarial perturbation to improve the accuracy and precision of the algorithm while minimizing the complexity of the model training process;

[0011] Step 5: Define the proposed adversarial perturbation ACFA model, denoted as ;

[0012] Step 6: Model training, where model parameter learning is achieved by minimizing the objective function to obtain the optimal model parameters;

[0013] Step 7: Adjust the two hyperparameters ε and λ regarding the perturbation in the model to optimize the model performance;

[0014] Step 8: Select the evaluation metrics of the recommendation system as the basis for evaluating the model performance;

[0015] Step 9: Record and organize the experimental results, compare with the baseline model, and form conclusions.

[0016] Furthermore, in the above Step 2, the new feature matrix is obtained and denoted as where the CNN feature f CNN ) and the AES feature f AES are respectively extracted from the product dataset through the CNN convolutional neural network and the BDN aesthetics network.

[0017] Still further, in the above Step 3, after adding the aesthetic features, the definition of the CFA model without adversarial perturbation is expressed as:

[0018]

[0019] where M is the user preference matrix, U represents the user embedding matrix, V represents the product embedding matrix, F is the feature matrix, p and q respectively represent the user and the product, is obtained based on the latent features, V captures the user preference information, while is obtained based on the image features, M encodes the user preferences, and the construction of the feature matrix F *q has been given in Step 2.

[0020] Even further, in the above Step 3, since the CFA model uses the typical Bayesian personalized ranking optimization criterion to optimize the implicit feedback of users, the BPR loss is used as the corresponding objective function, which is expressed as follows:

[0021]

[0022] where D is the training set, Θ is the model parameter, p, q, and q′ respectively represent the user, the product purchased by the user, and the product not purchased by the user, and λ ΘDenote the coefficients corresponding to each parameter Θ, and ∥·∥ is the L2 norm, where It is expressed as:

[0023]

[0024] Furthermore, in step 4, since the first important process of adversarial learning is to construct perturbations that can maximize the BPR loss (the worst recommendation effect), the introduced adversarial perturbations are determined as:

[0025] Δ adv = argmax Δ,∥Δ∥≤ε L′ BPR (D|Θ)

[0026] where L′ BPR (D|Θ) = L BPR (D|Θ + Δ), D is the training set, Θ is the parameter of the model, ε ≥ 0 is the hyperparameter controlling the perturbation amplitude, and ∥·∥ is the L2 norm. And choose to add perturbations to the model parameters. By adding perturbations at the model parameters, on the one hand, it can avoid the cross - influence between the network for extracting image features and the model training process, improve the accuracy and precision of the algorithm, and on the other hand, it can avoid excessive complexity during the model training process and improve the efficiency of the algorithm.

[0027] Furthermore, in step 5, after determining the position where the adversarial perturbation is added, the adversarial perturbation model ACFA is defined as:

[0028]

[0029] where Δ * ∈R K represents the adversarial perturbation added to the aesthetic feature matrix.

[0030] Furthermore, in step 6, during the model training process, for an instance (p, q, q′), it is necessary to update the model parameters by minimizing the corresponding objective function under the condition that the adversarial perturbation has been determined. Since the relevant objective function is fixed, the model parameter learning at this time can be regarded as a problem of finding the minimum value and can be approximated as gradient descent, as shown below:

[0031]

[0032] where Θ = {U, V, M}, η represents the learning rate, and the Adagrad optimizer in TensorFlow is selected to participate in the model parameter learning, which can overcome the limitation that SGD cannot adaptively adjust the learning rate.

[0033] Furthermore, in step 7, the addition of adversarial perturbations introduces two additional hyperparameters ε and λ to control the perturbation amplitude, i.e., the perturbation size, and the influence of the loss function generated by the adversarial perturbations on the total loss function, respectively. The values of ε and λ will be changed respectively, and the model training will be re-performed after each change of the values. The hyperparameter values corresponding to the best model performance will be used as the final values.

[0034] The adversarial learning of the present invention is an effective method for enhancing the anti-interference ability of the model. By adding adversarial perturbations to reduce the performance of the model to the lowest level, and then using the method of adversarial learning to train the model, not only can the recommendation accuracy of the model be improved, but also the robustness of the recommendation model can be enhanced.

[0035] The beneficial effects of the present invention are as follows:

[0036] 1. Extract the aesthetic features in the implicit feedback dataset and integrate them into the tensor factorization model to capture the aesthetic preferences of users and recommend products that match their styles for users;

[0037] 2. Apply the adversarial learning method, add perturbations to the model parameters to maximize the loss of the model, and then update the model parameters through training to improve the recommendation performance and achieve the effect of enhancing the robustness of the model;

[0038] 3. The present invention mainly studies the influence of product aesthetic features on the model, providing ideas for subsequent work. For example, other context features such as auditory features and spectral features can be added to the basic model for recommendation;

[0039] 4. Mainly apply the SGD learning algorithm in deep learning to optimize the ACFA model. Since the objective function contains non-linear functions and the number of training instances is huge, it is easy to improve the efficiency by using SGD. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of a collaborative filtering recommendation method based on aesthetic features and adversarial learning;

[0041] Figure 2 is a schematic diagram of the functional modules of a personalized recommendation system;

[0042] Figure 3 is a schematic diagram of the comparison between CNN features and aesthetic features;

[0043] Figure 4 is a framework diagram of the ACFA model;

[0044] Figure 5 is a comparison diagram of the recommendation performance of each model under the NDCG index;

[0045] Figure 6It is a comparison chart of the recommended performance of each model under the HR indicator;

[0046] Figure 7 It is the influence of the hyperparameter λ on ACFA under the NDCG and HR indicators;

[0047] Figure 8 It is the influence of the hyperparameter ε on ACFA under the NDCG and HR indicators; Specific implementation manner

[0048] The following elaborates on the preferred embodiments of the present invention in conjunction with the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making the protection scope of the present invention more clearly defined.

[0049] Refer to Figures 1 to 8 , a collaborative filtering recommendation method based on aesthetic features and adversarial learning, comprising the following steps:

[0050] Step 1: Extract the CNN feature f CNN and the AES feature f AES from the product dataset respectively, where the AES feature is the aesthetic feature;

[0051] Step 2: Concatenate the CNN feature and the AES feature to obtain a new feature matrix denoted as where the CNN feature f CNN and the AES feature f AES are respectively extracted from the product dataset through the CNN convolutional neural network and the BDN aesthetics network;

[0052] Step 3: Add aesthetic features on the basis of collaborative filtering, define the proposed original CFA model, denoted by , after adding aesthetic features, the definition of the CFA model without adding adversarial perturbations is expressed as:

[0053]

[0054] where M is the user preference matrix, U represents the user embedding matrix, V represents the product embedding matrix, F is the feature matrix, p and q respectively represent the user and the product, is obtained based on the latent features, V captures the user preference information, while is obtained based on the picture features, M is the encoding of the user preference, and the construction of the feature matrix F *q has been given in step 2;

[0055] Since the CFA model uses the typical Bayesian personalized ranking optimization criterion to optimize the implicit feedback of users, the BPR loss is used as the corresponding objective function, which is expressed as follows:

[0056]

[0057] Among them, D is the training set, Θ is the parameter of the model, p, q, and q′ respectively represent the user, the products purchased by this user, and the products not purchased by this user, and λ Θ represents the coefficient corresponding to each parameter Θ, ∥·∥ is the L2 norm, where is expressed as:

[0058]

[0059] Step 4: Determine the target and addition position of the adversarial perturbation to improve the accuracy and precision of the algorithm, and at the same time minimize the complexity of the model training process. Since the first important process of adversarial learning is to construct a perturbation that maximizes the BPR loss (the worst recommendation effect), the introduced adversarial perturbation is determined as:

[0060] Δ adv = argmax Δ,∥Δ∥≤ε L′ BPR (D|Θ)

[0061] Among them, L′ BPR (D|Θ) = L BPR (D|Θ + Δ), D is the training set, Θ is the parameter of the model, ε ≥ 0 is a hyperparameter controlling the perturbation amplitude, ∥·∥ is the L2 norm, and it is chosen to add the perturbation to the model parameters. By adding the perturbation to the model parameters, on the one hand, it can avoid the cross-influence between the network for extracting image features and the model training process, improving the accuracy and precision of the algorithm, and on the other hand, it can avoid excessive complexity during the model training process, enhancing the efficiency of the algorithm;

[0062] Step 5: Define the proposed adversarial perturbation ACFA model, denoted by . After determining the position where the adversarial perturbation is added, the adversarial perturbation model ACFA is defined as:

[0063]

[0064] Among them, Δ * ∈R K represents the adversarial perturbation added to the aesthetic feature matrix;

[0065] Step 6: Model training: Model parameter learning is achieved by minimizing the objective function to obtain the optimal model parameters. During the model training process, for an instance (p, q, q′), it is necessary to update the model parameters by minimizing the corresponding objective function under the condition of a determined adversarial perturbation. Because the relevant objective function is fixed, the model parameter learning at this time can be regarded as a minimization problem, which can be approximated as gradient descent, as shown below:

[0066]

[0067] Where Θ = {U, V, M}, η represents the learning rate, and the Adagrad optimizer in Tensorflow is selected to participate in model parameter learning, which can overcome the limitation of SGD that cannot adaptively adjust the learning rate;

[0068] Step 7: Adjust the two perturbation hyperparameters ε and λ in the model to achieve optimal model performance. The addition of adversarial perturbations introduces two additional hyperparameters ε and λ to control the perturbation amplitude (i.e., the perturbation size) and the influence of the loss function generated by the adversarial perturbation on the total loss function. The values of ε and λ are changed respectively. After each change, the model is retrained. The hyperparameter values corresponding to the optimal model performance are used as the final values.

[0069] Step 8: Select appropriate recommendation system evaluation indicators, such as HR and NDCG, as the basis for evaluating model performance;

[0070] Step 9: Record and organize the experimental results, compare them with the baseline model, and draw conclusions.

[0071] In this embodiment, the steps of the collaborative filtering recommendation method based on aesthetic features and adversarial learning are followed, and the flow chart is as follows: Figure 1 This method belongs to the category of personalized recommendation. Figure 2 This is a diagram of the functional modules of the personalized recommendation system.

[0072] In the feature collection stage, the Amazon dataset is used, and the extraction process of CNN features and AES features and their differences are as follows: Figure 3 As shown in the figure, after feature extraction, a .txt file is generated, which contains both CNN features and AES features.

[0073] During the model training phase, the framework diagram of the ACFA model is as follows Figure 4As shown, the algorithm involved is implemented based on the TensorFlow framework and uses the Python language. Due to different versions of Python, some functions used will also be different. The computer performance or configuration will also affect the time consumed in the algorithm training process. The selected dataset is randomly divided into three parts, namely the training dataset (accounting for 80% of the total quantity), the validation dataset (accounting for 10% of the total quantity), and the test dataset (accounting for 10% of the total quantity). Among them, the training set is used to train the model parameters, the validation set is used to adjust the hyperparameters, and the test set is used for performance comparison. Figure 5 and Figure 6 respectively show the comparison of the recommendation performance between our model and the comparative models under the NDCG and HR metrics. As can be seen from the figure, compared with multiple comparative models, the recommendation performance of our ACFA model has been improved, indicating the effectiveness of the present invention.

[0074] During the process of adjusting the model hyperparameters, first, study the influence of the regularization hyperparameter λ on the model. Fix the hyperparameter ε that controls the perturbation amplitude at 0.01, continuously change the value of the hyperparameter λ, and find a value that makes the model performance reach the best. Figure 7 shows the influence of the hyperparameter λ on ACFA under the NDCG and HR metrics; secondly, study the influence of the hyperparameter ε that controls the perturbation amplitude on the model. Fix the regularization hyperparameter λ at the optimal value, continuously change the value of the hyperparameter ε, and also find a value that makes the model performance reach the best. Figure 8 shows the influence of the hyperparameter ε on ACFA under the NDCG and HR metrics. Finally, fix the two optimal values as the final values of the adversarial perturbation hyperparameters.

[0075] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A collaborative filtering recommendation method based on aesthetic features and adversarial learning, characterized in that, The method includes the following steps: Step 1: Extract CNN features and AES features from the product dataset separately. The AES features are aesthetic features. and AES features , where the AES features are aesthetic features; Step 2: Concatenate the CNN features and the AES features to obtain a new feature matrix F q ; Step 3: Add aesthetic features on the basis of collaborative filtering to define the proposed original CFA model, denoted by ; Step 4: Determine the target and addition location of adversarial perturbations to improve the accuracy and precision of the algorithm while minimizing the complexity of the model training process; Step 5: Define the proposed adversarial perturbation ACFA model, denoted by ; Step 6: Model training, where model parameter learning is achieved by minimizing the objective function to obtain the optimal model parameters; Step 7: Adjust the two hyperparameters regarding perturbations in the model and to optimize the model performance; Step 8: Select the evaluation metrics of the recommendation system as the basis for evaluating the model performance; Step 9: Record and organize the experimental results, compare with the baseline model, and form a conclusion; In the said step 2, the obtained new feature matrix is expressed as , where the CNN features and the AES features are respectively extracted from the product dataset through the CNN convolutional neural network and the BDN aesthetics network; In step 3, after adding aesthetic features, the definition of the CFA model without adversarial perturbations is shown in (1): （1） Among them, M is the user preference matrix, U represents the embedding matrix of users, V represents the embedding matrix of products, F is the feature matrix, p and q represent users and products respectively, is obtained based on latent features, V captures the preference information of users, while is obtained based on image features, M is the encoding of user preferences, and the construction of the feature matrix has been given in step 2; In step 3, since the CFA model uses the typical Bayesian personalized ranking optimization criterion to optimize the implicit feedback of users, the BPR loss is used as the corresponding objective function, as shown in (2): （2） Among them, D is the training set, are the parameters of the model, p , q , q’ respectively represent the user, the products purchased by the user, and the products not purchased by the user, represents each parameter corresponding coefficient, is L 2 norm, where is expressed as: （3） In step 4, since the first important process of adversarial learning is to construct perturbations that can maximize the BPR loss, the introduced adversarial perturbations are determined as: （4） Among them, , D is the training set, are the parameters of the model, is the hyperparameter that controls the perturbation amplitude, is L 2 norm; and choose to add perturbations to the model parameters. By adding perturbations at the model parameters, on the one hand, it can avoid the cross - influence between the network for extracting image features and the model training process, improve the accuracy and precision of the algorithm, and on the other hand, it can avoid excessive complexity during the model training process and improve the efficiency of the algorithm; In step 5, after determining the location where the adversarial perturbations are added, the adversarial perturbation model ACFA is defined as shown in (5): （5） Among them, represents the adversarial perturbation added to the aesthetic feature matrix; In step 6, during the model training process, for an instance ( p , q , q’ ), under the condition that the adversarial perturbation has been determined, the update of the model parameters needs to be realized by minimizing the corresponding objective function. Since the relevant objective function remains fixed, the learning of the model parameters at this time is regarded as a problem of finding the minimum value, and it can be approximated as gradient descent, as shown in (6): （6） Among them, , represents the learning rate. The Adagrad optimizer in TensorFlow is selected to participate in the learning of model parameters, which can overcome the limitation that SGD cannot adaptively adjust the learning rate.

2. The collaborative filtering recommendation method based on aesthetic features and adversarial learning according to claim 1, wherein In step 7, the addition of adversarial perturbations introduces two additional hyperparameters and to control the perturbation amplitude, i.e., the perturbation size, and the influence of the loss function generated by the adversarial perturbation on the total loss function, respectively. The values of and will be changed respectively. After each change of the value, the model will be retrained, and the hyperparameter values corresponding to the best model performance will be used as the final values.

Images