FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection

Abstract

The invention relates to a FgFisNET agricultural product recommendation method for fine-grained feature interactive selection, which is used for fine-grained feature interaction and important feature interaction selection. According to the method, firstly, effective feature interaction is learned by combining an inner product and a hadamard product, and effective modeling is carried out on feature interaction in a sparse data set; then introducing a group of architecture parameters for each group of feature interaction, learning the relative importance of each feature through gradient descent, deleting unimportant feature interaction, reserving the architecture parameters as attention units to help learning of the model, and finally performing CTR prediction of agricultural product e-commerce by using the trained model. According to the method, fine-grained feature interaction and important feature interaction selection are fused, and the performance of the model is improved. The effectiveness of the method is verified through a real agricultural product e-commerce platform, and remarkable economic value is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning recommendation algorithms, and specifically is a fine-grained feature interactive selection FgFisNET agricultural product recommendation method. Background Art

[0002] With the development of science and technology, the rise of agricultural e-commerce websites has become a new outlet to drive agricultural product consumption in the future. Digital transformation and innovation are new hot spots in the development of agricultural product e-commerce. In order to provide users with a better shopping experience, e-commerce platforms use recommendation technology to provide users with personalized recommendations, reduce product overload, and provide users with a variety of choices.

[0003] CTR prediction is crucial in the recommendation system of agricultural products e-commerce, whose task is to predict the probability of users clicking on recommended items, and then make many recommendation decisions based on the predicted click-through rate. Many models have been proposed in this field, such as logistic regression (LR), polynomial 2 (poly2), matrix factorization (MF), factorization machine (FM), gradient boosted decision tree (GBDT). With the great success of deep learning in many research fields such as computer vision and natural language processing, many CTR models based on deep learning have been proposed in recent years, such as neural network (FNN) with factorization machine, Wide&Deep, attention factorization machine (AFM), DeepFM, XDeepFM. The core of these recommendation algorithms is to extract important low-order and high-order feature interactions. However, in all these models, feature interactions are either simply enumerated by all features or manually designed to identify important features. Enumerating all feature interactions brings a lot of memory and computational costs, and useless interactions may introduce noise and complicate training. At the same time, manually identifying important interactions is labor-intensive and may miss some counterintuitive interactions. Moreover, many CTR prediction works calculate feature interactions in a simple way, such as inner product and Hadamard product, which cannot effectively model feature interactions in sparse datasets. Summary of the invention

[0004] In view of the shortcomings of feature interaction methods and feature interaction selection in existing recommendation algorithms, the main technology solved by the present invention is to use product attributes, user attributes and date as input features of the model, adopt a feature interaction method of combining inner product and Hadamard product, and use joint optimization and unified training to select important feature interactions for training, thereby improving the performance of the model.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0006] A FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection includes the following steps:

[0007] 1) Obtain user behavior log data from the agricultural products e-commerce platform, clean it, and divide the cleaned data into training sets and test sets;

[0008] 2) Encode the discrete features in the log data to obtain sparse vector features;

[0009] 3) Embed the sparse vector features and then concatenate them with the numerical features in the log data as the input vector of the FgFisNET model;

[0010] 4) Use fine-grained products to perform fine-grained feature interactions on input vectors;

[0011] 5) Assign a weight to each feature interaction between the feature interaction layer and the deep neural network in the FgFisNET model, and perform fine-grained feature interaction selection based on the weight;

[0012] 6) Input the fine-grained feature interaction selection results into the deep neural network to capture implicit high-order feature interactions, generate dense real-valued feature vectors, input them into the sigmoid function, and finally merge the logistic regression part to obtain the predicted value of the click rate;

[0013] 7) Sort the predicted click rates from high to low, and recommend the top several agricultural products.

[0014] In the step 1), data cleaning specifically includes: using the mode to fill in the behavior data with missing values; using the median to replace the outlier data; and adjusting the inconsistent data according to the same features recorded in other fields to make them consistent.

[0015] The step 2) is specifically as follows: using one-hot encoding for each discrete feature to convert each feature into a sparse 01 vector representation.

[0016] The step 4) is specifically: combining the inner product and outer product of the features, and sharing a matrix for each feature interaction:

[0017] p ij =v i w⊙v j

[0018] Among them, p ij is the result of fine-grained feature interaction, v i is the embedding vector of the ith domain, v j is the j-th domain embedding vector, w is the weight matrix corresponding to the interaction between the ith domain embedding vector and the j-th domain embedding vector, and ⊙ is the Hadamard product.

[0019] In step 5), the fine-grained feature interactive selection includes the following steps:

[0020] Joint optimization stage: Through gradient descent, the weights of all feature cross terms are trained using the RDA optimizer, and the FgFisNET model parameters are trained using the Adam optimizer;

[0021] Unified training phase: Select features with weights greater than 0 generated in the joint optimization phase for interaction, and store the interaction results in the FgFisNET model as an Attention mechanism, and train them again along with the model parameters using the Adam optimizer.

[0022] In the joint optimization stage, the specific updating method of the weight ∝ is:

[0023]

[0024] Among them, t is the global step, Z is the input data, c and u are hyperparameters, ∝0 is the initial parameter vector, γ is the learning rate, is the loss function with respect to the parameter vector ∝ t The gradient of the current data sample Z t+1 The cumulative sum of t is the parameter vector of the tth iteration step, Z t+1 is the current data sample.

[0025] In the unified training stage, the feature interaction is specifically as follows:

[0026]

[0027] Where m is the number of embedded output domains, and G is a gate switch with a value of 0 or 1.

[0028] In step 6), capturing implicit high-order feature interactions is specifically as follows: let a(0)=[v1, v2, ..., vn] represent the output of fine-grained feature interactions, n represents the number after feature interaction selection, input a(0) into a deep neural network, perform feedforward learning, and generate a dense real-valued feature vector. The feedforward learning process is:

[0029] a(l)=σ(w(l)a(l-1)+b(l)

[0030] Where l is the depth of the neural network, σ is the activation function, and w(l), b(l), and a(l) are the model weights, biases, and outputs of the lth layer, respectively.

[0031] In step 6), the predicted value of the click rate is obtained by using the sigmoid function, specifically: the generated dense real-valued feature vector is input into the sigmoid function to predict the click rate, and the logistic regression part is combined. The prediction expression is:

[0032]

[0033] Among them, the click rate prediction value σ is the activation function, m is the feature length, w i is the weight of the linear part, w0 is the bias term, x i is the i-th feature in the input feature vector, a (l) To represent the output of the neural network layer l, b (l+1 ) is the bias term of the l+1th layer of the neural network.

[0034] The FgFisNET model minimizes the cross entropy between the predicted value and the click label by training the objective function, and the objective function loss is:

[0035]

[0036] Among them, y i is the true label of the i-th instance, is the predicted value of click rate, and N is the total length of the sample.

[0037] The present invention has the following beneficial effects and advantages:

[0038] 1. The present invention helps to effectively model feature interactions in sparse data sets;

[0039] 2. The present invention is conducive to improving the performance of the model learning high-order implicit feature interaction;

[0040] 3. Accurate predictions of agricultural products that users are likely to click on help provide users with a better shopping experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the present invention;

[0042] Figure 2 It is a fine-grained feature interaction structure diagram of the present invention;

[0043] Figure 3 Interactive selection of structural diagrams for features of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0045] In response to the problems in CTR prediction of current recommendation algorithms, this paper proposes a fine-grained feature interaction selection FgFisNET agricultural product e-commerce recommendation algorithm. Fine-grained feature interaction can effectively model feature interactions in sparse data sets; selecting important feature interactions and deleting redundant feature interactions can help reduce the complexity of calculations and improve the performance of the model, thereby making the model prediction more accurate.

[0046] like Figure 1 As shown in FIG, the agricultural product recommendation algorithm based on fine-grained feature interactive selection includes the following steps:

[0047] Step 1: Obtain user behavior logs of the agricultural product e-commerce platform, including product attribute information, user attribute information, date, and click tags;

[0048] Step 2: Clean the obtained agricultural product e-commerce user behavior logs and then divide them into training sets and test sets;

[0049] Step 3: Encode discrete features in product attributes and user attributes;

[0050] Step 4: Then embed the high-dimensional sparse vectors in the encoded product attributes and user attributes, and map the high-dimensional sparse vectors into low-dimensional dense vectors. Then concatenate the original numerical features and use them as the input vector of FgFisNET;

[0051] Step 5: Use fine-grained products to perform fine-grained feature interactions on the input vectors and effectively model feature interactions in sparse datasets;

[0052] Step 6: Assign a weight w to each feature interaction between the feature interaction layer and the deep neural network (DNN), train the weights w and model parameters v of all feature interactions through the joint optimization phase, and select important feature interaction items based on the weight w.

[0053] Step 7: Through the unified training phase, delete the feature interactions with weight w equal to 0, select the feature interaction items with weight w greater than 0, and make the weight w exist in the model as an Attention mechanism, and train again with the model parameters to improve the performance of the model.

[0054] Step 8: The fine-grained feature interaction selection result is input into DNN, which implicitly captures high-order feature interactions and generates a dense real-valued feature vector, which is input into the sigmoid function and finally merged with the logistic regression part to obtain the click-through rate prediction output.

[0055] Step 9: Apply the proposed FgFisNET model to the agricultural products e-commerce recommendation platform to predict the products that users may click in the future.

[0056] The data cleaning specifically includes: filling the mode of the behavior data with missing values; replacing the abnormal values ​​with the median; and adjusting the inconsistent data recorded in other fields according to the same features to make them consistent.

[0057] The feature processing is specifically as follows: feature encoding, using one-hot encoding for each discrete feature to convert each feature into a sparse 01 vector representation; feature concatenation, concatenating all encoded features and numerical features together; feature normalization, for continuous features, reducing their numerical range to a smaller interval; feature embedding, using embedding technology to convert high-dimensional sparse 01 vectors into low-dimensional dense vectors.

[0058] like Figure 2 As shown, the fine-grained feature interaction is specifically: combining the inner product and the outer product, sharing a matrix for each feature interaction, and the calculation expression of the feature interaction is:

[0059] p ij =v i w⊙v j

[0060] like Figure 3 As shown, the feature interaction selection is specifically as follows: in the joint optimization stage, the weights ∝ of all feature cross terms are trained using the RDA optimizer through gradient descent, and the model parameters are trained using the Adam optimizer. The specific update method of the weight ∝ is as follows:

[0061]

[0062] Among them, t is the global step, Z is the input data, c and u are hyperparameters, ∝0 is the initial parameter vector, γ is the learning rate, is the loss function with respect to the parameter vector ∝ t The gradient of the current data sample Z t+1 The cumulative sum of t is the parameter vector of the tth iteration step, Z t+1 is the current data sample; in the unified training phase, through the joint optimization phase, some features with weights of 0 will be obtained. We directly discard these features and select feature interactions with weights greater than 0. The expression of pairwise feature crossover becomes:

[0063]

[0064] Where G is a gate switch, which takes a value of 0 or 1. At this time, the weight ∝ is no longer a feature selection parameter, but is stored in the model as an Attention mechanism, and is trained again with the Adam optimizer along with the model parameters.

[0065] The capture of implicit high-order feature interactions is specifically as follows: the input of the DNN is the output of the fine-grained feature interaction, let a(0) = [v1, v2, ..., vn] represent the output of the fine-grained feature interaction, and n represents the number after feature interaction selection. Then a(0) is input into the DNN, and the feedforward process expression is:

[0066] a(l)=σ(w(l)a(l-1)+b(l)

[0067] Where l is the depth of the neural network, σ is the activation function, w(l), b(l) and a(l) are the model weights, biases and outputs of the lth layer respectively, and finally a dense real-valued feature vector is generated.

[0068] The model prediction is specifically as follows: the generated dense real-valued feature vector is input into the sigmoid function to predict the click rate, and the logistic regression part is combined. The prediction expression is:

[0069]

[0070] Click-through rate prediction σ is the activation function, m is the feature length, w i is the weight of the linear part.

[0071] The objective function of the model training is specifically to minimize the cross entropy of the predicted value and the click label. The objective function expression is:

[0072]

[0073] where y i is the true label of the i-th instance, is the predicted value of click rate, and N is the total length of the sample.

[0074] Example Analysis 1:

[0075] Step 1 Data acquisition: Select 100,000 user behavior records from January 1, 2023 to July 1, 2023 of Liaoning Province Agricultural Product Quality and Origin Environment Information Online, including product attribute information, user attribute information, date, and click tags.

[0076] Step 2: Data cleaning: Use the mode to fill in the behavior data with missing values; use the median to replace the outliers; adjust the inconsistent data according to the same features in other fields to make them consistent. The cleaned data is divided into training set and test set, with user behavior data from January 1, 2023 to June 1, 2023 as the training set, and user behavior data from June 1, 2023 to July 1, 2023 as the test set.

[0077] Step 3: Feature processing: Use one-hot encoding for discrete features in the training set, converting each feature into a sparse 01 vector representation. Then use embedding technology to convert the high-dimensional sparse 01 vector into a low-dimensional dense vector. Finally, concatenate it with the numerical features in the training set to form the input vector of the model.

[0078] Step 4: Fine-grained feature interaction: Construct feature interaction terms for the input vector, combine inner and outer products to construct fine-grained products, and learn a matrix for each feature interaction term.

[0079] Step 5: Feature interaction selection: Using the gating mechanism, a weight w is assigned to each feature interaction between the feature interaction layer and the deep neural network (DNN), and feature interactions with non-zero weight w are selected.

[0080] Step 6: Capture implicit high-order feature interactions: Input the retained important feature interactions into the neural network to obtain implicit high-order feature crosses, and finally generate a dense real-valued feature vector.

[0081] Step 7: Model training and testing: Jointly optimize and uniformly train the designed FgFisNET model, and incorporate logistic regression to test and verify the model.

[0082] Step 8: Model performance evaluation: Use the area under the ROC curve and the coordinate axis (AUC) and cross entropy (logloss) as evaluation indicators and compare them with existing mainstream models.

[0083] The performance comparison of the above experiments and different mainstream recommendation algorithms is shown in Table 1:

[0084] Table 1

[0085] method Log loss AUC FM 0.3843 0.7788 FMM 0.3785 0.7809 AFM 0.3804 0.7817 DeepFM 0.3775 0.7831 FgFisNET 0.3764 0.7862 .

Claims

1. A FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection, characterized in that: The following steps are involved: 1) Obtain user behavior log data from the agricultural products e-commerce platform, clean it, and divide the cleaned data into training sets and test sets; 2) Encode the discrete features in the log data to obtain sparse vector features; 3) Embed the sparse vector features and then concatenate them with the numerical features in the log data as the input vector of the FgFisNET model; 4) Use fine-grained products to perform fine-grained feature interactions on input vectors; 5) Assign a weight to each feature interaction between the feature interaction layer and the deep neural network in the FgFisNET model, and perform fine-grained feature interaction selection based on the weight; 6) Input the fine-grained feature interaction selection results into the deep neural network to capture implicit high-order feature interactions, generate dense real-valued feature vectors, input them into the sigmoid function, and finally merge the logistic regression part to obtain the predicted value of the click rate; 7) Sort the predicted click rates from high to low, and recommend the top several agricultural products.

2. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: In the step 1), data cleaning specifically includes: using the mode to fill in the behavior data with missing values; using the median to replace the outlier data; and adjusting the inconsistent data according to the same features recorded in other fields to make them consistent.

3. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: The step 2) is specifically as follows: using one-hot encoding for each discrete feature to convert each feature into a sparse 01 vector representation.

4. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: The step 4) is specifically: combining the inner product and outer product of the features, and sharing a matrix for each feature interaction: p ij =in i w⊙v j Among them, p ij is the result of fine-grained feature interaction, v i is the embedding vector of the ith domain, v j is the j-th domain embedding vector, w is the weight matrix corresponding to the interaction between the ith domain embedding vector and the j-th domain embedding vector, and ⊙ is the Hadamard product.

5. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: In step 5), the fine-grained feature interactive selection includes the following steps: Joint optimization stage: Through gradient descent, the weights of all feature cross terms are trained using the RDA optimizer, and the FgFisNET model parameters are trained using the Adam optimizer; Unified training phase: Select features with weights greater than 0 generated in the joint optimization phase for interaction, and store the interaction results in the FgFisNET model as an Attention mechanism, and train them again along with the model parameters using the Adam optimizer.

6. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 5 is characterized in that: In the joint optimization stage, the specific updating method of the weight ∝ is: Among them, t is the global step, Z is the input data, c and u are hyperparameters, ∝0 is the initial parameter vector, γ is the learning rate, is the loss function with respect to the parameter vector ∝ t The gradient of the current data sample Z t+1 The cumulative sum of t is the parameter vector of the tth iteration step, Z t+1 is the current data sample.

7. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 5 is characterized in that: In the unified training stage, the feature interaction is specifically as follows: Where m is the number of embedded output domains, and G is a gate switch with a value of 0 or 1.

8. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: In step 6), capturing implicit high-order feature interactions is specifically as follows: let a(0)=[v1, v2, ..., vn] represent the output of fine-grained feature interactions, n represents the number after feature interaction selection, input a(0) into a deep neural network, perform feedforward learning, and generate a dense real-valued feature vector. The feedforward learning process is: a(l)=σ(w(l)a(l-1)+b(l) Where 1 is the depth of the neural network, σ is the activation function, and w(1), b(1), and a(1) are the model weights, biases, and outputs of the first layer, respectively.

9. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: In step 6), the predicted value of the click rate is obtained by using the sigmoid function, specifically: the generated dense real-valued feature vector is input into the sigmoid function to predict the click rate, and the logistic regression part is combined. The prediction expression is: Among them, the click rate prediction value σ is the activation function, m is the feature length, w i is the weight of the linear part, w0 is the bias term, x i is the i-th feature in the input feature vector, a (l) To represent the output of the neural network layer l, b (l+1) is the bias term of the l+1th layer of the neural network.

10. The FgFisNET agricultural product recommendation method based on fine-grained feature interactive selection according to claim 1 is characterized in that: The FgFisNET model minimizes the cross entropy between the predicted value and the click label by training the objective function, and the objective function loss is: Among them, y i is the true label of the i-th instance, is the predicted value of click rate, and N is the total length of the sample.

Images