A global factor decomposition multi-feature fusion network interest prediction method

Abstract

The application discloses a kind of global factor decomposition multi-feature fusion network interest prediction method, including the following steps: S1: user text information and project text information are collected, and embedding layer feature matrix is generated;S2: user global feature and project global feature are generated;S3: the embedding layer feature matrix of redistribution is calculated;S4: the embedding layer feature matrix of redistribution is mixed interaction, and interaction feature vector is obtained;S5: interaction feature vector and embedding layer feature matrix are spliced, and obtain fusion vector;S6: fusion vector is used as the input of deep neural network, trains deep neural network, and uses the interest prediction value of trained deep neural network to obtain.This multi-feature fusion network interest prediction method considers global feature and high-low order interaction, increases the global awareness of feature, increases the weight of important feature and reduces the weight of unimportant feature;The precision of model in interest prediction task is improved.

Description

Technical Field

[0001] This invention belongs to the field of interest prediction technology, specifically relating to a multi-feature fusion network interest prediction method based on global factor decomposition. Background Technology

[0002] In an era of information overload, an effective recommendation system can not only facilitate users' information-seeking process but also enhance customer loyalty and increase company profits. Interest prediction, as one of the most crucial tasks of a recommendation system, estimates the probability of a user clicking on a recommended item and determines whether that item will be recommended to the user. Therefore, the accuracy of interest prediction is a key factor affecting user experience and platform revenue.

[0003] Currently, existing recommender system methods fall into three main categories: collaborative filtering, which utilizes user-item interaction data; content recommendation, which maximizes the use of user-item related content; and sequence recommendation, which mines contextual information related to user behavior. The ultimate goal of all these methods is to recommend content that users are interested in. The key to a successful recommender system is the accuracy of the recommended content and whether it meets the user's requirements. Summary of the Invention

[0004] The purpose of this invention is to address the current research's lack of awareness of global feature interaction, and to propose a multi-feature fusion network interest prediction method based on global factor decomposition.

[0005] The technical solution of this invention is: a multi-feature fusion network interest prediction method based on global factor decomposition, comprising the following steps:

[0006] S1: Collect user text information and project text information, and generate an embedding layer feature matrix;

[0007] S2: Generate user global features and project global features based on the embedding layer feature matrix;

[0008] S3: Calculate the redistributed embedding layer feature matrix based on user global features, project global features, and embedding layer feature matrix;

[0009] S4: Perform a hybrid interaction on the redistributed embedding layer feature matrix to obtain the interaction feature vector;

[0010] S5: Concatenate the interaction feature vector and the embedding layer feature matrix to obtain the fused vector;

[0011] S6: Use the fused vector as input to the deep neural network, train the deep neural network, and use the trained deep neural network to obtain the interest prediction value.

[0012] The beneficial effects of this invention are: the multi-feature fusion network interest prediction method proposed in this invention takes into account the interaction between global features and high and low order features, increases the global awareness of features, increases the weight of important features and reduces the weight of unimportant features; thus improving the accuracy of the model in the interest prediction task.

[0013] Further, in step S1, the embedding layer feature matrix E includes the user's feature embedding vector U and the item's feature embedding vector I, and its expression is:

[0014] E = [U, I] = [e1, e2, ..., e m ,..,e m+n ]

[0015] In the formula, e1,...,e m Let m be the user embedding features, and e be the user embedding features. m+1 ,...,e m+n This represents the embedded features of m+n items.

[0016] Furthermore, in step S2, the user's global feature U g and project global features I g The expressions are as follows:

[0017]

[0018]

[0019] In the formula, e i Let e ​​represent the embedded feature of the i-th user. j This represents the embedded feature of the j-th item.

[0020] Furthermore, step S3 includes the following sub-steps:

[0021] S31: Use the user global features and project global features as input to the embedding layer, and calculate the mean of the embedding layer feature matrix;

[0022] S32: Calculate the weight vector of the embedding layer feature matrix based on the mean of the embedding layer feature matrix;

[0023] S33: Calculate the redistributed embedding layer feature matrix based on the weight vector of the embedding layer feature matrix.

[0024] Furthermore, in step S31, the mean z of the embedding layer feature matrix... i The calculation formula is:

[0025]

[0026] In the formula, e iLet represent the embedded feature of the i-th user, and k represent the embedding dimension of each feature;

[0027] In step S32, the formula for calculating the weight vector A of the embedding layer feature matrix is:

[0028] A = σ²(W²σ¹(W¹Z))

[0029] In the formula, Z represents the mean vector of the feature matrix of the embedding layer, σ1(·) represents the first activation function, σ2(·) represents the second activation function, W1 represents the first learning parameter, and W2 represents the second learning parameter;

[0030] In step S33, the formula for calculating the redistributed embedding layer feature matrix V is:

[0031] V = [a1·e1,…,a] p ·e p ] = [v1,…,v p}

[0032] In the formula, E represents the feature matrix of the embedding layer, a1,…,a p Represents the weighting factors, e1,…,e p Represents the embedded features, v1,…,v p This represents the feature vector of the redistributed embedding layer.

[0033] Furthermore, in step S4, the interaction feature vector p ij The calculation formula is:

[0034] p ij =v i ·W⊙v j

[0035] In the formula, v1,…,v j Let W represent the feature vector of the redistributed embedding layer, and let W represent the parameter matrix.

[0036] Furthermore, in step S6, the expression for the loss function Logloss of the deep neural network is:

[0037]

[0038] In the formula, N represents the size of the training set, and y h The label represents the training sample h. This represents the output of a deep neural network. Attached Figure Description

[0039] Figure 1 A flowchart of a multi-feature fusion network interest prediction method;

[0040] Figure 2 A schematic diagram of category feature embedding;

[0041] Figure 3 This is a schematic diagram of a compression excitation network;

[0042] Figure 4 A schematic diagram of the feature interaction calculation method. Detailed Implementation

[0043] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0044] like Figure 1 As shown, this invention provides a multi-feature fusion network interest prediction method based on global factorization, comprising the following steps:

[0045] S1: Collect user text information and project text information, and generate an embedding layer feature matrix;

[0046] S2: Generate user global features and project global features based on the embedding layer feature matrix;

[0047] S3: Calculate the redistributed embedding layer feature matrix based on user global features, project global features, and embedding layer feature matrix;

[0048] S4: Perform a hybrid interaction on the redistributed embedding layer feature matrix to obtain the interaction feature vector;

[0049] S5: Concatenate the interaction feature vector and the embedding layer feature matrix to obtain the fused vector;

[0050] S6: Use the fused vector as input to the deep neural network, train the deep neural network, and use the trained deep neural network to obtain the interest prediction value.

[0051] This invention expands the features of the embedding layer by adding user global features and project global features. By splicing the explicit feature interaction input of the width module and the feature embedding of the depth module, a multi-feature fusion network based on global factor decomposition is proposed to more comprehensively mine effective feature interaction information and improve the accuracy of interest prediction.

[0052] To enrich the embedding layer's features, user-global and project-global features were added. Increasing global features enhances the global awareness of feature interactions. Beyond simply adding global features, local features can also be added to improve model performance. Local features can be those that play a crucial role in different recommendation applications. Secondly, information from the width model and the depth model is fused. The width model's input is obtained by dynamically assigning feature weights through a compressed excitation network based on the original embedding features (including global features) and then using a vector-based hybrid interaction method. The depth model's input is the original embedding features (excluding global features). The information from the width model and the depth model is then concatenated and fed into a multilayer perceptron to explore their interactions. Finally, the output of the global factorization layer (width model) and the output of the last layer of the multilayer perceptron are used for prediction using logistic regression.

[0053] In this embodiment of the invention, in step S1, the embedding layer feature matrix E includes the user's feature embedding vector U and the item's feature embedding vector I, and its expression is:

[0054] E = [U, I] = [e1, e2, ..., e m ,..,e m+n ]

[0055] In the formula, e1,...,e m Let m be the user embedding features, and e be the user embedding features. m+1 ,...,e m+n This represents the embedded features of m+n items.

[0056] User and project features are obtained through relevant textual information. In the process of extracting user and project features from textual information, the data exhibits sparsity and high dimensionality, meaning that a feature vector has multiple zero factors, and the number of zero factors is greater than the number of non-zero factors. Furthermore, the obtained user and project features have weak temporal and spatial correlations. Therefore, a feature embedding layer is introduced to map these feature data into a low-dimensional and dense feature vector.

[0057] Specifically, the embedding layer features are represented by a low-dimensional vector, i.e.:

[0058] e i =V i x i

[0059] Among them, V i It is a sparse input feature x i The embedding matrix, x i It can be represented by one-hot codes or multiple one-hot code vectors.

[0060] Figure 2This refers to the overall process of sparse input features and sparse feature embedding. After passing through the embedding layer, user features constitute the user's feature embedding vector U = [e1, e2, ..., e...]. m The feature embedding vector of the project is I = [e m+1 ,e m+2 ,...,e m+n Therefore, the feature matrix of the embedding layer is a feature matrix E = [U, I] = [e1, e2, ..., e] that includes both item and user features. m ,..,e m+n ].

[0061] In this embodiment of the invention, in step S2, in the recommendation domain, to increase the global awareness of feature interactions, user global features and item global features are added. User global feature U g and project global features I g The expressions are as follows:

[0062]

[0063]

[0064] In the formula, e i Let e ​​represent the embedded feature of the i-th user. j This represents the embedded feature of the j-th item.

[0065] To avoid the feature product producing extremely small values, the minimax method is used to normalize the user global features and the project global features to the range [0,1], that is:

[0066] U gf =(U g -min(U g )) / (max(U g )-min(U g ))

[0067] I gf =(I g -min(U g )) / (max(I g )-min(I g ))

[0068] After obtaining the normalized user global features and project global features, the user global features, project global features, and embedded layer features are combined into a feature matrix. Global factor decomposition is then used to obtain the interaction between first-order and second-order display features, thereby obtaining the user's relevance to different project features. The higher the relevance value, the higher the user's attention to the project feature.

[0069] Therefore, the global factorization machine model includes first-order linear feature interactions and second-order linear feature interactions, and its mathematical expression is:

[0070]

[0071] Where, w∈R d and V i ∈R k .<w,x> Reflects first-order linear characteristics. This represents the impact of second-order linear feature interactions.

[0072] In this embodiment of the invention, step S3 includes the following sub-steps:

[0073] S31: Use the user global features and project global features as input to the embedding layer, and calculate the mean of the embedding layer feature matrix;

[0074] S32: Calculate the weight vector of the embedding layer feature matrix based on the mean of the embedding layer feature matrix;

[0075] S33: Calculate the redistributed embedding layer feature matrix based on the weight vector of the embedding layer feature matrix.

[0076] In this embodiment of the invention, in step S31, the mean z of the embedding layer feature matrix... i The calculation formula is:

[0077]

[0078] In the formula, F sq (·) represents z i The function expression, e i Let represent the embedded feature of the i-th user, and k represent the embedding dimension of each feature;

[0079] In step S32, the formula for calculating the weight vector A of the embedding layer feature matrix is:

[0080] A = F ex (Z)=σ2(W2σ1(W1Z))

[0081] In the formula, F ex (·) represents the functional expression of A, Z represents the mean vector of the embedding layer feature matrix, σ1(·) represents the first activation function, σ2(·) represents the second activation function, W1 represents the first learning parameter, and W2 represents the second learning parameter;

[0082] In step S33, the formula for calculating the redistributed embedding layer feature matrix V is:

[0083] V = F ReWeight (A,E)=[a1·e1,…,ap ·e p ] = [v1,…,v p ]

[0084] In the formula, F ReWeight (·) represents the functional expression of V, E represents the feature matrix of the embedding layer, and a1,…,a p Represents the weighting factors, e1,…,e p Represents the embedded features, v1,…,v p This represents the feature vector of the redistributed embedding layer.

[0085] In the recommendation field, different recommendation tasks focus on two key points: one is the richness of features, which refers not only to features of individual data categories but also to their combinations; that is, in addition to focusing on the quantity of features, it is also necessary to explore the inherent relationships between features. The other is how to assign higher weights to more important features. Therefore, a dynamic approach to assigning feature weights is adopted.

[0086] Introducing user-global and project-global features introduces a global awareness of both users and the project. The lack of consideration for incorporating such global features led to suboptimal model performance. Therefore, introducing global features improves the model's expressive power.

[0087] The specific process of dynamically assigning weights to the feature matrix is ​​as follows: In addition to the original category features, user global features and item global features are added as inputs to the embedding layer. Then, the weights in the feature matrix are dynamically assigned through the compression activation network layer to obtain a feature matrix of the same size as the input. This matrix is ​​a redistributed feature matrix, assigning different weights to different features.

[0088] like Figure 3 As shown, the compression activation network consists of three parts: squeezing, activation, and redistribution. First, squeezing summarizes the embedding layer features and cross features, resulting in compressed information. Information compression can be achieved by finding the maximum, minimum, or mean value, i.e., assuming the feature matrix E = [e1, e2, ..., e...]. h The information contained therein is obtained by taking the average value. (Formula) In this context, the mean z of the features is used. i This is because the mean can encompass the overall information of the feature. Secondly, the activation part involves obtaining an m-dimensional weight vector through a fully connected layer, representing the weight of each feature. ReLU is used as the activation function during the calculation. Finally, the redistribution part amplifies the obtained weights according to the feature embedding ratio and then multiplies them to obtain a new embedded feature vector, thus completing the dynamic allocation of feature weights.

[0089] In this embodiment of the invention, in step S4, the interactive feature vector p ij The calculation formula is:

[0090] p ij =v i ·W⊙v j

[0091] In the formula, v1,…,v j Let W represent the feature vector of the redistributed embedding layer, and let W represent the parameter matrix.

[0092] like Figure 4 As shown, the explicit interaction features employ a hybrid interaction combining inner product interaction and Hadamard product interaction.

[0093] Hybrid interaction is essentially a form of implicit interaction, but it combines explicit and implicit interactions. Therefore, it employs a hybrid interaction approach that simultaneously integrates explicit and implicit feature interactions. Specifically, inner product interaction is represented as:

[0094]

[0095] R x ={(i,j)} i∈{1,…,p},j∈{1,…,p},j>i

[0096] Among them, v i Let be the embedding vector of the i-th feature domain, and · be the inner product.

[0097] The Hadamard product interaction is represented as:

[0098]

[0099] R x ={(i,j)} i∈{1,…,p},j∈{1,…,p},j>i

[0100] Among them, v i Let be the embedding vector of the i-th feature domain. For Hadama accumulation.

[0101] Hybrid interaction is represented as:

[0102] p ij =v i ·W⊙v j

[0103] Where W∈R k×k v i v j ∈R k , 1≤i≤m, 1≤j≤mv i v j These represent the embedding vectors of the categorical features. W∈R k×kThese are the parameters of the feature embeddings for all categories, i.e., W∈R k×k It is a parameter matrix.

[0104] The feature combination layer combines the redistributed features obtained through the compressed activation network with the interaction feature vector P = [p1, p2, ..., p] obtained through hybrid interaction. k The feature matrix of the embedding layer, Q = E = [U, I] = [e1, e2, ..., e], is obtained through the feature vector mapping of the embedding layer. m ,..,e m+n Perform concatenation. The final fusion vector G = [P, Q] = [g1, g2, ..., g k+s ].

[0105] In this embodiment of the invention, in step S6, the expression for the loss function Logloss of the deep neural network is:

[0106]

[0107] In the formula, N represents the size of the training set, and y h The label represents the training sample h. This represents the output of a deep neural network.

[0108] A deep network is a feedforward neural network used to learn implicit high-order feature interactions. The concatenated interaction feature vectors are input into the neural network. A deep network can be expressed in the following form:

[0109] z1=σ1(W1f(G1)+b1)

[0110] z2=σ2(W2z1+b2)

[0111] ...

[0112] z l =σ l (W l z l-1 +b l )

[0113] Where l is the number of hidden layers, W l b l and σ l These represent the parameter matrix, bias vector, and activation function of the l-th layer, respectively. By specifying non-linear activation functions, such as Sigmoid, Tanh, and ReLU, this network learns higher-order latent feature interactions in a non-linear manner. These latent higher-order interactions can learn implicit associations between features, improving the model's generalization ability. Furthermore, the network depth and layer size can be easily adjusted to optimize model performance.

[0114] Next is the combination of deep and shallow networks. This combination ensures both the generalization and memorization capabilities of the model. The shallow network is the part responsible for global factorization, while the deep network combines the compressed activation network and the deep network. The final interest prediction value is obtained by concatenating the outputs of these two networks.

[0115] In summary, the overall formula for the output of the proposed model is:

[0116]

[0117] Among them, Y e Y represents the output of the deep model. i This represents the output of the width model, where σ is a sigmoid function.

[0118] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1.A global factor decomposition based multi-feature fusion network for interest prediction, characterized in that, The method comprises the following steps: S1: collecting user text information and item text information, and generating an embedding layer feature matrix; S2: generating user global features and item global features according to the embedding layer feature matrix; User global features U g and item global features I g are respectively: wherein e i denotes the i user embedding feature, e j denotes the j item embedding feature, m denotes the m user embedding feature, and n denotes the n item embedding feature; S3: calculating a re-allocated embedding layer feature matrix according to the user global features, the item global features and the embedding layer feature matrix; The step S3 comprises the following sub-steps: S31: taking the user global features and the item global features as inputs of the embedding layer, and calculating a mean value of the embedding layer feature matrix; S32: calculating a weight vector of the embedding layer feature matrix according to the mean value of the embedding layer feature matrix; S33: calculating the re-allocated embedding layer feature matrix according to the weight vector of the embedding layer feature matrix; S4: performing mixed interaction on the re-allocated embedding layer feature matrix to obtain an interaction feature vector; S5: splicing the interaction feature vector and the embedding layer feature matrix to obtain a fusion vector; S6: taking the fusion vector as an input of a deep neural network, training the deep neural network, and obtaining an interest prediction value by using the trained deep neural network. 2.The global factor-decomposed multi-feature fusion network interest prediction method of claim 1, wherein, In the step S1, the embedding layer feature matrix E comprising a feature embedding vector of a user U and a feature embedding vector of an item I whose expression is: wherein e 1,..., e m denote m user embedding features, e m+1 ,..., e m+n denote n item embedding features. 3.The global factor-decomposed multi-feature fusion network interest prediction method of claim 1, wherein, In the step S31, the mean value of the embedding layer feature matrix is embedded z i The calculation formula is: wherein, e i represents the number of users, i k represents the embedding dimension of each feature.​ In the step S32, the weight vector of the embedding layer feature matrix A The calculation formula is: wherein Z denotes a mean vector of the embedding layer feature matrix, σ 1(•) denotes a first activation function, σ 2(•) denotes a second activation function, W 1 denotes a first learning parameter, W 2 denotes a second learning parameter; In the step S33, the re-distributed embedded layer feature matrix V The calculation formula is: wherein a 1, …, a p denotes a weight factor, e 1, …, e p denotes an embedding feature, v 1, …, v p denotes a re-distributed embedding layer feature vector. 4.The global factor-decomposed multi-feature fusion network interest prediction method of claim 1, wherein, In the step S4, the interaction feature vector p ij The calculation formula is: In the formula, v 1,… v j represents a re-distributed embedded layer feature vector, W represents a parameter matrix. 5.The global factor-decomposed multi-feature fusion network interest prediction method of claim 1, wherein, In the step S6, the loss function of the deep neural network Logloss The expression is: wherein N denotes the training set size, y h denotes the training sample h label, denotes the output of the deep neural network.

Images