A recommendation method fusing cross-network and feature importance selection
By integrating cross-network and feature importance selection recommendation methods, and utilizing SENet network and deep neural network to capture feature interaction information, the problem of insufficient feature interaction mining ability in recommendation models is solved, and more accurate and efficient recommendation results are achieved.
Patent Information
- Application Number
- CN202410649171.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-23
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2044-05-23
AI Technical Summary
Existing recommendation models have limited capabilities in feature interaction mining, resulting in monotonous recommendation results. Furthermore, traditional models cannot effectively and automatically learn a limited number of feature interactions, which affects recommendation performance.
We employ the SENet network for feature weighting and combine it with a deep neural network through a bilinear feature interaction layer and a parallel cross-network to capture feature interaction information. We then design a recommendation method that integrates the cross-network and feature importance selection.
This improved the performance and generalization ability of the recommendation model, reduced training costs, and enabled more accurate personalized recommendations.
Smart Images
Figure CN118427442B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of intelligent recommendation, and particularly relates to a recommendation method fusing cross network and feature importance selection. BACKGROUND
[0002] A recommendation algorithm is a core algorithm of a recommendation system, traditional recommendation algorithms include content-based recommendation, collaborative filtering-based recommendation and hybrid recommendation, which do not consider the attributes of users and items, only analyze a similarity matrix of users and items, so that the recommendation result is single. At present, a recommendation algorithm based on deep learning has reached or even surpassed a traditional recommendation algorithm, but the feature interaction mining capability of most recommendation models is limited, so an effective recommendation model is needed to improve the interaction capability of features.
[0003] Feature crossing is the most important step in a recommendation model, some existing models cannot effectively automatically learn limited-order feature interaction, and some models treat all single features and feature combinations equally, thereby affecting the final recommendation effect SUMMARY
[0004] The application aims to solve the problems in the prior art, and provides a recommendation method fusing cross network and feature importance selection, which performs feature weighting operation before feature interaction through an effective SENet network, further captures feature interaction information by using a deep neural network and a cross network in parallel, and designs a recommendation method fusing cross network and feature importance selection. The method realizes more accurate recommendation effect under the premise of the same data set, and has wide application prospect in practice.
[0005] To achieve the above object, the technical scheme of the application is as follows: a recommendation method fusing cross network and feature importance selection, comprising:
[0006] S1, pre-processing a data set, and constructing an initial vector for each user and item;
[0007] S2, performing weighting processing on the initial vector by using a feature selection network (Squeeze-and-Excitation Network, SENet) SENet network, and obtaining a corresponding updated partial vector 1 through a bilinear feature interaction layer;
[0008] S3, obtaining a corresponding updated partial vector 2 by passing the initial vector through the bilinear feature interaction layer;
[0009] S4, inputting the partial vector 1 and the partial vector 2 into a parallel cross network and a deep neural network for secondary processing, and mining a more rich feature combination vector;
[0010] S5. After secondary processing, a bridging module is used to enhance the information between explicit and implicit features to obtain the corresponding concatenated vector.
[0011] S6. Linearly integrate the obtained concatenated vector and the vector after secondary processing, and obtain the final prediction probability through the activation function to generate the recommendation result.
[0012] In one embodiment of the present invention, step S1 is specifically implemented as follows:
[0013] The preprocessing of the publicly available movie rating dataset involves encoding the categorical features and normalizing the numerical features, constructing an initial vector for each user and item: E = [e1,…,e f This includes user vectors and project vectors, e i Let f represent the i-th user vector or the project vector, and let f represent the number of vectors in the initial vector.
[0014] In one embodiment of the present invention, the encoding method is Label Encoder.
[0015] In one embodiment of the present invention, the specific process of obtaining the corresponding updated partial vector 1 includes: compressing each low-dimensional dense vector after embedding based on the idea of global average pooling, and calculating the mean to obtain a global feature vector; obtaining the importance weight of each feature based on the global feature vector; obtaining the reweighted output vector of each feature based on the importance weight; and obtaining the updated partial vector 1 based on the bilinear feature interaction layer.
[0016] In one embodiment of the present invention, step S2 is specifically implemented as follows:
[0017] The SENet network first compresses each low-dimensional dense vector after embedding through a squeezing phase and calculates its mean to represent relevant feature information. The formula is obtained, where (t) represents the sample index, i.e., different samples in a batch of data or multiple instances, and k represents the vector dimension, transforming the original E = [e1,…,e f Transformed into Z = [z1, ..., z] f Then, through the excitation phase, that is, using Z calculated in the squeeze phase, the weights corresponding to each feature are learned. The learning method uses two fully connected layers to perform feature interaction, with the formula A = F. ex (Z)=σ2(W2σ1(W1Z)), where r is the reduction ratio, i.e. compression ratio, which needs to be set according to actual data set, and sigma 1 and sigma 2 are nonlinear activation functions, to obtain A = [a1, …, a f ]; finally, through a re-calibration (Re-weight) stage, the weight size obtained in the Excitation is multiplied by the corresponding feature after embedding processing, to complete the weighting operation of feature importance, and the formula is V = F ReWeight (A, E) = [a1·e1, …, a f ·e f ] = [v1, …, v f ]; the weighted feature vector V is input into a bilinear feature interaction layer, the bilinear feature interaction layer combines inner product and Hadamard product and introduces an additional parameter matrix to learn feature cross, and the formula is p ij = v i ·W⊙v j , wherein W is a parameter matrix.
[0018] In an embodiment of the present application, the step S3 is specifically implemented as follows: the initial vector in the step S1 is input into a bilinear feature interaction layer, the bilinear feature interaction layer combines inner product and Hadamard product and introduces an additional parameter matrix to learn feature cross, and the formula is p ij = v i ·W⊙v j , wherein W is a parameter matrix.
[0019] In an embodiment of the present application, the W has three calculation modes, which are a field-all type, a field-each type and a field-interaction type.
[0020] In an embodiment of the present application, the secondary processing includes: inputting the spliced partial vector 1 and partial vector 2 into a deep neural network to learn the interaction of high-order features through multiple hidden layers, to obtain a learned high-order feature representation vector; and inputting the spliced partial vector 1 and partial vector 2 into a cross network to cross-process the feature vectors by using a low-rank technology, to obtain a learned cross feature representation vector.
[0021] In an embodiment of the present application, the step S4 is specifically implemented as follows:
[0022] The vectors obtained in the step S2 and step S3 are concatenated to obtain a vector C = [c1, …, c f(f-1) ], f(f-1) represents the number of vectors after splicing, and C is input into a parallel deep neural network and a cross network integrated with a low-rank technology to obtain h L and x L; wherein,
[0023] The deep neural network is used to capture nonlinear high-order features, and is a fully connected multilayer feedforward neural network, and the forward propagation process is expressed by a formula h l+1 = f(W l h l +b l ), wherein l represents the lth layer of the hidden layer, and a total of L layers, h l represents the output of the lth layer, h l+1 represents the output of the l+1th layer, W l represents the weight of the lth layer, b l represents the bias term of the lth layer, and f represents an activation function; the cross network formula integrated with the low-rank technology is wherein x0 represents an initial input, l represents the lth layer of the cross layer, and a total of L layers, x l represents the output of the lth layer in the cross layer, x l+1 represents the output of the l+1th layer in the cross layer, U l and V l are low-rank matrices obtained through dynamic learning, and b l is a bias term.
[0024] In an embodiment of the present application, the process of strengthening the information between the explicit and implicit features by the bridging module comprises: obtaining the corresponding splicing vector by the bridging module through the vectors output by the deep neural network and the cross network.
[0025] Compared with the prior art, the present application has the following beneficial effects:
[0026] (1) The present application can more accurately capture feature interaction information by fusing the cross network and the feature importance selection layer, and improves the performance and generalization ability of the model.
[0027] (2) The present application reduces the training cost of the model while ensuring the performance by introducing the low-rank technology in the newly combined cross network, and improves the practicability and scalability of the model.
[0028] (3) The present application can realize more reliable recommendation by using the recommendation model based on deep learning, and meets the personalized needs of users. BRIEF DESCRIPTION OF DRAWINGS
[0029] Figure 1 is a recommendation method flowchart in an embodiment of the present application;
[0030] Figure 2 is a network architecture diagram of the recommendation method of fusing the cross network and the feature importance selection in an embodiment of the present application. DETAILED DESCRIPTION
[0031] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.
[0032] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0033] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0034] Example
[0035] like Figure 1 , 2 As shown, this embodiment provides a recommendation method that integrates cross-network and feature importance selection, which is carried out according to the following steps:
[0036] Step 1: Data preprocessing, constructing an initial vector for each user and project: E = [e1, ..., e f This includes user vectors and item vectors. This embodiment uses a publicly available movie rating dataset for analysis, which contains a large number of movie reviews from movie-watching platforms, including movie information, user information, rating time, etc. The categorical features in the publicly available movie rating dataset are encoded. In this embodiment, the encoding method is Label Encoder. Categorical features are encoded and then input into the embedding layer, while numerical features are normalized and then directly input into the model. Implementers can choose other encoding tools according to their actual needs; this embodiment does not impose any restrictions on this.
[0037] Step 2: Input the initial vector from Step 1 into the SENet network to obtain the weighted feature vector V = [v1, ..., v1, ..., v2, ..., v3, ..., v4, ..., v5, ..., v6, ..., v7, ..., v8, ..., v9, ..., v1 ... f The vector is input into the bilinear feature interaction layer to obtain the vector. Specifically:
[0038] Network architectures that integrate cross-network and feature importance selection recommendation methods, such as Figure 2 As shown, the SENet network first uses a Squeeze stage to compress each low-dimensional dense vector after embedding and calculates its mean to represent the relevant feature information. The formula is obtained, where k represents the vector dimension, and the original E = [e1,…,e f Transformed into Z = [z1, ..., z] f]; then through the Excitation stage, that is, learning the weight corresponding to each feature by using Z calculated in the Squeeze stage, and the learning method is mainly through two fully connected layers to interact features, and the formula is A = F ex (Z) = σ2(W2σ1(W1Z)), where σ1 and σ2 are nonlinear activation functions, and A = [a1,..., a f ] is obtained; finally, through the Re-weight stage, the weight size obtained in the Excitation stage is multiplied by the corresponding Embedding feature to complete the weighting operation of the importance of the feature, and the formula V = F ReWeight (A, E) = [a1·e1,..., a f ·e f ] = [v1,..., v f ] is obtained. The vector is input into a bilinear feature interaction layer, which combines inner product and Hadamard product and introduces an additional parameter matrix to learn feature cross, aiming to mine richer interaction information. The formula is p ij = v i ·W⊙v j , where the parameter matrix W has three calculation methods, Field-All Type, Field-Each Type, and Field-Interaction Type, and the optimal method can be selected according to the specific data set, which is not limited in the embodiment.
[0039] Step 3: input the vector obtained in step 1 into the bilinear feature interaction layer to obtain the vector Specifically,
[0040] As described above, the formula q ij = v i ·W⊙v j , similarly, the parameter matrix W has three calculation methods, and the optimal method can be selected according to the specific data set, which is not limited in the embodiment.
[0041] Step 4: concatenate the vectors obtained in steps 2 and 3 to obtain the vector C = [c1,..., c f(f-1) ], and send C to the parallel deep neural network and the cross network integrated with low-rank technology to obtain h L and x L , specifically,
[0042] The deep neural network is used to capture nonlinear high-order features, which is a fully connected multilayer feedforward neural network, and its forward propagation process can be represented by the calculation formula h l+1 = f(W l h l +b l), where l represents the l-th hidden layer, and there are a total of L layers, h l h represents the output of the l-th layer. l+1 W represents the output of the (l+1)th layer. l b represents the weight of the l-th layer. l Let f represent the bias term of the l-th layer and f represent the activation function; the formula for the cross-network incorporating low-rank techniques is: Where x0 represents the initial input, l represents the l-th crossover layer, and there are a total of L layers. l x represents the output of the l-th layer in the cross layer. l+1 U represents the output of the (l+1)th layer in the cross layer. l and V l The low-rank matrix b is obtained through dynamic learning. l It is the bias term. The two parallel networks respectively obtain the output vector h. L and x L .
[0043] Step 5: Take the h obtained in step 4 L and x L Output vector f through the bridge module L Specifically:
[0044] The bridge module obtains the output vector f through a concatenation + fully connected layer (FC) computation method. L Using a linear layer with the ReLU activation function for output is a data merging operation. By connecting data along specific dimensions, the new data structure incorporates information from the original data, providing richer information to subsequent layers. The formula is f. L =RELU(w T [x L ,h L ]+b), where w T This is the weight matrix of the fully connected layer, where b is the bias term, [x l ,h l [This is the concatenated output vector.]
[0045] Step 6: Linearly integrate the vectors obtained in Steps 4 and 5 and apply an activation function to obtain the final predicted probability, specifically:
[0046] The output layers of each part are linearly integrated, and the final prediction result is output using the Sigmoid function. The formula is as follows: Where w T This is the weight matrix of the fully connected layer, b is the bias term, and x is the weight matrix of the fully connected layer. L It is the output of the cross-network, hL is an output of the deep neural network, f L is an output of the bridging module.
[0047] The above description is merely that of the preferred specific embodiments of the application, and is not intended to limit the scope of the application. Any change and modification that can be easily conceived by those skilled in the art within the technical scope of the application should be encompassed within the scope of the application. Therefore, the scope of the application should be defined by the scope of the claims.
Claims
1. A recommendation method that fuses a cross-network and feature importance selection, characterized by, The method comprises the following steps: S1, preprocessing the data set to construct an initial vector for each user and item; S2, weighting the initial vector using a feature selection network SENet and passing it through a bilinear feature interaction layer to obtain a corresponding updated partial vector 1; S3, passing the initial vector through a bilinear feature interaction layer to obtain a corresponding updated partial vector 2; S4, inputting the partial vector 1 and the partial vector 2 into a parallel cross-network and a deep neural network for secondary processing to mine more rich feature combination vectors; S5, after the secondary processing, using a bridging module to strengthen the information between explicit and implicit features to obtain a corresponding spliced vector; S6, linearly integrating the obtained spliced vector and the secondary processed vector through an activation function to obtain the final prediction probability and generate a recommendation result. The specific process of obtaining the corresponding updated partial vector 1 comprises: compressing and averaging each low-dimensional dense vector after embedding processing based on the idea of global average pooling to obtain a global feature vector; obtaining the importance weight of each feature based on the global feature vector; obtaining the output vector of each feature after reweighting based on the importance weight; and obtaining the updated partial vector 1 based on the bilinear feature interaction layer. The step S2 is specifically implemented as follows: SENet first compresses each low-dimensional dense vector after embedding processing and takes the average to represent the relevant feature information, which is obtained by where e i represents the i-th vector in the initial vector, (t) represents the sample index, i.e., different samples in a batch of data or multiple instances, and k represents the vector dimension. The initial vector E = [e1, …, e f ] is converted to Z = [z1, …, z f ]. Then, the excitation stage is performed, i.e., the Z obtained by the compression stage is used to learn the weight corresponding to each feature. The learning method is to perform feature interaction through two fully connected layers, and the formula is A = F ex (Z) = σ2(W2σ1(W1Z)), where r is the compression ratio, σ1 and σ2 are nonlinear activation functions, and A = [a1, …, a f ] is obtained. Finally, the recalibration stage is performed, i.e., the weight size obtained in the excitation stage is multiplied by the corresponding feature after embedding processing to complete the weighting operation of feature importance, and the formula is V = F ReWeight (A, E) = [a1·e1, …, a f ·e f ] = [v1, …, v f ]. The weighted feature vector V is input into the bilinear feature interaction layer, which combines inner product and Hadamard product and introduces an additional parameter matrix to learn feature cross, and the formula is p ij = v i ·W⊙v j , where W is the parameter matrix.
2. The method of claim 1, wherein the method further comprises: The step S1 is specifically implemented as follows: Preprocess the movie rating public dataset, that is, encode the category type features in the movie rating public dataset, normalize the numerical type features in the movie rating public dataset, and construct an initial vector for each user and item: E=[e1,…,e f ], wherein the initial vector includes a user vector and an item vector, e i represents the i-th user vector or an item vector, and f represents the number of vectors in the initial vector.
3. The method of claim 2, wherein the method further comprises: The encoding mode is Label Encoder.
4. The method of claim 2, wherein the method further comprises: The step S3 is implemented as follows: input the initial vector in step S1 into a bilinear feature interaction layer, combine inner product and Hadamard product in the bilinear feature interaction layer, and introduce an additional parameter matrix to learn feature cross, and the formula is p ij = v i · W ⊙ v j where W is a parameter matrix.
5. The method of claim 1 or 4, wherein the method further comprises: The W has three calculation modes, which are full field type, single field type and field interaction type.
6. The method of claim 1, wherein the method further comprises: The secondary processing comprises: inputting the spliced partial vector 1 and the partial vector 2 into a deep neural network to learn the interaction of high-order features through multiple hidden layers to obtain a learned high-order feature representation vector; and inputting the spliced partial vector 1 and the partial vector 2 into a cross-network to cross-process the feature vectors using low-rank technology to obtain a learned cross-feature representation vector.
7. The method of claim 1, wherein the method further comprises: The step S4 is specifically implemented as follows: The vectors obtained in step S2 and step S3 are concatenated to obtain a vector C = [c1, …, c f(f-1) ], where f(f-1) represents the number of concatenated vectors, and C is sent to a parallel deep neural network and a cross network with low rank technology to obtain h L and x L ; wherein, Deep neural networks are used to capture nonlinear high-order features. They are fully connected, multi-layer feedforward neural networks, and their forward propagation process is expressed by the formula h. l+1 =f(W l h l +b l ), where l represents the l-th hidden layer, and there are a total of L layers, h l h represents the output of the l-th layer. l+1 W represents the output of the (l+1)th layer. l b represents the weight of the l-th layer. l Let f represent the bias term of the l-th layer and f represent the activation function; the formula for the cross-network incorporating low-rank techniques is: Where x0 represents the initial input, l represents the l-th crossover layer, and there are a total of L layers. l x represents the output of the l-th layer in the cross layer. l+1 U represents the output of the (l+1)th layer in the cross layer. l and V l The low-rank matrix b is obtained through dynamic learning. l It is a bias term.
8. The method of claim 1, wherein the method further comprises: The process of using a bridging module to strengthen the information between explicit and implicit features comprises: obtaining a corresponding spliced vector by passing the vectors output by the deep neural network and the cross-network through the bridging module.