User interest estimation method based on multi-structure feature intersection
By adopting a multi-structure feature cross-over method in the user interest estimate method, combining FM, DNN, DCN and SENET networks, the problem that traditional models cannot capture nonlinear feature relationships is solved, and more efficient user interest estimates and precise marketing are achieved .
Patent Information
- Application Number
- CN202411440573.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-11
AI Technical Summary
Traditional linear models cannot effectively capture the nonlinear relationship between user interest characteristics, and are difficult to meet the needs of higher-order feature combinations, and cannot personalize the contribution of different feature combinations to the final goal.
Using a user interest prediction method based on multi-structure feature intersection, multi-structure cross-learning and dynamic weight learning are realized by converting high-dimensional sparse features into dense vector expressions, combining FM, DNN, DCN and SENET networks. .
It improves the expression and generalization ability of the model, can more accurately explore deep feature relationships, improves the accuracy and self-optimization ability of user interest estimates, and supports more accurate marketing and operation strategies.
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer network technology, and specifically relates to a user interest estimation method based on multi-structure feature intersection. Background Art
[0002] Nowadays, the bonus period of mobile Internet is basically over. It is particularly important to have a more accurate understanding of users and to provide more focused services to users. By exploring user interests, we can provide decision support and optimize operation strategies for business departments. By analyzing user behavior data, consumption habits, etc., we define the user interest prediction problem as predicting the probability that users can achieve the expected conversion if they are exposed to materials of interest again in the future.
[0003] User interests are hidden in short-term and long-term historical behaviors. Long-term behavioral characteristics describe users' relatively stable habits, while short-term characteristics reflect users' recent demand preferences. How to mine intrinsic associations based on the above features plays a vital role. In the user interest mining scenario, the interaction and combination relationship between features have a great impact on the prediction results. However, traditional linear models cannot capture the nonlinear relationship between these features. The expressive power of the model can be improved through feature crossover. DeepFM realizes the second-order crossover of features and gets rid of the reliance on artificial expert strategies, reducing feature engineering. However, it is difficult to meet the requirements of higher-order feature combinations, and the contribution of different feature combinations to the final goal cannot be personalized, so improvements are needed. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention provides a method for estimating user interests based on the intersection of multiple structural features, which has the advantage of realizing user interest estimation.
[0005] To achieve the above object, the present invention provides the following technical solution: a method for estimating user interest based on multi-structure feature intersection, the specific steps are as follows:
[0006] Step 1: Sample construction
[0007] The quality of training samples often determines the upper limit of a model's generalization ability. To ensure the completeness and reliability of the data, we extract the behavior logs of seasonally active users on the main site and construct a data set through sampling.
[0008] Step 2: Feature Engineering
[0009] Divide common features into fine-grained categories and specific scenarios to explore data distribution, missing values, outliers, etc. For features with missing values, decide to use deletion, filling, interpolation, etc. based on the model effect. Considering how to reflect user interests and preferences, the features are processed as follows:
[0010] User history of interaction with different interest materials, such as the user's browsing frequency, search frequency, and clue behavior in the content related to a certain interest material;
[0011] The decay of the user's historical behavior. For example, the decaying linear weighting of the user's current behaviors can express the user's life cycle.
[0012] User behavior sequence, for example, the user's historical click sequence on business-related materials;
[0013] Step 3: Model structure
[0014] The multi-structure feature crossover algorithm model consists of four parts. In the model input part, high-dimensional sparse features are converted into dense vector expressions to achieve dimensionality reduction, which is used to improve the expressiveness and generalization ability of the model. FM is used for factor decomposition, so that the algorithm can automatically learn the second-order cross information between features; combined with DNN, the underlying parameters are shared, and deep feature representations are learned; the DCN network is introduced to achieve high-order cross-learning of displayed features. Since the time complexity and space complexity of DCN are linearly related to the input dimension, efficient training can be achieved; the above substructures have the same distribution of feature weights, which will cause strongly correlated features and secondary features to have the same weights, affecting the final effect. Therefore, SENET is used in the last substructure to dynamically learn the weights between features, highlighting the importance of important features in the output results. The multi-structure feature crossover algorithm adopts a parallel structure for the four parts, and finally sums the outputs of each substructure, and predicts the results through the Sigma id activation function;
[0015] Step 4: Model training
[0016] There are many important hyperparameters that need to be adjusted in the model training process, such as the selection of the number of fully connected layers and the number of nodes, the adjustment of the learning rate, etc. Regularization and Dropout can be used to better train the model and ensure good generalization ability. The model is trained offline based on the interactive behavior characteristics of users' historical different behavior cycles and interested materials, and user interests are estimated based on the predicted probability value.
[0017] Preferably, the behavior log of the user at the main site described in step one specifically includes high-frequency interaction behaviors and behaviors that do not meet business needs between the user and the materials of interest in the historical period, wherein the data on high-frequency interaction behaviors between the user and the materials of interest in the historical period are used as positive samples, and those that do not meet business needs are used as negative samples.
[0018] Preferably, the common features described in step 2 specifically include duration and clicks.
[0019] Preferably, how to respond to the user's interest preferences in step 2 needs to be specifically considered at two levels. The first level is based on the interaction between the user's history and the interest materials; the second level is to consider the user's current behavior.
[0020] Preferably, the DCN network described in step 3 can make up for the deficiency that FM only realizes second-order crossover.
[0021] Preferably, the shared underlying parameters described in step three reduce the number of parameters of the model and reduce the complexity of the model.
[0022] Preferably, the FM described in step three enables the second-order cross information between features to be extracted without requiring much manual feature engineering intervention.
[0023] Preferably, the regularization described in step 4 includes L2 regularization and L1 regularization, which is a technique for reducing the complexity of the model.
[0024] Preferably, the Dropout described in step 4 refers to temporarily discarding neural network units from the network with a certain probability during the training process of the deep learning network.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] The present invention adopts a user interest estimation method based on the intersection of multiple structural features. The whole method is logically intuitive, easy to implement, and can mine deep feature relationships. The coverage of user preferences is higher after combination. The offline posteriori of the model ensures the accuracy and has a certain self-optimization ability. It can better promote the formulation of recommendation, search, push and other strategies within the support platform, circle user groups, and carry out precision marketing and refined operations. DETAILED DESCRIPTION
[0027] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.
[0028] The present invention provides a technical solution: a method for estimating user interests based on multi-structure feature intersection, the specific steps are as follows:
[0029] Step 1: Sample construction
[0030] The quality of training samples often determines the upper limit of a model's generalization ability. To ensure the completeness and reliability of the data, we extract the behavior logs of seasonally active users on the main site and construct a data set through sampling.
[0031] Step 2: Feature Engineering
[0032] Divide common features into fine-grained categories and specific scenarios to explore data distribution, missing values, outliers, etc. For features with missing values, decide to use deletion, filling, interpolation, etc. based on the model effect. Considering how to reflect user interests and preferences, the features are processed as follows:
[0033] User history of interaction with different interest materials, such as the user's browsing frequency, search frequency, and clue behavior in the content related to a certain interest material;
[0034] The decay of the user's historical behavior. For example, the decaying linear weighting of the user's current behaviors can express the user's life cycle.
[0035] User behavior sequence, for example, the user's historical click sequence on business-related materials;
[0036] Step 3: Model structure
[0037] The multi-structure feature crossover algorithm model consists of four parts. In the model input part, high-dimensional sparse features are converted into dense vector expressions to achieve dimensionality reduction, which is used to improve the expressiveness and generalization ability of the model. FM is used for factor decomposition, so that the algorithm can automatically learn the second-order cross information between features; combined with DNN, the underlying parameters are shared, and deep feature representations are learned; the DCN network is introduced to achieve high-order cross-learning of displayed features. Since the time complexity and space complexity of DCN are linearly related to the input dimension, efficient training can be achieved; the above substructures have the same distribution of feature weights, which will cause strongly correlated features and secondary features to have the same weights, affecting the final effect. Therefore, SENET is used in the last substructure to dynamically learn the weights between features, highlighting the importance of important features in the output results. The multi-structure feature crossover algorithm adopts a parallel structure for the four parts, and finally sums the outputs of each substructure, and predicts the results through the Sigma id activation function;
[0038] Step 4: Model training
[0039] There are many important hyperparameters that need to be adjusted in the model training process, such as the selection of the number of fully connected layers and the number of nodes, the adjustment of the learning rate, etc. Regularization and Dropout can be used to better train the model and ensure good generalization ability. The model is trained offline based on the interactive behavior characteristics of users' historical different behavior cycles and interested materials, and user interests are estimated based on the predicted probability value.
[0040] The multi-structure feature intersection model of the present invention is composed of four substructures, all of which share feature inputs. Substructure one adopts the FM model and utilizes second-order feature intersection to extract low-order features; substructure two adopts a fully connected network to realize feature extraction of a multi-layer network; substructure three adopts a DCN network structure to realize multi-order feature intersection; substructure four adopts a SENET network structure to dynamically learn feature weights for input features, thereby enabling mining and estimation of user interests based on the model.
[0041] Among them, the user's behavior log on the main site in step one specifically includes high-frequency interaction behaviors between users and materials of interest in historical time and behaviors that do not meet business needs. Among them, the data on high-frequency interaction behaviors between users and materials of interest in historical time are used as positive samples, and those that do not meet business needs are used as negative samples.
[0042] The division of positive and negative samples makes the adoption of data samples clearer and more accurate, and the construction of the data set more complete and intuitive.
[0043] The common features in step 2 specifically include duration and clicks.
[0044] The fine-grained division of common features is conducive to better feature representation.
[0045] Among them, how to reflect the user's interest preferences in step 2 requires specific consideration of two levels. The first level is based on the interaction between the user's history and interest materials; the second level is to consider the user's current behavior.
[0046] The first level can reflect what materials of interest users are willing to pay for, and the second level can reflect whether users' interests will migrate and change in the future, that is, to what extent users' interest estimates rely on historical interaction behaviors.
[0047] Among them, the DCN network in step three can make up for the deficiency that FM only realizes second-order crossover.
[0048] The introduction of the DCN network improves the cross-learning of low-order and high-order features, thereby improving the completeness and applicability of feature learning.
[0049] Among them, the shared underlying parameters in step three reduce the number of model parameters and reduce the complexity of the model.
[0050] The reduction in the number of model parameters and the reduction in model complexity are conducive to optimizing the model, thereby increasing the speed of model construction.
[0051] Among them, the FM in step three makes it possible to extract the second-order cross-information between features without requiring too much manual feature engineering intervention.
[0052] The use of FM makes the model structure highly scalable when processing large-scale data and high-dimensional sparse features, reduces manual intervention, and improves the efficiency of obtaining second-order cross information.
[0053] Among them, the regularization in step 4 includes L2 regularization and L1 regularization, which is a technology based on reducing the complexity of the model.
[0054] The essence of regularization is to control the number of features learned by the model to minimize it, thereby preventing the introduction of sampling errors in the training set during the training process.
[0055] Among them, Dropout in step 4 means that during the training process of the deep learning network, for the neural network units, they are temporarily discarded from the network according to a certain probability.
[0056] For stochastic gradient descent, due to random dropout, each mini-batch is training a different network.
[0057] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0058] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A user interest estimation method based on multi-structure feature intersection, characterized in that: The specific steps are as follows: Step 1: Sample construction The quality of training samples often determines the upper limit of a model's generalization ability. To ensure the completeness and reliability of the data, we extract the behavior logs of seasonally active users on the main site and construct a data set through sampling. Step 2: Feature Engineering Divide common features into fine-grained categories and specific scenarios to explore data distribution, missing values, outliers, etc. For features with missing values, decide to use deletion, filling, interpolation, etc. based on the model effect. Considering how to reflect user interests and preferences, the features are processed as follows: User history of interaction with different interest materials, such as the user's browsing frequency, search frequency, and clue behavior in the content related to a certain interest material; The decay of the user's historical behavior. For example, the decaying linear weighting of the user's current behaviors can express the user's life cycle. User behavior sequence, for example, the user's historical click sequence on business-related materials; Step 3: Model structure The multi-structure feature crossover algorithm model consists of four parts. In the model input part, high-dimensional sparse features are converted into dense vector expressions to achieve dimensionality reduction, which is used to improve the expressiveness and generalization ability of the model. FM is used for factor decomposition, so that the algorithm can automatically learn the second-order cross information between features; combined with DNN, the underlying parameters are shared, and deep feature representations are learned; the DCN network is introduced to achieve high-order cross-learning of displayed features. Since the time complexity and space complexity of DCN are linearly related to the input dimension, efficient training can be achieved; the above substructures have the same distribution of feature weights, which will cause strongly correlated features and secondary features to have the same weights, affecting the final effect. Therefore, SENET is used in the last substructure to dynamically learn the weights between features, highlighting the importance of important features in the output results. The multi-structure feature crossover algorithm adopts a parallel structure for the four parts, and finally sums the outputs of each substructure, and predicts the results through the Sigmoid activation function; Step 4: Model training There are many important hyperparameters that need to be adjusted in the model training process, such as the selection of the number of fully connected layers and the number of nodes, the adjustment of the learning rate, etc. Regularization and Dropout can be used to better train the model and ensure good generalization ability. The model is trained offline based on the interactive behavior characteristics of users' historical different behavior cycles and interested materials, and user interests are estimated based on the predicted probability value.
2. The method for estimating user interests based on multi-structural feature intersection according to claim 1, characterized in that: The user's behavior log on the main site described in step one specifically includes high-frequency interaction behaviors between the user and the materials of interest in the historical period and behaviors that do not meet business needs, among which the data on high-frequency interaction behaviors between the user and the materials of interest in the historical period are used as positive samples, and those that do not meet business needs are used as negative samples.
3. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: The common features described in step 2 specifically include duration and clicks.
4. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: How to respond to user interest preferences as described in step 2 requires specific consideration of two levels. The first level is based on the interaction between user history and interest materials; the second level is to consider the user's current behavior.
5. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: The DCN network described in step 3 can make up for the deficiency that FM only realizes second-order crossover.
6. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: The shared underlying parameters described in step 3 reduce the number of model parameters and the complexity of the model.
7. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: The FM described in step 3 enables the second-order cross-information between features to be extracted without much manual feature engineering intervention.
8. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: The regularization described in step 4 includes L2 regularization and L1 regularization, which is a technique for reducing the complexity of the model.
9. The method for estimating user interests based on multi-structure feature intersection according to claim 1, characterized in that: The Dropout described in step 4 refers to temporarily discarding neural network units from the network with a certain probability during the training process of the deep learning network.