A recommendation method based on multi-view feature selection

Abstract

The present invention discloses a recommendation method based on multi-view feature selection. In view of the differences in feature selection for given feature fields in different data in the recommendation system, and the problem of bias towards main features in the process of adaptively selecting features for each data instance, the present invention proposes multi-view feature selection to effectively select rich feature information for each instance. Specifically, the present invention adopts a multi-view network composed of multiple sub-networks, so that each sub-network learns data features of different feature patterns, reduces the deviation from the dominant pattern, and promotes the balance of the feature selection process. In addition, an effective importance scoring modeling strategy is adopted to perform strategy modeling independently for each field, so that there is no dependency between features, and ultimately effectively improves the efficiency of feature selection in recommendation.

Description

Technical Field

[0001] The present invention relates to the field of big data recommendation based on multi-view feature selection, and solves the problem of recommendation model prediction by adaptively selecting features for each data instance through adaptive feature selection. Background Art

[0002] In real-world web platforms, recommender systems (RS) encounter a large number of features collected from users (e.g., age, gender), items (e.g., brand, price), and their interactions (e.g., dwell time, location), and the effective utilization of these features plays a crucial role in the quality of recommendations. To capture the complex relationships between features, RS models adopt sophisticated architectures with powerful capabilities. However, it has been noted that some of these features lack relevance or exhibit redundancy in the context of user-item interactions, so blindly inputting all features into the model often leads to poor accuracy and slow model optimization. To address this issue, feature selection, a technique for selecting the most informative subset of features, has recently received increasing attention.

[0003] Over the past few decades, a variety of feature selection methods have been studied, ranging from manual and statistical approaches to traditional machine learning methods. However, their effectiveness is often limited because feature selection is performed independently of the subsequent RS model, ignoring the model's predictive behavior. Recent methods have adopted AutoML methods to automatically identify and select the most predictive features during the model optimization process and have shown effectiveness for RS models.

[0004] A new strategy was recently introduced to adaptively select features appropriate for each user-item interaction pair, taking into account that the importance of a given feature field can vary significantly across pairs. To handle this dynamic nature of each pair, AdaFS employs a controller network to compute the importance of each feature field for each pair. It then generates a weighted feature embedding by multiplying the importances. Through adaptive selection, AdaFS significantly improves performance over previous methods that aim to select a globally fixed subset of feature fields.

[0005] Nevertheless, AdaFS still has much room for improvement. First, it adopts a single controller network to select features for all user-item pairs. This makes it easy for the selection process to be biased towards a small number of large groups of pairs with frequently occurring features, resulting in limited improvement on data with relatively low-frequency minor features. The effectiveness of adaptive selection can be further improved by explicitly preventing the bias problem. Second, AdaFS applies a reweighting step to ensure that the sum of the calculated importances remains constant before being reflected in the subsequent RS model. However, this reweighting causes the overall scale of the importance scores to fluctuate depending on the number of selected fields, e.g., for k selected fields, the average importance ratio is proportional to 1 / k, which creates unnecessary dependencies between features and leads to poor performance. Summary of the Invention

[0006] To address the limitations and challenges of existing solutions, this paper proposes Multi-view Feature Selection (MvFS), which can effectively select informative features for each data instance. First, multiple sub-networks are adopted within the controller. Each sub-network in MvFS is designed to specialize in processing data with different feature patterns, preventing it from being biased towards a few dominant patterns and promoting a more balanced feature selection process. In addition, MvFS adopts a new importance score modeling strategy that is applied to each field independently without the need for a reweighting step. Extensive experiments on real datasets are conducted to verify the superiority of MvFS, and detailed analysis is provided to show the effectiveness of each proposed component.

[0007] A recommendation method based on multi-view feature selection of the present invention comprises the following steps:

[0008] S1: The construction process of the recommendation model with feature selection. First, the input construction is used to adaptively select the information-rich gender, price, and age feature fields for each user-item pair. Second, the recommendation model with adaptive feature selection is designed.

[0009] S2: Multi-view feature selection process, a controller is proposed that can select rich information features while avoiding bias towards a few main features. The controller consists of two components: (1) a multi-view network that calculates feature importance by merging various views of multiple sub-networks, (2) importance score modeling that determines the final importance score of feature selection; (3) a recommendation model with adaptive feature selection;

[0010] S3: Optimize the modeled network structure. First, train the controller and RS model to predict the interaction between user-item pairs. th The weighted feature vector of the instance is denoted as H m and denote the corresponding ground truth label as y m, the loss function is defined as follows:

[0011]

[0012] where θ RS represents the parameters of the RS model, including the embedding component and subsequent layers, θ C Represents the parameters of the controller.

[0013] Furthermore, the specific steps of S1 are:

[0014] S11. Input the N feature fields given in the original data and construct each data instance as X = [x1,…,x N ], where x n is a pair of n th The value of the feature field is encoded into a one-hot vector, let P n Indicates n th After projecting each feature field (i.e., embedding lookup), the data instance X is converted into a dense vector E as follows:

[0015] E=[e1,e2,…,e N ]

[0016] where e n =P n x n Represents the embedding of each field;

[0017] Furthermore, the specific steps of S2 are:

[0018] S21. First, the feature vector E is used as input through a multi-view network to calculate the importance of each feature field. However, if only a single network is used to calculate the feature importance, the controller will easily be biased towards several frequently appearing main features. To solve this problem, the present invention adopts the idea of ​​expert mixture and uses a divide-and-conquer strategy composed of multiple different sub-networks. Each sub-network learns to process a part of the input space. By focusing on multiple sub-networks with data with different feature patterns for modeling, the controller is prevented from biasing and a more balanced feature selection is achieved.

[0019] Specifically, the multi-view network consists of K sub-networks (SN1, SN2, ..., SN K ) is used to quantify the importance of feature fields, and a gating module g is used to regulate the influence of each sub-network on the feature pattern of each data. First, the SN K (E)∈R N Calculate the feature importance and generate an importance vector where each element represents the importance of each field. Then, use g to calculate the subnetwork r∈RK The impact is as follows:

[0020] r=σ(W g C+b g ),C=[SN1(E),…,SN K (E)]

[0021] Where σ is the sigmoid function, W g ∈R K×KN and b g ∈R K is the learnable weight matrix and bias vector of the gating module;

[0022] The final feature importance vector I∈R N is calculated by aggregating the outputs of the sub-networks based on the gating results as follows:

[0023]

[0024] where r k represents k in r th value;

[0025] S22. Importance score modeling: The calculated importance of each field is used to model the final importance score. A gradual transition from soft selection to hard selection is adopted during the training process. In the early stage, the recommendation model explores various feature combinations through soft selection. As the training progresses, the recommendation model gradually prioritizes information-rich features through hard selection while ignoring unimportant and redundant features. th The final importance score of the feature field is defined as follows:

[0026] s n =0.5*(1+tanh(τ·(I n -l)))

[0027] Where l is the chosen threshold, the unit step function and the tanh function are used to model the transition, and τ = max(5,1 + 0.001t) is set to control the smoothness of tanh, where t represents the training step. This choice gradually makes s n Become binary values, thus enabling a smooth transition to hard selection during training. While scoring modeling is applied independently to each field, this approach uses a reweighting step across fields, which creates unnecessary dependencies between selected features.

[0028] S23, design a recommendation model with adaptive feature selection. The steps are as follows: In order to selectively utilize the information features of each data, first obtain the importance score s of each feature field in S23 n, and then generate a weighted feature vector by calculating the importance score and multiplying it with the corresponding feature embedding E. The final weighted feature vector H is obtained as follows:

[0029] H=[s1e1,s2e2,…,s N e N ]

[0030] where s n s n is n calculated by the controller th The importance score of the field, when the design choice is a soft choice, s n is a real value. When the design choice is a hard choice, s n is a binary value. Finally, the weighted feature vector is input into the subsequent recommendation model:

[0031]

[0032] in is the model prediction, RS(·) is the recommendation model, which is regarded as a black box, and feature selection only affects the model input. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Flowchart of this method. DETAILED DESCRIPTION

[0034] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below with reference to examples and drawings.

[0035] As attached Figure 1 As shown in FIG, a recommendation method based on multi-view feature selection includes the following steps:

[0036] Step 1: The recommendation model construction process with feature selection first selects information-rich gender, price, and age feature fields for each user-item pair through input construction, and then designs a recommendation model with adaptive feature selection:

[0037] For the N feature fields given in the original data, we first instantiate each data as X = [x1,…,x N ], where x n is a pair of n th The value of the feature field is encoded into a one-hot vector, let P n Indicates n th After projecting each feature field (i.e., embedding lookup), the data instance X is converted into a dense vector E as follows:

[0038] E=[e1,e2,…,e N]

[0039] where e n =P n x n Represents the embedding of each field;

[0040] Step 2: Multi-view feature selection process. A controller is proposed that can select rich and informative features while avoiding bias towards a few main features. The controller consists of two components: (1) a multi-view network that calculates feature importance by merging various views of multiple sub-networks, (2) importance score modeling that determines the final importance score of feature selection; and (3) a recommendation model with adaptive feature selection.

[0041] First, the feature vector E is used as input through a multi-view network to calculate the importance of each feature field. However, if only a single network is used to calculate the feature importance, the controller will easily be biased towards several frequently appearing main features. To solve this problem, the present invention adopts the idea of ​​expert mixture and uses a divide-and-conquer strategy composed of multiple different sub-networks. Each sub-network learns to process a part of the input space. By focusing on multiple sub-networks with data with different feature patterns for modeling, the controller is prevented from biasing and a more balanced feature selection is achieved.

[0042] Specifically, the multi-view network consists of K sub-networks (SN1, SN2, ..., SN K ) is used to quantify the importance of feature fields, and a gating module g is used to regulate the influence of each sub-network on the feature pattern of each data. First, the SN K (E)∈R N Calculate the feature importance and generate an importance vector where each element represents the importance of each field. Then, use g to calculate the subnetwork r∈R K The impact is as follows:

[0043] r=σ(W g C+b g ),C=[SN1(E),…,SN K (E)]

[0044] Where σ is the sigmoid function, W g ∈R K×KN and b g ∈R K is the learnable weight matrix and bias vector of the gating module;

[0045] The final feature importance vector I∈R N is calculated by aggregating the outputs of the sub-networks based on the gating results as follows:

[0046]

[0047] where r k represents k in r th value;

[0048] Importance score modeling: The calculated importance of each field is used to model the final importance score. A gradual transition from soft selection to hard selection is adopted during the training process. In the early stage, the recommendation model explores various feature combinations through soft selection. As the training progresses, the recommendation model gradually prioritizes informative features through hard selection while ignoring unimportant and redundant features. th The final importance score of the feature field is defined as follows:

[0049] s n =0.5*(1+tanh(τ·(I n -l)))

[0050] Where l is the chosen threshold, the unit step function and the tanh function are used to model the transition, and τ = max(5,1 + 0.001t) is set to control the smoothness of tanh, where t represents the training step. This choice gradually makes s n Become binary values, thus enabling a smooth transition to hard selection during training. While scoring modeling is applied independently to each field, this approach uses a reweighting step across fields, which creates unnecessary dependencies between selected features.

[0051] The design of a recommendation model with adaptive feature selection is as follows: In order to selectively utilize the information features of each data, first obtain the importance score s of each feature field in S23 n , and then generate a weighted feature vector by calculating the importance score and multiplying it with the corresponding feature embedding E. The final weighted feature vector H is obtained as follows:

[0052] H=[s1e1,s2e2,…,s N e N ]

[0053] where s n s n is n calculated by the controller th The importance score of the field, when the design choice is a soft choice, s n is a real value. When the design choice is a hard choice, s n is a binary value. Finally, the weighted feature vector is input into the subsequent recommendation model:

[0054]

[0055] in is the model prediction, RS(·) is the recommendation model, which is regarded as a black box, and feature selection only affects the model input.

[0056] Step 3: Optimize the modeled network structure. First, train the controller and RS model to predict the interaction between user-item pairs. th The weighted feature vector of the instance is denoted as H m and denote the corresponding ground truth label as y m , the loss function is defined as follows:

[0057]

[0058] where θ RS represents the parameters of the RS model, including the embedding component and subsequent layers, θ C Represents the parameters of the controller.

Claims

1. A recommendation method based on multi-view feature selection, characterized by The following steps are involved: S1: The construction process of the recommendation model with feature selection. First, the input construction is used to adaptively select the information-rich gender, price, and age feature fields for each user-item pair. Second, the recommendation model with adaptive feature selection is designed. S2: Multi-view feature selection process, a controller is proposed that can select rich information features while avoiding bias towards a few main features. The components of the controller are as follows: (1) multi-view network, which calculates feature importance by merging various views of the sub-network, (2) importance score modeling, which determines the final importance score of feature selection; (3) design of recommendation model with adaptive feature selection; S3: Optimize the modeled network structure. First, train the controller and RS model to predict the interaction between user-item pairs. The weighted feature vector of an instance is expressed as and denote the corresponding ground truth labels as , the loss function is defined as follows: ； in represents the parameters of the RS model, including the embedding component and subsequent layers, represents the parameters of the controller, and RS represents the recommendation model.

2. The recommendation method based on multi-view feature selection according to claim 1, characterized in that In step S1: the recommendation model construction process with feature selection is; S11. Input the N feature fields given in the original data and construct each data instance as ,in It is a pair The value of the feature field is encoded as a one-hot vector, so express After projecting each feature field, the data instance X is converted into a dense vector E as follows: ； in Represents the embedding of each field.

3. The recommendation method based on multi-view feature selection according to claim 1, characterized in that The specific method of step S2 is: S21. First, the feature vector E is used as input through a multi-view network to calculate the importance of each feature field. The idea of ​​expert mixture is adopted, and a divide-and-conquer strategy consisting of multiple different sub-networks is used. Each sub-network learns to process a part of the input space. By focusing on multiple sub-networks with different feature patterns for modeling, the controller is prevented from bias and a more balanced feature selection is achieved. Specifically, the multi-view network consists of K sub-networks It is composed of a gating module g for quantifying the importance of feature fields and a gating module g for regulating the influence of each sub-network on the feature pattern of each data. Calculate the feature importance and generate an importance vector where each element represents the importance of each field. Then, use g to calculate the sub-network The impact is as follows: ； in is the sigmoid function, and is the learnable weight matrix and bias vector of the gating module; Final feature importance vector is calculated by aggregating the outputs of the sub-networks based on the gating results as follows: ； in Indicates the value; S22. Importance score modeling: The calculated importance of each field is used to model the final importance score. A gradual transition from soft selection to hard selection is adopted during the training process. In the early stages, the recommendation model explores various feature combinations through soft selection. As training progresses, it gradually prioritizes informative features through hard selection while ignoring unimportant and redundant features. The final importance score of the feature field is defined as follows: ； in is the chosen threshold, using the unit step function and the tanh function approximation to model the transition, setting To control the smoothness of tanh, where t represents the training step, this choice gradually makes Become binary values, thus enabling a smooth transition to hard selection during training, where scoring modeling is applied independently to each field, using a reweighting step across fields that creates unnecessary dependencies between selected features; S23, design a recommendation model with adaptive feature selection. The steps are as follows: In order to selectively utilize the information features of each data, first obtain the importance score of each feature field in S23 , and then generate a weighted feature vector by calculating the importance score and multiplying it with the corresponding feature embedding E. The final weighted feature vector H is obtained as follows: ； in The controller calculates Importance score of the field, when the design choice is a soft choice, is a real value, when the design choice is a hard choice, is a binary value. Finally, the weighted feature vector is input into the subsequent recommendation model: ； in is the model prediction, It is a recommendation model, which is treated as a black box and feature selection only affects the model input.

Images