Set-based top-down adaptive order feature crossing method
By employing a top-down adaptive order feature crossing method, which utilizes neural networks to generate feature subsets and perform pruning, the problems of difficult parameter adjustment and high computational cost in existing technologies are solved. This achieves efficient and accurate feature crossing, thereby improving the performance of recommendation systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2022-09-23
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies require manual parameter adjustment during feature crossing, are computationally expensive, and introduce a lot of noise, resulting in low recommendation accuracy.
A top-down adaptive order feature crossing method is adopted, which generates feature subsets through neural networks and uses soft threshold pruning to reduce noise and adaptively determine the order of feature crossing. The feature crossing is performed by combining additive functions and deep neural networks.
It improves the accuracy and speed of the recommendation system, reduces computational costs, and minimizes noise interference.
Smart Images
Figure CN115510320B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for top-down adaptive order feature cross-interaction based on sets. Background Technology
[0002] Predicting the probability of a user clicking on an item (CTR: Click-Through Rate Prediction) is a crucial problem in recommender systems. Typically, CTR prediction tasks incorporate four types of features: numerical features (e.g., user age, average article reading time); category features (user ID or item ID, usually represented using one-hot encoding); sequence features (the sequence of items viewed by the user over a past time period); and graph features (topological neighbor features between users and items). One of the keys to achieving better performance in CTR prediction is learning effective feature interactions, a problem that has garnered significant attention in recent years. One of the biggest challenges is the high computational cost of calculating high-order interactions between all original features, as the number of feature combinations grows exponentially with the number of features. In real-world applications, original features are often highly sparse, with millions of dimensions. For example, identifier features like user / item IDs are extremely sparse after being encoded into a single vector; the same is true for multi-file vectors built from upstream tasks (such as visual information). Computing high-order feature interactions on such sparse features with millions of dimensions is very time-consuming. At the same time, due to the large number of features, some noisy data will inevitably be introduced, resulting in unsatisfactory performance of the CTR model.
[0003] Existing methods for learning feature crosses can be broadly divided into two categories: the first category aims to achieve higher-order feature crosses, while the second category focuses on how to better measure the importance of feature crosses.
[0004] The classic feature interaction method is Factorization Machine (FM), which introduces second-order interactions by enumerating all second-order interactions and performing a dot product on each interaction pair. To capture higher-order information, Homogeneous Optimal Factorization Machine (HOFM) provides an efficient method for constructing higher-order feature interactions; however, HOFM has a significant drawback: when constructing K-order interactions, it requires computing all K-order feature pairs, which incurs high computational costs. To address this issue, DeepFM combines FM with deep neural networks. It uses the FM model to construct second-order interactions and the deep neural network to implicitly capture higher-order interactions. XDeepFM improves upon DeepFM by explicitly enumerating second-order interactions using the CIN module. To reduce the number of interaction pairs, CIN performs a weighted summation of the interaction results before further processing higher-order interactions. Although CIN reduces the computational cost of higher-order interactions, it still doesn't solve the problem posed by FM, namely the need to enumerate all combinations of feature pairs. DCN and its variant DCNV2 compress the entire feature matrix into a vector and use cross-layer information to extract feature interactions, achieving better performance in capturing implicit interactions. Recently, with the rise of Transformer models, AutoInt was proposed, utilizing a multi-head self-attention module to construct feature interactions. The query and key are used to evaluate feature similarity, and then the features are weighted and summed to obtain higher-order interaction information. It suffers from the problem of a large number of combinations of all weighted features, where low-weight features may introduce noise. To pursue more refined feature interactions, AFM maps the embedding space to a logarithmic space and eliminates features with zero weights. However, experimental results show that without any further constraints, it is almost impossible for the model to learn zero weights.
[0005] In terms of feature importance, evaluating the importance of feature interactions is very useful. It can be used to remove unimportant information, thereby reducing noise and improving the accuracy of CTR prediction. Many studies have focused on mining the importance weights of feature interactions. FwFM improves FM by assigning learnable weights to each interaction pair. IAFM considers both feature aspects and domain aspects, learning flexible interaction weights at two levels. FiBiNet uses a Squeeze-Excitation network to learn the weights and incorporates bilinear layers to model feature interactions.
[0006] Existing solutions suffer from several problems: First, all explicit feature cross-combination schemes require manual specification of the cross-combination order. This necessitates manual parameter tuning for different datasets to achieve optimal results. However, in real-world scenarios, insufficient resources for searching are often unavailable. Therefore, a solution is needed that can automatically learn the cross-combination order from the data. Second, existing solutions are computationally expensive. Because they construct feature sets through exhaustive enumeration, the time complexity is exponential, rendering most models incapable of performing high-order feature cross-combinations. Third, existing solutions introduce significant noise. Since most models exhaustively enumerate all feature sets, meaningless feature combinations, such as <occupation, weather>, are inevitably introduced. While existing solutions score these feature combinations based on feature importance to reduce noise, the results are unsatisfactory. Summary of the Invention
[0007] This application aims to at least partially address one of the technical problems in the related art.
[0008] Therefore, the first objective of this application is to propose a set-based top-down adaptive order feature crossing method, which solves the technical problems of existing methods requiring manual adjustment, high computational cost, and introducing a large amount of noise, and realizes effective feature crossing to improve the accuracy and speed of recommendations.
[0009] The second objective of this application is to propose a set-based top-down adaptive order feature crossing device.
[0010] To achieve the above objectives, the first aspect of this application proposes a set-based top-down adaptive order feature cross method, comprising: obtaining a complete feature set, wherein the complete feature set includes numerical features, category features, sequence features, and graph features; inputting the complete feature set into a neural network, outputting the probability of each feature belonging to each set, and then using a soft threshold-based pruning scheme to prune according to the probability of each feature belonging to each set to generate a first feature subset; pruning again using a soft threshold based on the first feature subset to generate a second feature subset; and performing feature cross on the second feature subset to obtain a click-through rate prediction result, wherein the click-through rate prediction result is the probability of a user clicking on an item.
[0011] The set-based top-down adaptive order feature cross method of this application differs from traditional factorization machines. By adopting a top-down approach to obtain feature combinations, it reduces the large amount of noise introduced by exhaustive search and can generate adaptive order feature crosses based on samples. Through the feature subset generation module and the set attention pruning module, fine-grained feature combinations of different sizes can be obtained, integrating useful information and ultimately improving the accuracy of recommendations.
[0012] Optionally, in one embodiment of this application, the numerical features include user age and average article reading time; the category features include user ID and product ID; the sequence features include the sequence of products that the user has browsed in the past time period; and the graph features include the topological neighbor features between users and products.
[0013] Optionally, in one embodiment of this application, the neural network is a neural network based on extended re-squeezing, and the first feature subset includes two features: permutation invariance of the input and variable number of input elements.
[0014] Optionally, in one embodiment of this application, pruning is performed again using a soft threshold based on the first feature subset, including:
[0015] Multiple attention scores are calculated using the derived key matrix and the first feature subset;
[0016] Based on the attention score of each group, pruning is performed again using a soft threshold to obtain the second feature set.
[0017] Optionally, in one embodiment of this application, feature crossing is performed on the second feature subset, including:
[0018] For the second feature subset, an additive function is used to obtain the crossover result for each feature subset;
[0019] The cross results are used to construct a high-order cross vector for the feature subset through a deep neural network. Then, the high-order cross vectors generated by each feature subset are concatenated to obtain the concatenated result.
[0020] The concatenated result is input into a deep neural network to obtain the final click-through rate prediction result.
[0021] To achieve the above objectives, a second aspect of the present invention proposes a set-based top-down adaptive order feature cross-processing device, comprising an acquisition module, a feature subset generation module, a set attention pruning module, and a prediction module, wherein...
[0022] The acquisition module is used to acquire the complete feature set, which includes numerical features, category features, sequence features, and graph features.
[0023] The feature subset generation module is used to input the entire feature set into the neural network, output the probability of each feature belonging to each set, and then use a soft threshold-based pruning scheme to prune according to the probability of each feature belonging to each set to generate the first feature subset.
[0024] The attention pruning module is used to prune the first feature subset again using a soft threshold to generate a second feature subset.
[0025] The prediction module is used to perform feature cross-referencing on the second feature subset to obtain the click-through rate prediction result, where the click-through rate prediction result is the probability that a user clicks on an item.
[0026] Optionally, in one embodiment of this application, the numerical features include user age and average article reading time; the category features include user ID and product ID; the sequence features include the sequence of products that the user has browsed in the past time period; and the graph features include the topological neighbor features between users and products.
[0027] Optionally, in one embodiment of this application, the neural network is a neural network based on extended re-squeezing, and the feature subset includes two features: permutation invariance of the input and variable number of input elements.
[0028] Optionally, in one embodiment of this application, the attention pruning module is specifically used for:
[0029] Multiple attention scores are calculated using the derived key matrix and the first feature subset;
[0030] Based on the attention score of each group, pruning is performed again using a soft threshold to obtain the second feature set.
[0031] Optionally, in one embodiment of this application, the prediction module is specifically used for:
[0032] For the second feature subset, an additive function is used to obtain the crossover result for each feature subset;
[0033] The cross results are used to construct a high-order cross vector for the feature subset through a deep neural network. Then, the high-order cross vectors generated by each feature subset are concatenated to obtain the concatenated result.
[0034] The concatenated result is input into a deep neural network to obtain the final click-through rate prediction result.
[0035] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0036] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0037] Figure 1 This is a comparison diagram of the factorization machine-based method and the set-based method in the embodiments of this application;
[0038] Figure 2 This is a flowchart illustrating the set-based top-down adaptive order feature cross method provided in Embodiment 1 of this application.
[0039] Figure 3 This is a structural diagram of the set-based top-down adaptive order feature cross recommendation method according to an embodiment of this application;
[0040] Figure 4 This is a schematic diagram of a set-based top-down adaptive order feature crossover device provided in an embodiment of this application. Detailed Implementation
[0041] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0042] The following describes, with reference to the accompanying drawings, a method and apparatus for set-based top-down adaptive order feature crossing based on embodiments of this application.
[0043] Unlike bottom-up feature combination methods based on FM, this application proposes a top-down adaptive order feature cross method based on sets. The differences between the two approaches are as follows: Figure 1 As shown, the traditional bottom-up approach to factorization machines uses enumeration to exhaustively list all 2nd / 3rd / Nth-order cross combinations, and then performs cross combinations on the features in each combination. This approach undoubtedly introduces a large number of meaningless cross combinations. The top-down approach, however, reshapes the problem as follows: first, how to extract information by dividing the entire feature set into multiple feature subsets with varying numbers of features, thus achieving adaptive factorization; second, after constructing the feature subsets, choosing the appropriate aggregation method to perform higher-order feature cross combinations on the subsets.
[0044] Figure 2 This is a flowchart illustrating the set-based top-down adaptive order feature cross method provided in Embodiment 1 of this application.
[0045] like Figure 2As shown, this set-based top-down adaptive order feature cross method includes the following steps:
[0046] Step 201: Obtain the complete feature set, which includes numerical features, category features, sequence features, and graph features.
[0047] Step 202: Input the entire feature set into the neural network and output the probability of each feature belonging to each set. Then, use a soft threshold-based pruning scheme to prune according to the probability of each feature belonging to each set to generate the first feature subset.
[0048] Step 203: Based on the first feature subset, prune again using a soft threshold to generate the second feature subset;
[0049] Step 204: Perform feature cross-referencing on the second feature subset to obtain the click-through rate prediction result, where the click-through rate prediction result is the probability of a user clicking on an item.
[0050] The set-based top-down adaptive order feature cross method of this application differs from traditional factorization machines. By adopting a top-down approach to obtain feature combinations, it reduces the large amount of noise introduced by exhaustive search and can generate adaptive order feature crosses based on samples. Through the feature subset generation module and the set attention pruning module, fine-grained feature combinations of different sizes can be obtained, integrating useful information and ultimately improving the accuracy of recommendations.
[0051] Optionally, in one embodiment of this application, the numerical features include user age and average article reading time; the category features include user ID and product ID; the sequence features include the sequence of products that the user has browsed in the past time period; and the graph features include the topological neighbor features between users and products.
[0052] Optionally, in one embodiment of this application, the neural network is a neural network based on extended re-squeezing, and the first feature subset includes two features: permutation invariance of the input and variable number of input elements.
[0053] Optionally, in one embodiment of this application, pruning is performed again using a soft threshold based on the first feature subset, including:
[0054] Multiple attention scores are calculated using the derived key matrix and the first feature subset;
[0055] Based on the attention score of each group, pruning is performed again using a soft threshold to obtain the second feature set.
[0056] Optionally, in one embodiment of this application, feature crossing is performed on the second feature subset, including:
[0057] For the second feature subset, an additive function is used to obtain the crossover result for each feature subset;
[0058] The cross results are used to construct a high-order cross vector for the feature subset through a deep neural network. Then, the high-order cross vectors generated by each feature subset are concatenated to obtain the concatenated result.
[0059] The concatenated result is then fed into a deep neural network to obtain the final click-through rate prediction result.
[0060] The following is combined with Figure 3 This application introduces another set-based top-down adaptive order feature cross recommendation method.
[0061] Figure 3 This is a structural diagram of the top-down adaptive order feature cross recommendation method based on sets in this application, as shown below. Figure 3As shown, the entire method is divided into three parts. For the first part, based on the "generate and then prune" architecture, we designed a feature subset generation module. From a holistic perspective, the feature subset generation module takes the entire feature set as input. This entire feature set is then passed through a neural network based on ExcitationThenSqueeze to model the correlation and importance of all features, generating the probability that a feature belongs to each set. Specifically, this neural network is a two-layer network. ExcitationThenSqueeze means that the first layer transforms the vector into a vector with a higher dimension than the original vector, and the second layer transforms the vector into a vector with a lower dimension than the original vector. To increase the sparsity and diversity of each feature subset, we adopted a soft-threshold pruning scheme. This method allows for the creation of different threshold matrices for each feature. When the probability of a feature corresponding to a set is less than the corresponding threshold, the feature is removed from the set. After passing through the first-stage feature subset generation module, each feature subset has a different number of features. The feature subsets generated in this stage have two main characteristics: First, due to the two significant properties of ensemble data—invariance to input permutations and variability in the number of input elements—we cannot use commonly used deep neural networks for ensemble modeling because deep neural networks do not possess these two properties. Second, although the feature subsets obtained by the feature subset generation module have undergone some pruning, they still contain a large number of elements. Directly performing feature crossover at this point would result in coarse-grained crossover and significant noise. To address these issues, we propose an ensemble attention pruning module based on self-attention and soft pruning. Specifically, the ensemble attention pruning module takes each feature subset as input and uses the derived key matrix and the feature subset to calculate multiple sets of attention scores to model the relationships and importance between features. For each set of attention scores, a soft threshold is applied for further pruning, removing attention scores with lower values to obtain a fine-grained feature set. Thus, through a top-down architecture and a generation and re-pruning scheme, we have constructed multiple feature subsets, each with elements varying according to the sample, thereby achieving adaptive order.
[0062] After obtaining the candidate set of feature crossovers, we perform feature crossovers on the feature subsets to obtain the final recommendation result. This scheme uses two different crossover functions: an additive function and a deep neural network crossover function. For the fine-grained set generated by the ensemble attention pruning module, we use a simple and efficient additive function to obtain the crossover result for each feature subset. Then, we use a deep neural network to construct a higher-order crossover for the feature subset. Finally, multiple feature subsets are crossovered again using a deep neural network to obtain the final click-through rate prediction result.
[0063] The training process of the aforementioned deep neural network is as follows: after feature crossing, a probability value is predicted to correspond to the probability in the click-through rate prediction. Then, a loss is obtained through a binary classification loss function, and the neural network is trained using gradient descent with the Adam optimizer.
[0064] The higher-order cross of the feature subset constructed above using a deep neural network is Figure 3 The output of the SAPN (SubsetAttention Pruning Network) module, for example, for a feature set [F,D], represents F features, each with D dimensions. Then, a transformation matrix [D,K] is set, and matrix multiplication is used to obtain the attention score [F,K], where K represents the score vector of each feature in K spaces. At this point, soft thresholding is performed on the [F,K] matrix, i.e., f([F,K])->[F,K]. Pruning means setting weights less than the threshold to zero. Then, reassignment is performed again, and additive crossover is performed, specifically [K,F]*[F,D]->[K,D], which is one crossover. A higher-order crossover vector is then obtained through the DNN; since multiple feature subsets were segmented in the first stage, each subset will generate a higher-order crossover vector. Finally, the higher-order vectors generated by each subset are concatenated and input into the final neural network to map to the product score.
[0065] To implement the above embodiments, this application also proposes a set-based top-down adaptive order feature crossing device.
[0066] Figure 4 This is a schematic diagram of a set-based top-down adaptive order feature crossover device provided in an embodiment of this application.
[0067] like Figure 4 As shown, this set-based top-down adaptive order feature cross-processing device includes an acquisition module, a feature subset generation module, a set attention pruning module, and a prediction module, wherein...
[0068] The acquisition module is used to acquire the complete feature set, which includes numerical features, category features, sequence features, and graph features.
[0069] The feature subset generation module is used to input the entire feature set into the neural network, output the probability of each feature belonging to each set, and then use a soft threshold-based pruning scheme to prune according to the probability of each feature belonging to each set to generate the first feature subset.
[0070] The attention pruning module is used to prune the first feature subset again using a soft threshold to generate a second feature subset.
[0071] The prediction module is used to perform feature cross-referencing on the second feature subset to obtain the click-through rate prediction result, where the click-through rate prediction result is the probability that a user clicks on an item.
[0072] Optionally, in one embodiment of this application, the numerical features include user age and average article reading time; the category features include user ID and product ID; the sequence features include the sequence of products that the user has browsed in the past time period; and the graph features include the topological neighbor features between users and products.
[0073] Optionally, in one embodiment of this application, the feature subset generation module is specifically used for:
[0074] The entire feature set is input into a neural network based on expansion and re-squeezing, and the output is the probability that each feature belongs to each set.
[0075] The feature subset includes two features: the permutation invariance of the input and the variable number of input elements.
[0076] Optionally, in one embodiment of this application, the attention pruning module is specifically used for:
[0077] Multiple attention scores are calculated using the derived key matrix and the first feature subset;
[0078] Based on the attention score of each group, pruning is performed again using a soft threshold to obtain the second feature set.
[0079] Optionally, in one embodiment of this application, the prediction module is specifically used for:
[0080] For the second feature subset, an additive function is used to obtain the crossover result for each feature subset;
[0081] The cross-results are used to construct a high-order cross vector for the feature subset using a deep neural network. Then, the high-order cross vectors generated by each feature subset are concatenated and input into the final neural network to obtain the final click-through rate prediction result.
[0082] It should be noted that the foregoing explanation of the method embodiment for set-based top-down adaptive order feature crossing also applies to the apparatus for set-based top-down adaptive order feature crossing in this embodiment, and will not be repeated here.
[0083] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0084] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0085] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0086] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0087] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0088] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0090] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A top-down adaptive order feature cross method based on sets, characterized in that, Includes the following steps: Obtain the complete feature set, wherein the complete feature set includes numerical features, category features, sequence features, and graph features; The entire set of features is input into the neural network, which outputs the probability of each feature belonging to each set. Then, a soft threshold-based pruning scheme is used to prune the features according to the probability of each feature belonging to each set, generating the first feature subset. Based on the first feature subset, pruning is performed again using a soft threshold to generate a second feature subset; The second feature subset is subjected to feature cross-interaction to obtain the click-through rate prediction result, wherein the click-through rate prediction result is the probability of a user clicking on an item; The entire set of features is input into the neural network, and the probability of each feature belonging to each set is output, including: The entire feature set is processed through an extended re-squeezing neural network to model the correlation and importance of all features and generate the probability of each feature belonging to each set. The feature crossing of the second feature subset includes: For the second feature subset, an additive function is used to obtain the cross-result for each feature subset; The cross results are used to construct a high-order cross vector for the feature subset through a deep neural network. Then, the high-order cross vectors generated by each feature subset are concatenated to obtain the concatenated result. The splicing result is input into the deep neural network to obtain the final click-through rate prediction result.
2. The method as described in claim 1, characterized in that, The numerical features include user age and average article reading time; the category features include user ID and product ID; the sequence features include the sequence of products that the user has viewed in the past time period; and the graph features include the topological neighbor features between users and products.
3. The method as described in claim 1, characterized in that, The first feature subset includes two features: permutation invariance of the input and variable number of input elements.
4. The method as described in claim 1, characterized in that, The step of pruning again using a soft threshold based on the first feature subset includes: Multiple attention scores are calculated using the derived key matrix and the first feature subset; Based on the attention score of each group, pruning is performed again using a soft threshold to obtain the second feature set.
5. A device for top-down adaptive order feature intersection based on sets, characterized in that, It includes an acquisition module, a feature subset generation module, an ensemble attention pruning module, and a prediction module, among which, The acquisition module is used to acquire a complete set of features, wherein the complete set of features includes numerical features, category features, sequence features, and graph features; The feature subset generation module is used to input the full set of features into the neural network, output the probability of each feature belonging to each set, and then use a soft threshold-based pruning scheme to prune according to the probability of each feature belonging to each set to generate the first feature subset. The ensemble attention pruning module is used to prune the first feature subset again using a soft threshold to generate a second feature subset. The prediction module is used to perform feature cross-referencing on the second feature subset to obtain a click-through rate prediction result, wherein the click-through rate prediction result is the probability of a user clicking on an item; The entire set of features is input into the neural network, and the probability of each feature belonging to each set is output, including: The entire feature set is processed through an extended re-squeezing neural network to model the correlation and importance of all features and generate the probability of each feature belonging to each set. The prediction module is specifically used for: For the second feature subset, an additive function is used to obtain the cross-result for each feature subset; The cross results are used to construct a high-order cross vector for the feature subset through a deep neural network. Then, the high-order cross vectors generated by each feature subset are concatenated to obtain the concatenated result. The splicing result is input into the deep neural network to obtain the final click-through rate prediction result.
6. The apparatus as claimed in claim 5, characterized in that, The numerical features include user age and average article reading time; the category features include user ID and product ID; the sequence features include the sequence of products that the user has viewed in the past time period; and the graph features include the topological neighbor features between users and products.
7. The apparatus as claimed in claim 5, characterized in that, The neural network is a neural network based on extended re-squeezing, and the feature subset includes two features: permutation invariance of the input and variable number of input elements.
8. The apparatus as claimed in claim 5, characterized in that, The collective attention pruning module is specifically used for: Multiple attention scores are calculated using the derived key matrix and the first feature subset; Based on the attention score of each group, pruning is performed again using a soft threshold to obtain the second feature set.