Feature preprocessing method, device, electronic device and storage medium

Abstract

The present invention discloses a feature preprocessing method, device, electronic device and storage medium; the present invention can obtain multiple target features, and obtain at least one domain vector corresponding to the target feature, the target feature is used to characterize the information to be processed in all fields, and each domain vector is used to characterize the information to be processed in each field; for the target feature, the corresponding domain vector is used to perform feature cross processing to obtain multiple cross features; the target feature and the cross feature are subjected to feature fusion to obtain a fused feature; the fused feature is subjected to importance processing to obtain the processed feature. In the present invention, by obtaining domain vectors of different fields corresponding to the target feature, and using the domain vector to perform feature cross processing to obtain a cross feature, the diversity of the feature can be improved; the cross feature and the target feature are fused, and the importance processing is performed based on the fused feature, which can improve the accuracy of the feature. Therefore, this scheme can improve the diversity and accuracy of the feature.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a feature preprocessing method, device, electronic device and storage medium. Background Art

[0002] In search, recommendation, and advertising scenarios, data features are generally preprocessed (feature engineering) before algorithm modeling is performed on them. The preprocessing process includes but is not limited to feature extraction, feature fusion and crossover, feature selection, and feature transformation.

[0003] In recent years, most of them have simulated feature engineering by adding model structures to automatically preprocess features, thereby reducing labor costs. Deep learning models generally encode features before subsequent forward propagation; for example, the input of a deep learning model encodes feature A into an embedding vector, each dimension of which is a dense value, so feature preprocessing is transformed into preprocessing of feature embedding vectors.

[0004] To this end, more and more deep learning models that incorporate automated feature engineering have emerged. By integrating some new structures, the model can adaptively adjust the feature vector during the learning process. The general adjustment directions include feature crossover, feature weighting, etc. Feature crossover is generally a combination of two or more features to form a new feature, and feature weighting is generally used to evaluate the importance of features. Among them, the FNN (Factorization Machine supported Neural Network) model directly combines the FM model and the DNN model to learn feature crossover information; the DCN (Deep Cross Network) model introduces the CrossNetwork structure to automatically perform high-order crossover of features; the PNN (Product-based Neural Network) model introduces the Product Layer to realize the inner product and outer product crossover of features, and the FiBiNET (Feature Importance and Bilinear feature Interaction Network) introduces the SENet structure and the Bilinear-Interaction structure to actively learn the weight coefficients of different features.

[0005] However, the features in the current preprocessing process are relatively simple and the accuracy of the features is low. Summary of the invention

[0006] The present invention provides a feature preprocessing method, device, electronic device and storage medium, which can improve the diversity and accuracy of features.

[0007] The present invention provides a feature preprocessing method, comprising:

[0008] Acquire multiple target features and acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field;

[0009] For the target features, the corresponding domain vector is used to perform feature cross processing to obtain multiple cross features;

[0010] Perform feature fusion on the target features and cross features to obtain fused features;

[0011] Importance processing is performed on the fused features to obtain the processed features.

[0012] The present invention also provides a feature preprocessing device, comprising:

[0013] An acquisition unit, used to acquire multiple target features and to acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field;

[0014] The cross unit is used to perform feature cross processing on the target feature using the corresponding domain vector to obtain multiple cross features;

[0015] A fusion unit is used to fuse the target feature and the cross feature to obtain a fusion feature;

[0016] The processing unit is used to perform importance processing on the fused features to obtain processed features.

[0017] In some embodiments, the cross unit is specifically used for:

[0018] Based on the i-th domain vector and the j-th domain vector, all domain vectors corresponding to the g-th target feature are aggregated to obtain an aggregated domain vector; wherein the i-th domain vector is the i-th domain vector in the domain vector corresponding to the h-th target feature, the j-th domain vector is the j-th domain vector in the domain vector corresponding to the g-th target feature, and i, j, h, and g are all positive integers;

[0019] Perform vector cross processing on the aggregated domain vector and all domain vectors corresponding to the h-th target feature to obtain the cross feature.

[0020] In some embodiments, the cross unit is specifically used for:

[0021] Determine the similarity between the i-th domain vector and the j-th domain vector;

[0022] Using similarity, update the j-th domain vector to obtain an updated j-th domain vector;

[0023] All updated j-th domain vectors are accumulated to obtain an aggregate domain vector.

[0024] In some embodiments, the cross unit is specifically used for:

[0025] Multiply the aggregate domain vector and the i-th domain vector element by element to obtain the i-th domain vector after multiplication;

[0026] All the multiplied i-th domain vectors are accumulated to obtain the cross feature.

[0027] In some embodiments, the feature preprocessing device is further used to:

[0028] Get the number of crossovers;

[0029] From a plurality of cross features, select a cross number of cross features;

[0030] The target features and cross features are fused to obtain fused features, including:

[0031] All target features and several cross features are fused to obtain fused features.

[0032] In some embodiments, the processing unit is specifically configured to:

[0033] Determine the importance weights corresponding to the fusion features;

[0034] Importance weights are used to weight the fusion features to obtain weighted fusion features;

[0035] The fused features and the weighted fused features are re-fused to obtain the processed features.

[0036] In some embodiments, the processing unit is specifically configured to:

[0037] Obtaining importance assessment network;

[0038] Use importance evaluation network to determine the importance weights corresponding to fusion features;

[0039] The importance evaluation network includes a first fully connected layer and a second fully connected layer. The first fully connected layer corresponds to a first weight matrix and a first activation function, and the second fully connected layer corresponds to a second weight matrix and a second activation function. The importance evaluation network is used to determine the importance weights corresponding to the fusion features, including:

[0040] Using a first weight matrix, weighting the fusion feature to obtain a first weighted fusion feature;

[0041] Using the first activation function, activating the first weighted fusion feature to obtain an activated feature;

[0042] The activation features are weighted using a second weight matrix to obtain weighted activation features.

[0043] A second activation function is used to activate the weighted activation features to obtain the importance weights corresponding to the fused features.

[0044] In some embodiments, the dimension of the first fully connected layer is K×N (K < N), and the dimension of the second fully connected layer is N×K, where N is the dimension of the fused features.

[0045] In some embodiments, the processed features include processed user features and processed item features. The processed item features are the features corresponding to the items. The feature preprocessing device is further configured to:

[0046] Rank the items based on the processed user features and the processed item features to obtain ranked items.

[0047] Display the ranked items.

[0048] The present invention also provides an electronic device, including a memory and a processor. The memory stores multiple instructions. The processor loads the instructions from the memory to execute the steps in any one of the feature preprocessing methods provided by the present invention.

[0049] The present invention also provides a computer-readable storage medium. The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the steps in any one of the feature preprocessing methods provided by the present invention.

[0050] The present invention discloses a feature preprocessing method, apparatus, electronic device, and storage medium. The present invention can obtain multiple target features and at least one domain vector corresponding to the target features. The target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field. Feature cross-processing is performed on the target features using the corresponding domain vectors to obtain multiple cross features. Feature fusion is performed on the target features and the cross features to obtain fused features. Importance processing is performed on the fused features to obtain processed features.

[0051] In the present invention, by obtaining domain vectors in different fields corresponding to the target features and performing feature cross using the domain vectors to obtain cross features, the diversity of the features can be improved. Feature fusion is performed on the cross features and the target features, and importance processing is performed based on the fused features, so that the accuracy of the features can be improved. Thus, the present solution can improve the diversity and accuracy of the features. Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions in the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0053] Figure 1a It is a scene schematic diagram of the feature preprocessing method provided by the present invention;

[0054] Figure 1b It is a flow chart of the feature preprocessing method provided by the present invention;

[0055] Figure 1c It is a schematic diagram of feature comparison provided by the present invention;

[0056] Figure 1d It is a structural schematic diagram of importance processing provided by the present invention;

[0057] Figure 2a It is a structural schematic diagram of the recommendation model provided by the present invention;

[0058] Figure 2b is a schematic diagram of the recommendation result provided by the present invention;

[0059] Figure 3 It is a structural schematic diagram of a feature preprocessing device provided by the present invention;

[0060] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0062] The invention provides a feature preprocessing method, a device, an electronic device and a storage medium.

[0063] The feature preprocessing device may be integrated in an electronic device, which may be a terminal, a server, or other device. The terminal may be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop computer, a desktop computer, a vehicle-mounted device, or other device; the server may be a single server or a server cluster composed of multiple servers. In some embodiments, the server may also be implemented in the form of a terminal.

[0064] In some embodiments, the feature preprocessing device may also be integrated into multiple electronic devices. For example, the feature preprocessing device may be integrated into a terminal and a server, and the terminal and the server may jointly implement the feature preprocessing method of the present invention.

[0065] For example, refer to Figure 1a The electronic device can obtain multiple target features and at least one domain vector corresponding to the target features, the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field; for the target features, the corresponding domain vectors are used to perform feature cross processing to obtain multiple cross features; the target features and the cross features are subjected to feature fusion to obtain fused features; the fused features are subjected to importance processing to obtain processed features.

[0066] In this embodiment, by obtaining domain vectors of different fields corresponding to the target feature and using the domain vectors to perform feature crossover to obtain crossover features, the diversity of features can be improved; the crossover features and target features are fused, and importance processing is performed based on the fused features to improve the accuracy of features. Therefore, this solution can improve the diversity and accuracy of features.

[0067] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0068] Artificial Intelligence (AI) is a technology that uses digital computers to simulate human perception of the environment, acquire knowledge, and use knowledge. This technology can enable machines to have functions similar to human perception, reasoning, and decision-making. Artificial intelligence technology mainly includes computer vision technology, speech processing technology, natural language processing technology, machine learning, deep learning, and other major directions.

[0069] When natural language processing technology is applied in the recommendation system, it can include candidate item recall, correlation calculation, and participation in model sorting (CTR / CVR) as a feature. Among them, candidate item recall is used to generate a set of items to be recommended, and generally the corresponding item set is obtained according to various recall algorithms. Correlation calculations are prevalent in various steps of the recommendation system, such as various text similarity algorithms in the recall algorithm and some correlation calculations used in user portrait calculations. In the sorting layer after the item set is recalled, text features can often provide a lot of information, thus becoming an important sorting feature.

[0070] In this embodiment, a feature preprocessing method is provided, such as Figure 1b As shown, the specific process of the feature preprocessing method can be as follows:

[0071] 110. Acquire multiple target features and acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field.

[0072] The target features and domain vectors may include user features and item features, both of which can be dense values. Figure 1c As shown, Figure 1c It is a schematic diagram of feature comparison, where the black origin represents a dense value; that is, a comparison diagram of the target feature A and the domain vector corresponding to the target feature A, assuming that the number of domains is n.

[0073] The information to be processed may be the user's historical behavior information, the user's basic information, item information, etc.; among them, the historical behavior information may be the number of clicks on a certain type of live broadcast, the number or duration of video playback, comments, replies, likes, etc.; basic information may include age, gender, etc.; item information may include video or live broadcast tags, basic item information, etc.

[0074] Fields can be divided according to business scenarios; for example, in the live broadcast recommendation scenario, it can be divided into game live broadcast field, entertainment live broadcast field, and voice live broadcast field, etc.

[0075] In some embodiments, the information to be processed in all fields is vectorized to obtain target features. The information to be processed in each field is vectorized separately to obtain a domain vector for each field. The specific implementation method of vectorization is not limited, and one or more combinations of methods such as One-hot encoding, information retrieval (IR) technology, distributed representation, matrix-based distribution representation, clustering-based distribution representation, neural network-based distribution representation (such as Word2vec), word embedding (Embedding) encoding, etc. can be used. In this way, a set of domain vectors can be expanded according to the target features, increasing the diversity of features.

[0076] 120. For the target feature, the corresponding domain vector is used to perform feature cross processing to obtain multiple cross features.

[0077] For multiple target features, corresponding domain vectors may be used to perform feature cross processing between each two of the target features. The cross features may be used to represent cross information combining information to be processed in different fields.

[0078] In some embodiments, the target feature is subjected to feature cross processing using the corresponding domain vector to obtain multiple cross features, which may include but are not limited to:

[0079] (i) Based on the i-th domain vector and the j-th domain vector, all domain vectors corresponding to the g-th target feature are aggregated to obtain an aggregated domain vector.

[0080] Among them, the i-th domain vector is the i-th domain vector in the domain vector corresponding to the h-th target feature, the j-th domain vector is the j-th domain vector in the domain vector corresponding to the g-th target feature, and i, j, h, and g are all positive integers. It should be noted that the i-th domain vector and the j-th domain vector may correspond to the same or different fields.

[0081] Optionally, based on the i-th domain vector and the j-th domain vector, all domain vectors corresponding to the g-th target feature are aggregated to obtain an aggregated domain vector, which may include but is not limited to:

[0082] i. Determine the similarity between the i-th domain vector and the j-th domain vector. The inner product of each i-th domain vector and each j-th domain vector may be calculated as the similarity.

[0083] ii. Using the similarity, update the j-th domain vector to obtain an updated j-th domain vector. For example, the similarity can be multiplied by the element of each dimension in the j-th domain vector to obtain an updated j-th domain vector.

[0084] iii. Accumulate all updated j-th domain vectors to obtain an aggregate domain vector.

[0085] Optionally, only the inner product of the domain vectors corresponding to the same domain in the h-th target feature and the g-th target feature may be calculated, and the corresponding domain vector in the g-th target feature may be updated using the inner product to obtain an updated domain vector; then the domain vectors may be accumulated to obtain an aggregated vector.

[0086] (ii) Perform vector cross processing on the aggregated domain vector and all domain vectors corresponding to the h-th target feature to obtain a cross feature.

[0087] Optionally, the aggregation domain vector and the i-th domain vector may be element-wise multiplied to obtain the multiplied i-th domain vector; and all the multiplied i-th domain vectors may be accumulated to obtain the cross feature.

[0088] For example, the process of feature interaction processing can be called feature extraction (FE), or FE structure. Assume that the g-th target feature is target feature A, the h-th target feature is target feature B, the number of fields is n, and the dimension of the target vector and domain vector is K. The domain vector corresponding to field-i of target feature A is The domain vector of the target feature B corresponding to field-i is The cross feature between target feature A and target feature B can be calculated using the following formula:

[0089]

[0090] Among them, FE_Attention(A,B) represents the cross feature, which is also a cross vector with dimension K; represents the inner product operation, and obtains the scalar ⊙ represents element-wise multiplication, resulting in a vector × when performing operations on scalars and vectors means multiplying the elements of each dimension of the scalar and vector separately.

[0091] The feature cross-processing of target feature A and target feature B is the mutual cross-processing of vectors in multiple fields (also known as the attention mechanism Attention), which avoids the coupling of feature cross-processing between different target features and the same target feature to a certain extent. For example, in the prior art, when A is cross-processed with B and C at the same time, the model will update A according to the cross-features of A and B and the cross-features of A and C at the same time during learning and updating, which will cause the cross-features to influence each other and have a certain impact on model training. In this embodiment, in the process of back propagation, the domain update will update the corresponding domain vector according to the similarity. For example, the domain vectors of A and B on the first field are relatively similar, and the other similarities are very low; while A and C are only similar in the embedding on the second field. Then, when back propagating and updating, the gradient of the sample corresponding to the cross-features of A and B will act more on the domain vector of the first field, and the sample corresponding to the cross-features of A and C will act more on the domain vector of the second field. That is, after the multi-domain Attention method performs feature cross processing, the update between cross features will be refined to the update of field domain vectors. The cross features of A and B will update some field vectors corresponding to A, while the cross features of A and C will update other field vectors corresponding to A, which effectively alleviates the problem of feature vector update coupling. In addition, feature cross-pollination of target features based on the trained feature cross network can improve feature accuracy.

[0092] 130. Perform feature fusion on the target feature and the cross feature to obtain the fused feature.

[0093] The specific implementation of the feature fusion in this embodiment is not limited, for example, the vectors may be concatenated head to tail.

[0094] In some embodiments, the number of crosses can be obtained. The number of crosses can be a hyperparameter and can be configured according to the actual application scenario. Then, from multiple cross features, the number of cross features can be selected. The selection can be performed by a trained feature interaction network, randomly, or according to the business. All target features and the number of cross features are subjected to feature fusion to obtain fused features.

[0095] For example, assume that there are three target features A, B, and C in total. The number of crosses can be set to 0 - 3. The cross features can include the cross feature AB obtained by performing feature cross - processing on A and B, the cross feature AC obtained by performing feature cross - processing on A and C, and the cross feature CB obtained by performing feature cross - processing on C and B. Four cross features such as none / AB / AC / BC can be selected.

[0096] 140. Perform importance processing on the fused features to obtain processed features.

[0097] Among them, the processed features can be used to represent the target features after pre - processing.

[0098] In some embodiments, performing importance processing on the fused features to obtain processed features may include, but is not limited to:

[0099] (1) Determine the importance weights corresponding to the fused features.

[0100] In some embodiments, an importance evaluation network can be obtained; the importance evaluation network is used to determine the importance weights corresponding to the fused features.

[0101] Optionally, the importance evaluation network includes a first fully - connected layer and a second fully - connected layer. The first fully - connected layer corresponds to a first weight matrix and a first activation function, and the second fully - connected layer corresponds to a second weight matrix and a second activation function. Among them, the first weight matrix and the second weight matrix can be obtained through training; the first activation function and the second activation function can be the same or different. For example, they can be ReLU functions.

[0102] The first weight matrix can be used to perform weighted processing on the fused features to obtain a first weighted fused feature; the first activation function is used to perform activation processing on the first weighted fused feature to obtain an activation feature; the second weight matrix is used to perform weighted processing on the activation feature to obtain a weighted activation feature; the second activation function is used to perform activation processing on the weighted activation feature to obtain the importance weights corresponding to the fused features.

[0103] In some embodiments, the dimension of the first fully - connected layer is K×N (K < N), and the dimension of the second fully - connected layer is N×K, where N is the dimension of the fused features.

[0104] (2) Weight the fused features using importance weights to obtain weighted fused features.

[0105] (3) Perform feature re - fusion on the fused features and the weighted fused features to obtain processed features.

[0106] For example, as Figure 1d shown, it is a schematic structural diagram of importance processing (Feature Screen, FS). As Figure 1d shown, the FS structure mainly consists of a first fully - connected layer and a second fully - connected layer (importance evaluation network). The dimension of the first fully - connected layer is K×N (K < N). The first fully - connected layer is mainly used to compress the fused features and can play a role in feature dimensionality reduction. The dimension of the second fully - connected layer is N×K. Among them, N is the dimension of the fused features. The second fully - connected layer is mainly used for weight generation to restore the intermediate - layer vector to the input dimension. The first activation function and the second activation function both select the ReLU function. Assuming the concatenated fused feature is X, then the importance processing can be performed using the following formula:

[0107] FS(X) = ReLU(W2ReLU(W1X))⊙X + X

[0108] Among them, FS(X) represents the processed feature, W1 represents the first weight matrix, W2 represents the first weight matrix, and ReLU represents the ReLU function.

[0109] The importance weight value output by the importance evaluation network is multiplied by the fused features, and after multiplication, it is fused with itself. Therefore, the FS structure can measure feature importance. Assuming this structure is applied to a recommendation model, the larger the output of the FS structure, the greater the impact of the feature component on the output of the recommendation model, which to a certain extent indicates the correlation between the feature component and the output. After the recommendation model has completed learning, important features and their components can be screened out through this structure, improving the interpretability of the model. According to the important features, relevant recommendation results can be further explained, alleviating the "black - box" nature of the recommendation model.

[0110] In some embodiments, the FS structure can be applied to all connections between layers and can calculate the weight coefficients of each feature component in the input of the next - layer network.

[0111] In some embodiments, in the recommendation system, the processed features may include processed user features and processed item features, and the processed item features are features corresponding to the items. Based on the processed user features and the processed item features, the items are sorted to obtain sorted items. A deep learning model may be used to sort the items, such as a DIN (deep interest network) model. The sorted items may then be displayed, for example, through a graphical user interface provided by an electronic device.

[0112] As can be seen from the above, this embodiment can obtain domain vectors of different fields corresponding to the target feature and use the domain vectors to cross the features to obtain cross features, thereby improving the diversity of features; cross features and target features are fused, and importance processing is performed based on the fused features to improve the accuracy of features. Therefore, this solution can improve the diversity and accuracy of features.

[0113] The feature preprocessing solution provided by the present invention can be applied in various recommendation, search, and advertising scenarios. For example, taking the live broadcast recommendation scenario as an example, Figure 2a As shown, this is a recommendation model provided by this embodiment, and the feature preprocessing scheme of the present invention is integrated into the DIN model to sort the live homepage. The feature preprocessing method of the present invention will be further described in detail below. The specific process of a feature preprocessing method is as follows:

[0114] 210. Acquire multiple target features, and acquire at least one domain vector corresponding to the target features.

[0115] Among them, the fields can be divided into the field of game live broadcast, the field of entertainment live broadcast, and the field of voice live broadcast. The target features can be user features or item features in all fields, such as the target features can be features corresponding to age, features corresponding to gender, the number of views of live broadcasts, etc. The first domain vector can be a user feature or item feature in the field of game live broadcast, such as the number of views of watching game live broadcasts; the second domain vector can be a user feature or item feature in the field of entertainment live broadcast, such as the number of views of watching entertainment live broadcasts; the third domain vector can be a user feature or item feature in the field of voice live broadcast, such as the number of views of listening to voice live broadcasts.

[0116] It should be noted that the target vector can use the embedding vector obtained by the embedding coding layer.

[0117] 220. For the target feature, the corresponding domain vector is used to perform feature cross processing to obtain multiple cross features.

[0118] It should be noted that user features and item features can be input into the FE structure separately to perform feature cross processing respectively, and obtain cross features corresponding to user features and cross features corresponding to item features. The FE structure introduces the concept of field on the basis of embedding encoding, so that feature encoding is expanded from single encoding to multi-domain encoding, so that features have their own personalized encoding methods maintained in different domains, and at the same time defines the Attention mechanism under multi-domain vectors, which greatly enriches the input features and feature interactions of the recommendation model.

[0119] 230. Perform feature fusion on the target feature and the cross feature to obtain a fused feature.

[0120] The user feature and the corresponding cross feature are spliced ​​to obtain the corresponding fused feature; the item feature and the corresponding cross feature are spliced ​​to obtain the corresponding fused feature. It should be noted that the fused feature corresponding to the user feature and the fused feature corresponding to the item feature can also be spliced ​​to obtain the final fused feature.

[0121] 240. Perform importance processing on the fused features to obtain processed features.

[0122] In this invention, the FS structure is applied between the feature interaction network and the DIN model to evaluate the impact of the feature components passed into the DIN on the learning of subsequent network layers. The FS structure can be used to learn importance weights, which are used to evaluate the importance of related vectors in the fused features, play a role in information filtering and increase the interpretability of the recommendation model.

[0123] The combination of FE structure and FS structure can evaluate the feature importance of original target features and cross-features based on feature intersection, so that the recommendation model can automatically expand and filter features, ensuring the diversity and accuracy of features.

[0124] 250. Based on the processed features, the items are sorted to obtain sorted items, and the sorted items are displayed.

[0125] The item may refer to a live broadcast room. The sorted live broadcast rooms may be displayed on a graphical user interface provided by the electronic device.

[0126] like Figure 2b , which is a schematic diagram of a recommendation result provided by this embodiment, and is an AUC (area under the curve) result of an offline validation set. It can be found that there is a stable improvement effect on the accuracy of the recommendation, and the absolute improvement is basically around 0.15%.

[0127] As can be seen from the above, this embodiment can improve the diversity of features by obtaining domain vectors of different fields corresponding to the target features and using the domain vectors to cross features to obtain cross features; cross features and target features are fused, and importance processing is performed based on the fused features to improve the accuracy of features. Therefore, this solution can improve the diversity and accuracy of features. Recommending items based on the preprocessed features can improve the accuracy of recommendations.

[0128] In order to better implement the above method, the present invention further provides a feature preprocessing device. For example, in this embodiment, the method of the present invention is described in detail by taking the feature preprocessing device specifically integrated in an electronic device as an example.

[0129] For example, Figure 3 As shown, the feature preprocessing device may include an acquisition unit 301, a cross unit 302, a fusion unit 303 and a processing unit 304, as follows:

[0130] An acquisition unit 301 is used to acquire multiple target features and acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field;

[0131] A crossover unit 302 is used to perform feature crossover processing on the target feature using the corresponding domain vector to obtain multiple crossover features;

[0132] A fusion unit 303 is used to fuse the target feature and the cross feature to obtain a fusion feature;

[0133] The processing unit 304 is used to perform importance processing on the fused features to obtain processed features.

[0134] In some embodiments, the cross unit 302 is specifically used to:

[0135] Based on the i-th domain vector and the j-th domain vector, all domain vectors corresponding to the g-th target feature are aggregated to obtain an aggregated domain vector; wherein the i-th domain vector is the i-th domain vector in the domain vector corresponding to the h-th target feature, the j-th domain vector is the j-th domain vector in the domain vector corresponding to the g-th target feature, and i, j, h, and g are all positive integers;

[0136] Perform vector cross processing on the aggregated domain vector and all domain vectors corresponding to the h-th target feature to obtain the cross feature.

[0137] In some embodiments, the cross unit 302 is specifically used to:

[0138] Determine the similarity between the i-th domain vector and the j-th domain vector;

[0139] Using similarity, update the j-th domain vector to obtain an updated j-th domain vector;

[0140] All updated j-th domain vectors are accumulated to obtain an aggregate domain vector.

[0141] In some embodiments, the cross-connect unit 302 is specifically used for:

[0142] Multiply the aggregate domain vector and the i-th domain vector element by element to obtain the i-th domain vector after multiplication;

[0143] All the multiplied i-th domain vectors are accumulated to obtain the cross feature.

[0144] In some embodiments, the feature preprocessing device is further used to:

[0145] Get the number of crossovers;

[0146] From a plurality of cross features, select a cross number of cross features;

[0147] The target features and cross features are fused to obtain fused features, including:

[0148] All target features and several cross features are fused to obtain fused features.

[0149] In some embodiments, the processing unit 304 is specifically configured to:

[0150] Determine the importance weights corresponding to the fusion features;

[0151] Importance weights are used to weight the fusion features to obtain weighted fusion features;

[0152] The fused features and the weighted fused features are re-fused to obtain the processed features.

[0153] In some embodiments, the processing unit 304 is specifically configured to:

[0154] Obtaining importance assessment network;

[0155] Use importance evaluation network to determine the importance weights corresponding to fusion features;

[0156] The importance evaluation network includes a first fully connected layer and a second fully connected layer. The first fully connected layer corresponds to a first weight matrix and a first activation function, and the second fully connected layer corresponds to a second weight matrix and a second activation function. The importance evaluation network is used to determine the importance weights corresponding to the fusion features, including:

[0157] Using a first weight matrix, weighting the fusion feature to obtain a first weighted fusion feature;

[0158] Using a first activation function, perform activation processing on the first weighted fusion feature to obtain an activation feature;

[0159] Using a second weight matrix, perform weighting processing on the activation feature to obtain a weighted activation feature;

[0160] Using a second activation function, perform activation processing on the weighted activation feature to obtain the importance weight corresponding to the fusion feature.

[0161] In some embodiments, the dimension of the first fully connected layer is K×N (K < N), and the dimension of the second fully connected layer is N×K, where N is the dimension of the fusion feature.

[0162] In some embodiments, the processed features include processed user features and processed item features. The processed item features are the features corresponding to the item. The feature preprocessing device is further configured to:

[0163] Based on the processed user features and the processed item features, sort the items to obtain sorted items;

[0164] Display the sorted items.

[0165] In specific implementation, the above-mentioned each unit can be implemented as an independent entity, or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of the above-mentioned each unit, reference can be made to the foregoing method embodiments, which will not be elaborated herein.

[0166] As can be seen from the above, the feature preprocessing device in this embodiment can obtain domain vectors in different fields corresponding to the target feature, and perform feature crossing using the domain vectors to obtain crossing features, which can improve the diversity of features; fuse the crossing features and the target feature, and perform importance processing based on the fusion feature, which can improve the accuracy of the features. Therefore, this solution can improve the diversity and accuracy of the features.

[0167] Correspondingly, an embodiment of the present application further provides an electronic device. For example, in this embodiment, the electronic device will be taken as an example of a terminal device for detailed description, as Figure 4 shown, Figure 4 is a schematic structural diagram of the electronic device provided by the embodiment of the present application. The electronic device 400 includes a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, and a computer program stored in the memory 402 and executable on the processor. Among them, the processor 401 is electrically connected to the memory 402. Those skilled in the art can understand that the structural diagram of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine some components, or arrange different components.

[0168] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device 400, executes various functions of the electronic device 400 and processes data by running or loading software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, thereby monitoring the electronic device 400 as a whole.

[0169] In the embodiment of the present application, the processor 401 in the electronic device 400 will load instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 will run the application programs stored in the memory 402 to implement various functions:

[0170] Acquire multiple target features and acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field;

[0171] For the target features, the corresponding domain vector is used to perform feature cross processing to obtain multiple cross features;

[0172] Perform feature fusion on the target features and cross features to obtain fused features;

[0173] Importance processing is performed on the fused features to obtain the processed features.

[0174] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0175] Optional, such as Figure 4 As shown, the electronic device 400 further includes: a touch screen 403, a radio frequency circuit 404, an audio circuit 405, an input unit 406, and a power supply 407. The processor 401 is electrically connected to the touch screen 403, the radio frequency circuit 404, the audio circuit 405, the input unit 406, and the power supply 407, respectively. Those skilled in the art can understand that Figure 4 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0176] The touch display screen 403 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 403 may include a display panel and a touch panel. Among them, the display panel may be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces may be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-Emitting Diode) and the like. The touch panel may be used to collect the user's touch operation on or near it (such as the user using any suitable object or attachment such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 401, and can receive the command sent by the processor 401 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 401 to determine the type of touch event, and then the processor 401 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 403 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 403 can also be used as a part of the input unit 406 to realize the input function.

[0177] In the embodiment of the present application, the processor 401 executes the live broadcast application to generate a graphical user interface on the touch screen 403, and the graphical user interface includes at least one live broadcast room. The touch screen 403 is used to present the graphical user interface and receive operation instructions generated by the user acting on the graphical user interface.

[0178] The radio frequency circuit 404 may be used to send and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to send and receive signals with the network device or other electronic devices.

[0179] The audio circuit 405 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 405 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 405 and converted into audio data, and then the audio data is output to the processor 401 for processing, and then sent to another electronic device through the radio frequency circuit 404, or the audio data is output to the memory 402 for further processing. The audio circuit 405 may also include an earplug jack to provide communication between an external headset and an electronic device.

[0180] The input unit 406 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0181] The power supply 407 is used to supply power to various components of the electronic device 400. Optionally, the power supply 407 can be logically connected to the processor 401 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 407 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0182] although Figure 4 Not shown, the electronic device 400 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0183] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0184] As can be seen from the above, the electronic device provided in this embodiment can obtain domain vectors of different fields corresponding to the target feature, and use the domain vectors to cross the features to obtain cross features, which can improve the diversity of features; cross features and target features are fused, and importance processing is performed based on the fused features, which can improve the accuracy of features. Therefore, this solution can improve the diversity and accuracy of features.

[0185] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0186] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored, and the computer program can be loaded by a processor to execute the steps in any feature preprocessing method provided in the embodiment of the present application. For example, the computer program can execute the following steps:

[0187] Acquire multiple target features and acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each domain vector is used to represent the information to be processed in each field;

[0188] For the target features, the corresponding domain vector is used to perform feature cross processing to obtain multiple cross features;

[0189] Perform feature fusion on the target features and cross features to obtain fused features;

[0190] Importance processing is performed on the fused features to obtain the processed features.

[0191] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0192] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0193] Since the computer program stored in the storage medium can execute the steps in any one of the feature preprocessing methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the feature preprocessing methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0194] The above is a detailed introduction to a feature preprocessing method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A feature preprocessing method, characterized in that: include: Acquire multiple target features and acquire at least one domain vector corresponding to the target features, wherein the target features are used to represent the information to be processed in all fields, and each of the domain vectors is used to represent the information to be processed in each field, wherein the information to be processed includes user information and / or item information; Based on the i-th domain vector and the j-th domain vector, all the domain vectors corresponding to the g-th target feature are aggregated to obtain an aggregated domain vector; wherein the i-th domain vector is the i-th domain vector in the domain vectors corresponding to the h-th target feature, the j-th domain vector is the j-th domain vector in the domain vectors corresponding to the g-th target feature, and i, j, h, and g are all positive integers; Performing vector cross processing on the aggregated domain vector and all the domain vectors corresponding to the h-th target feature to obtain a cross feature; Performing feature fusion on the target feature and the cross feature to obtain a fused feature; Importance processing is performed on the fused features to obtain processed features.

2. The feature preprocessing method according to claim 1, characterized in that: The step of aggregating all the domain vectors corresponding to the g-th target feature based on the i-th domain vector and the j-th domain vector to obtain an aggregated domain vector includes: Determining the similarity between the i-th domain vector and the j-th domain vector; Using the similarity, updating the j-th domain vector to obtain an updated j-th domain vector; All the updated j-th domain vectors are accumulated to obtain the aggregate domain vector.

3. The feature preprocessing method according to claim 1, characterized in that: The performing vector cross processing on the aggregated domain vector and all the domain vectors corresponding to the hth target feature to obtain the cross feature includes: Multiply the aggregate domain vector and the i-th domain vector element-wise to obtain the i-th domain vector after the multiplication; All the multiplied i-th domain vectors are accumulated to obtain the cross feature.

4. The feature preprocessing method according to claim 1, characterized in that: After obtaining the cross-feature, the method further includes: Get the number of crossovers; Selecting the intersecting number of intersecting features from the plurality of intersecting features; The step of fusing the target feature and the cross feature to obtain a fused feature includes: All the target features and the cross features are fused to obtain fused features.

5. The feature preprocessing method according to claim 1, characterized in that: The importance processing of the fused features to obtain processed features includes: Determine the importance weight corresponding to the fusion feature; The fusion feature is weighted by using the importance weight to obtain a weighted fusion feature; The fused features and the weighted fused features are re-fused to obtain processed features.

6. The feature preprocessing method according to claim 5, characterized in that: The determining the importance weight corresponding to the fusion feature includes: Obtaining importance assessment network; Using the importance evaluation network, determining the importance weights corresponding to the fusion features; The importance evaluation network includes a first fully connected layer and a second fully connected layer, the first fully connected layer corresponds to a first weight matrix and a first activation function, the second fully connected layer corresponds to a second weight matrix and a second activation function, and the importance evaluation network is used to determine the importance weight corresponding to the fusion feature, including: Using the first weight matrix, perform weighted processing on the fusion feature to obtain a first weighted fusion feature; Using a first activation function, perform activation processing on the first weighted fusion feature to obtain an activation feature; Using the second weight matrix to perform weighted processing on the activation feature to obtain a weighted activation feature; Using the second activation function, perform activation processing on the weighted activation feature to obtain the importance weight corresponding to the fusion feature.

7. The feature preprocessing method according to claim 6, characterized in that: The dimension of the first fully connected layer is K×N (K < N), and the dimension of the second fully connected layer is N×K, where N is the dimension of the fusion feature.

8. The feature preprocessing method according to claim 1, characterized in that: The processed features include processed user features and processed item features. The processed item features are features corresponding to the item. The method further includes: Based on the processed user features and the processed item features, perform sorting on the items to obtain sorted items; Display the sorted items.

9. A feature preprocessing device, characterized in that: Including: An acquisition unit, configured to acquire a plurality of target features and at least one domain vector corresponding to the target features. The target features are used to represent the information to be processed in all domains, and each domain vector is used to represent the information to be processed in each domain. The information to be processed includes user information and / or item information; A cross unit, configured to perform aggregation processing on all the domain vectors corresponding to the g-th target feature based on the i-th domain vector and the j-th domain vector to obtain an aggregated domain vector; where the i-th domain vector is the i-th domain vector among the domain vectors corresponding to the h-th target feature, and the j-th domain vector is the j-th domain vector among the domain vectors corresponding to the g-th target feature. i, j, h, and g are all positive integers. Perform vector cross processing on the aggregated domain vector and all the domain vectors corresponding to the h-th target feature to obtain a cross feature; A fusion unit, configured to perform feature fusion on the target feature and the cross feature to obtain a fusion feature; A processing unit, configured to perform importance processing on the fusion feature to obtain processed features.

10. An electronic device, characterized in that: Including a processor and a memory. The memory stores multiple instructions; the processor loads the instructions from the memory to execute the steps in the feature preprocessing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the feature preprocessing method according to any one of claims 1 to 8.

Images