Combined feature determination method and device, storage medium and electronic equipment

Abstract

The invention discloses a combined feature determination method and device, a storage medium and electronic equipment. The method comprises the steps that an initial recommendation model is trained based on an initial feature set, a first recommendation model corresponding to a first evaluation model index is obtained, the feature number of each feature group is determined according to a preset order, and a plurality of feature groups matched with the feature number of each feature group are obtained, and training the initial recommendation model based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to the second evaluation model index, and when the first evaluation model index and the second evaluation model index indicate that the model effect of the second recommendation model is superior to that of the first recommendation model, performing recommendation on the first recommendation model. And determining the candidate combined features as target combined features. According to the method and the apparatus, the technical problem of relatively low combined feature determination efficiency caused by a complex combined feature determination mode is solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular, to a method and apparatus for determining combined features, a storage medium, and an electronic device. Background Art

[0002] Currently, in the process of training a recommendation model, if one wants to train the recommendation model with combined features to improve the recommendation effect of the recommendation model, generally, a large number of initial features are simply concatenated, and then some of the concatenated combined features are selected relying on manual experience, and the combined features are used to input the recommendation model for training. For example, if there are N initial features and a combined feature is formed by concatenating two initial features, then such combined features will be generated. Due to the excessive number of combined features, the training cost of the recommendation model is high, the training time is long, and the evaluation cost is too large.

[0003] For the above problems, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of the present application provide a method and apparatus for determining combined features, a storage medium, and an electronic device, so as to at least solve the technical problem that the determination efficiency of combined features is low due to the complex determination method of combined features.

[0005] According to one aspect of the embodiments of the present application, a method for determining combined features is provided, including: training an initial recommendation model based on an initial feature set to obtain a first recommendation model corresponding to a first evaluation model metric; determining the number of features in each feature group according to a preset order, grouping the initial feature set to obtain a plurality of feature groups respectively matching the number of features in each feature group; when the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, training the initial recommendation model based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to a second evaluation model metric; when the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model, determining the candidate combined features as target combined features.

[0006] According to another aspect of the embodiments of the present application, there is also provided a device for determining combined features, including: an acquisition module, configured to train an initial recommendation model based on an initial feature set to obtain a first recommendation model corresponding to a first evaluation model metric; a grouping module, configured to determine the number of features in each feature group according to a preset order, group the initial feature set to obtain a plurality of feature groups respectively matching the number of features in each feature group; a training module, configured to, when the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, train the initial recommendation model based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to a second evaluation model metric; a determination module, configured to, when the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model, determine the candidate combined features as target combined features.

[0007] Optionally, the device is configured to group the initial feature set according to the preset order to determine the number of features in each feature group, and obtain a plurality of feature groups respectively matching the number of features in each feature group in the following manner: a processing unit, configured to input the initial feature set into a pre-trained target feature grouping model to determine a target probability matrix, where the number of rows of the target probability matrix is the same as the number of features in the initial feature set, the number of columns of the target probability matrix is the same as the number of feature groups of the plurality of feature groups, and the elements in the target probability matrix are used to represent the probability from a target initial feature group to a target feature group; a grouping unit, configured to group the initial feature set according to the target probability matrix to obtain the plurality of feature groups.

[0008] Optionally, the device is further configured to: before inputting the initial feature set into the pre-trained target feature grouping model, obtain a sample feature set and sample annotation results corresponding to each sample feature in the sample feature set, where the sample feature set has the same feature category as the initial feature set, and the sample annotation results are used to represent the true recommendation results corresponding to the sample features; input the sample feature set and the sample annotation results into an initial feature grouping model to be trained to obtain predicted recommendation results, and use the true recommendation results and the predicted recommendation results to train the initial feature grouping model to obtain the target feature grouping model.

[0009] Optionally, the device is configured to input the sample feature set and the sample annotation result into an initial feature grouping model to be trained, obtain a predicted recommendation result, and use the true recommendation result and the predicted recommendation result to train the initial feature grouping model to obtain the target feature grouping model in the following manner: Input the sample feature set and the sample annotation result into the initial feature grouping model to obtain an initial probability matrix, where the number of rows and columns of the initial probability matrix is the same as that of the target probability matrix, and the values of the elements in the initial probability matrix are randomly set; Automatically group the sample features in the sample feature set according to the initial probability matrix to obtain a plurality of sample feature groupings, where the number of features in the sample feature set grouped into each feature grouping of the plurality of feature groupings is the preset order; Perform an encoding operation on each sample feature in the sample feature set to determine a feature vector set, where the feature vectors in the feature vector set correspond one-to-one to the sample features in the sample feature set; Determine a plurality of feature grouping vectors according to the feature vector set and the plurality of sample feature groupings, where the plurality of feature grouping vectors correspond one-to-one to the plurality of sample feature groupings; Input the plurality of feature grouping vectors into a multi-layer perceptron to determine a predicted recommendation result, and use the true recommendation result and the predicted recommendation result to train the initial feature grouping model to obtain the target feature grouping model, where when the initial feature grouping model is trained into the target feature grouping model, the initial probability matrix is simultaneously trained into the target probability matrix.

[0010] Optionally, the device is configured to perform intra-group feature combination on the features of each feature grouping respectively and obtain candidate combined features corresponding to each feature grouping in the following manner: Perform intra-group feature combination on the features of each feature grouping respectively and obtain candidate combined features corresponding to each feature grouping in the following manner, where the feature grouping for which intra-group feature combination is performed each time is regarded as the current feature grouping: Screen out N current features from the current feature grouping, where the N current features are the N features with the highest probability of being grouped into the current feature grouping determined according to the target probability matrix in the current feature grouping, and N is a positive integer greater than or equal to 2; Concatenate the N current features to determine a group of candidate combined features corresponding to the current feature grouping among the multiple groups of candidate combined features.

[0011] Optionally, the device is configured to obtain a plurality of feature groupings respectively matching the number of features of each feature grouping in the following manner: When the plurality of feature groupings include X feature groupings and the initial feature set includes Y features, respectively and sequentially determine the probability M of the i-th feature grouping to the j-th feature grouping according to the target probability matrix ij, where 0 < i ≤ X, 0 < j ≤ Y, X is a positive integer, and Y is a positive integer greater than or equal to 2; according to the probability M ij to determine whether to group the i-th feature into the j-th feature group.

[0012] Optionally, the apparatus is further configured to: obtain training sample data, where the training sample data represents the sample data used for training the initial recommendation model; perform feature extraction on the training sample data to obtain the initial feature set; use the initial feature set to train the initial recommendation model to obtain the first recommendation model; use the initial feature set and the candidate combined features to train the initial recommendation model to obtain the second recommendation model.

[0013] Optionally, the apparatus is further configured to: obtain test sample data, where the test sample data represents the sample data used for testing the first recommendation model and the second recommendation model; input the test sample data into the first recommendation model to determine the first evaluation model metric; input the test sample data and the candidate combined features into the second recommendation model to determine the second evaluation model metric.

[0014] Optionally, the apparatus is further configured to: in the case where the second evaluation model metric is greater than the first evaluation model metric, determine the multiple groups of candidate combined features as the target combined features; or in the case where the second evaluation model metric is greater than the first evaluation model metric and the difference between the second evaluation model metric and the first evaluation model metric is greater than a preset threshold, determine the multiple groups of candidate combined features as the target combined features.

[0015] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium storing a computer program, where the computer program is configured to execute the above method for determining combined features when running.

[0016] According to another aspect of the embodiments of the present application, there is provided a computer program product or a computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the method for determining combined features as above.

[0017] According to another aspect of the embodiments of the present application, there is also provided an electronic device including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the above method for determining combined features through the computer program.

[0018] In the embodiment of the present application, the initial recommendation model is trained based on the initial feature set to obtain a first recommendation model corresponding to the first evaluation model metric; the number of features in each feature group is determined according to a preset order, and the initial feature set is grouped to obtain a plurality of feature groups respectively matching the number of features in each feature group; in the case where the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, the initial recommendation model is trained based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to the second evaluation model metric; in the case where the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model, the candidate combined features are determined as the target combined features. By automatically grouping the initial features of the initial feature combination in advance to obtain corresponding multiple feature groups, and then performing intra-group feature combination within each feature group, the candidate combined features are obtained. Finally, the model effect of the target recommendation model obtained by training the initial recommendation model according to the candidate combined features is compared with the model effect, so as to determine the target combined features, achieving the purpose of reasonably selecting the combined features. By using the target combined features to train the recommendation model, the technical effect of improving the model effect of the recommendation model is realized, and further solves the technical problem that the determination efficiency of the combined features is low due to the complex determination method of the combined features.

[0019] In addition, by comparing the model effect of training the initial recommendation model by combining the candidate combined features with the initial feature set with the model effect of training the initial recommendation model by using only the initial feature set, that is, comparing the model effect of the second recommendation model with the model effect of the first recommendation model, the production time of the combined features is greatly saved, and the production efficiency of the combined features is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0021] Figure 1 is a schematic diagram of the application environment of an optional method for determining combined features according to an embodiment of the present application;

[0022] Figure 2 is a schematic flowchart of an optional method for determining combined features according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of an optional method for determining combined features according to an embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application;

[0025] Figure 5 It is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application;

[0026] Figure 6 It is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application;

[0027] Figure 7 It is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application;

[0028] Figure 8 It is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application;

[0029] Figure 9 It is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application;

[0030] Figure 10 It is a schematic diagram of the structure of an optional device for determining combined features according to an embodiment of the present application;

[0031] Figure 11 It is a schematic diagram of the structure of an optional product for determining combined features according to an embodiment of the present application;

[0032] Figure 12 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. Detailed implementation manners

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0035] First, some nouns or terms that appear during the description of the embodiments of this application are applicable to the following explanations:

[0036] Combined feature: It refers to a new feature obtained by combining two or more features. Since this type of feature combines multiple single features, it usually has stronger expressive power.

[0037] Hyperparameter: A parameter that needs to be set manually before model training, rather than a parameter automatically learned by the model. The choice of hyperparameters directly affects the performance and training process of the model. Common hyperparameters include learning rate, regularization parameter, number of iterations, batch size, etc. These hyperparameters need to be adjusted according to the characteristics of the dataset and the model to achieve the best model performance. Different methods can be used to select hyperparameters, such as grid search, random search, Bayesian optimization, etc.

[0038] Differentiable Neural Architecture Search: It is a method for searching neural network architectures. Traditional neural network architecture search methods usually require discrete operations, such as convolution kernel size, connection method, etc., which makes the search process very time-consuming. While DARTS transforms the neural network architecture search problem into an optimization problem by introducing differentiability and can be solved using optimization algorithms such as gradient descent. The core idea of DARTS is to learn a supergraph, where nodes represent operations in the network and edges represent the dependencies between operations. The input of each node is the output of all its predecessor nodes and is represented by a weight to indicate the weight of the input. By searching for the optimal combination of operations in the supergraph, the optimal network architecture can be obtained.

[0039] The following is an illustration of this application in combination with embodiments:

[0040] According to one aspect of the embodiments of the present application, a method for determining combined features is provided. Optionally, in this embodiment, the method for determining combined features may be applied to a hardware environment composed of a server 101 and a terminal device 103 as shown in Figure 1 the figure. As shown in Figure 1 the figure, the server 101 is connected to the terminal device 103 through a network and can be used to provide services for the terminal device or an application installed on the terminal device. The application can be a video application, an instant messaging application, a browser application, an educational application, a game application, etc. A database 105 can be set up on the server or independently of the server to provide data storage services for the server 101. For example, a game data storage server. The above network may include, but is not limited to: a wired network, a wireless network. Among them, the wired network includes: a local area network, a metropolitan area network, and a wide area network. The wireless network includes: Bluetooth, WIFI, and other networks that implement wireless communication. The terminal device 103 can be a terminal configured with an application and may include, but is not limited to, at least one of the following: a mobile phone (such as an Android mobile phone, an iOS mobile phone, etc.), a laptop computer, a tablet computer, a handheld computer, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a mixed reality (MR) terminal, and other computer devices. The above server can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0041] Combined with Figure 1 the figure, the method for determining combined features can be executed by an electronic device. The electronic device can be a terminal device or a server. The method for determining combined features can be implemented separately by the terminal device or the server, or jointly implemented by the terminal device and the server.

[0042] The above is only an example, and this embodiment does not make specific limitations.

[0043] Optionally, as an alternative implementation, as shown in Figure 2 the figure, the method for determining combined features includes:

[0044] S202, training an initial recommendation model based on an initial feature set to obtain a first recommendation model corresponding to a first evaluation model metric;

[0045] Optionally, in the embodiments of the present application, the above initial feature set may include multiple features. Among them, features are used to describe the unique attributes or characteristics of things, and can be obtained through observation, measurement, calculation, etc. In different fields, the definition and application of combined features are also different. For example, in the field of computer science, features can be understood as attributes or metrics that can describe an object or event. For example, in the field of image recognition, pixel values, textures, edges, etc. of an image can be used as the initial features of the image. In the field of media information recommendation, age, gender, interested fields, etc. corresponding to an account can be used as the initial features of the account.

[0046] It should be noted that the present application does not limit the number of features in the above initial feature set. Different features in the initial feature set can be selected and combined to generate the above combined features. The combined features can represent different attributes or characteristics of a thing. For example, combining the feature [yellow] of thing A with the feature [plastic] to obtain the combined feature [yellow_plastic] of thing A.

[0047] Exemplarily, the above initial feature set includes multiple features. For example, the above initial feature set is [red, round, glass]. At this time, the above combined features may include [red_round], [red_glass], [round_glass], [red_round_glass].

[0048] It should be noted that the above initial recommendation model and the above first recommendation model may include, but are not limited to, collaborative filtering models, content-based recommendation models, hybrid models, matrix factorization models, graph-based recommendation models, deep learning models, etc.

[0049] In an exemplary embodiment, Figure 3 is a schematic diagram of an optional method for determining combined features according to the embodiments of the present application. As Figure 3 shown, the initial recommendation model 302 is trained using the initial features in the initial feature set to obtain the first recommendation model 304. Among them, the terminals used to train the initial recommendation model 302 may include, but are not limited to, computers, laptops, servers, GPU clusters, distributed computing clusters, etc. The features used to train the initial recommendation model 302 may come from the initial feature set 306, and the initial feature set 306 has features such as [age], [gender], and [work field].

[0050] S204, determine the number of features in each feature group according to the preset order, and group the initial feature set to obtain multiple feature groups respectively matching the number of features in each feature group;

[0051] It should be noted that the features in the above initial feature set are grouped. Assuming that the initial feature set contains S initial features, these S initial features are divided into N groups according to a preset order, that is, there are N feature groups, and there are a preset number of features in each feature group. In other words, the number of features in each feature group is the same, which is the value of the preset order. The same initial feature is allowed to be included in different feature groups. That is, both S and N are determined in advance.

[0052] In an exemplary embodiment, the preset order is 2. The initial feature set has the initial features [female], [angry], and [member]. The probability that the initial feature [female] is added to feature group A is 60%, the probability that the initial feature [angry] is added to feature group A is 50%, and the probability that the initial feature [member] is added to feature group A is 65%. The preset condition is that the number of features in feature group A is equal to the preset order. The probabilities of the initial features [female], [angry], and [member] being assigned to feature group A from largest to smallest are: the initial feature [member], the initial feature [female], and the initial feature [angry]. That is, the initial features [member] and [female] will be determined as the initial features in feature group A, and feature group A is [female, member].

[0053] It should be noted that the same initial feature can exist in different feature groups. For example, the initial feature is [ellipse], and there are two feature groups, [ellipse, red] and [ellipse, container] respectively.

[0054] S206, when the features of each feature group are respectively combined within the group and the candidate combined features corresponding to each feature group are obtained, the initial recommendation model is trained based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to the second evaluation model index;

[0055] Optionally, in the embodiments of the present application, the above candidate combined features can be understood as thing features including attributes or characteristics of different dimensions. For example, the candidate combined feature [black_cloth] of thing A indicates that the color of thing A is black and the material is cloth. The above second recommendation model can include, but is not limited to, a collaborative filtering model, a content-based recommendation model, a hybrid model, a matrix factorization model, a graph-based recommendation model, a deep learning model, etc.

[0056] Exemplarily, when there are X feature groups, that is, there are X candidate combined features, and the X feature groups correspond to the X candidate combined features in sequence. For example, the first feature group [feature A, feature B] corresponds to the 1st candidate combined feature [feature A_feature B], and the second feature group [feature C, feature D] corresponds to the 2nd candidate combined feature [feature C_feature D].

[0057] In an exemplary embodiment, Figure 4 is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application. As Figure 4 shown, by training the initial recommendation model 402, a second recommendation model 404 is generated. The features used to train the initial recommendation model 402 can come from the initial feature set 406, candidate combined features 408, and candidate combined features 410. There are three initial features in the initial feature set 406, namely [24 hours], [iron sheet], [1024 bytes], and [black]. The initial feature set is grouped to obtain feature group A [24 hours, iron sheet] and feature group B [1024 bytes, black]. That is, feature group A corresponds to candidate combined feature 408, and candidate combined feature 408 includes a feature [24 hours_iron sheet]. Feature group B corresponds to candidate combined feature 410, and candidate combined feature 410 includes a feature [1024 bytes_black]. That is to say, six feature vectors will be generated according to the six features, and these six feature vectors will be input into the initial recommendation model 402 for training to obtain the second recommendation model 404. These six features are [24 hours], [iron sheet], [1024 bytes], [black], [24 hours_iron sheet], and [1024 bytes_black].

[0058] In the case of not using combined features, four feature vectors can be generated according to four initial features, and these four feature vectors are input into the initial recommendation model 402 for training to obtain the above-mentioned first recommendation model. These four initial features are [24 hours], [iron sheet], [1024 bytes], and [black]. That is to say, the first recommendation model and the second recommendation model are trained by the same initial recommendation model. The training process of the first recommendation model does not require input of candidate combined features, and the training process of the second recommendation model requires input of candidate combined features.

[0059] S208. When the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model, the candidate combined feature is determined as the target combined feature.

[0060] Optionally, in the embodiment of the present application, the above-mentioned first evaluation model metric is used to represent the model effect of the above-mentioned first recommendation model, and the above-mentioned second evaluation model metric is used to represent the model effect of the above-mentioned first recommendation model. Among them, the model effect can be understood as the accuracy of the recommendation result finally output by the recommendation model, and can be determined by indicators such as GAUC (Generalized Area Under the ROC Curve), Recall@10, accuracy, recall rate, average accuracy, coverage rate, etc.

[0061] First, Recall@10 can be understood as the number of items that the object is truly interested in among the top ten items recommended to the object by the recommendation model. Recall@10 can be calculated through the following steps:

[0062] S1. According to the recommendation results of the model, obtain the top 10 items to be recommended to the object as set B;

[0063] S2. Obtain multiple items that the object is truly interested in as set A;

[0064] S3. Count the number of items in the intersection of set A and set B;

[0065] S4. Determine the ratio of the number of items obtained in S3 to the total number of items in set A as the value of Recall@10.

[0066] Among them, the larger the value of Recall@10, the more accurately the recommendation model can cover the items that the object is truly interested in when recommending items to the object.

[0067] Secondly, GAUC (Generalized Area Under the ROC Curve), that is, the generalized area under the ROC curve, is an index for evaluating the performance of the recommendation model. GAUC takes into account the preference distribution of different objects in the recommendation results and weights the recommendation results of each object. The higher the GAUC of the recommendation model, the better its recommendation effect among different objects. ROC (Receiver Operating Characteristic) is a commonly used tool for evaluating the performance of classification models. The ROC curve is usually a convex-upward curve. The curve closer to the upper left corner indicates better performance of the recommendation model. The ROC curve is drawn based on the prediction results of the recommendation model under different thresholds. The threshold represents the critical value for converting probability into classification results. Different thresholds will result in different recall rates and false positive rates, thus affecting the shape of the ROC curve. The area under the ROC curve (Area Under Curve, AUC) is one of the important indexes for evaluating the performance of the recommendation model. The closer the AUC value is to 1, the better the performance of the recommendation model; the closer it is to 0.5, the worse the recommendation ability of the recommendation model. When comparing the model effects, the recommendation model with the ROC curve closer to the upper left corner and the larger AUC value is usually considered a better model.

[0068] Exemplarily, in an exemplary embodiment, Figure 5 is a schematic diagram of another optional method for determining combined features according to the embodiments of the present application, as Figure 5As shown, both the above-mentioned first recommendation model and the second recommendation model are trained by a deep learning network, and this deep learning model is the above-mentioned initial recommendation model. The first evaluation model metric corresponding to the first recommendation model is the first GAUC, and the second evaluation model metric corresponding to the second recommendation model is the second GAUC. The first GAUC and the second GAUC can be calculated through the following steps:

[0069] S1. Respectively use the first recommendation model 502 and the second recommendation model 504 to recommend the combined feature [object_advertisement] in the test set, and obtain the corresponding first recommendation result 506 and the second recommendation result 508, indicating whether the object clicks on the advertisement;

[0070] S2. According to the first recommendation result 506, obtain the first recommendation list 510 of each object corresponding to the first recommendation model 502;

[0071] S3. According to the first recommendation result 508, obtain the second recommendation list 512 of each object corresponding to the second recommendation model 504;

[0072] S3. Determine whether the object clicks on the advertisement according to the object real data 514, and calculate the first GAUC and the second GAUC. The specific calculation method can refer to the following steps:

[0073] S3-1. According to the advertisements in the first recommendation list 510 and the object real click data 514, calculate the first AUC. Among them, the first AUC can be obtained by calculating the area under the ROC curve;

[0074] S3-2. According to the advertisements in the second recommendation list 512 and the object real click data 514, calculate the second AUC. Among them, the second AUC can be obtained by calculating the area under the ROC curve;

[0075] S3-3. Perform weighted average on the first AUCs of all objects in the first recommendation list to obtain the first GAUC;

[0076] S3-3. Perform weighted average on the second AUCs of all objects in the second recommendation list to obtain the second GAUC.

[0077] Among them, when the value of the first GAUC is less than the value of the second GAUC, it indicates that the model effect of the second recommendation model is better than that of the first recommendation model.

[0078] Furthermore, the above-mentioned preset index condition can be understood as that the model effect of the above-mentioned first recommendation model is not as good as that of the second recommendation model, that is, the recommendation result of the second recommendation model is more accurate than that of the first recommendation model.

[0079] In an exemplary embodiment, Figure 6 is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application. The method for determining combined features proposed by the present application can be applied to scenarios such as advertising recommendation, as Figure 6 shown:

[0080] S602, Extract features from the object account data and advertisement data in the media placement platforms such as independent applications, websites, and web pages to obtain an initial feature set. Among them, the object data may include but is not limited to the object account gender, the object account registration time, the object account registration age, etc., and the advertisement data may include but is not limited to the advertisement type, the advertisement release time, etc.;

[0081] S604, Train an initial advertisement recommendation model based on the initial feature set to obtain a first advertisement recommendation model;

[0082] S606, Automatically group the initial features in the initial feature set according to a preset order to obtain feature group A, feature group B, and feature group C;

[0083] S608, Perform intra-group feature combination on feature group A respectively to obtain candidate combined feature A, perform intra-group feature combination on feature group B to obtain candidate combined feature B, and perform intra-group feature combination on feature group C to obtain candidate combined feature C;

[0084] S610, Train the initial advertisement recommendation model based on the initial feature set and candidate combined feature A, candidate combined feature B, and candidate combined feature C to obtain a second advertisement recommendation model;

[0085] S612, Input test sample data, which includes object sample data and advertisement sample data. Among them, the object sample data may include but is not limited to the object account gender, the object account registration time, the object account registration age, etc., and the advertisement sample data may include but is not limited to the advertisement type, the advertisement release time, etc. Determine whether the value of Recall@10 corresponding to the second advertisement recommendation model is greater than the value of Recall@10 corresponding to the first advertisement recommendation model. In the case where the value of Recall@10 corresponding to the second advertisement recommendation model is less than or equal to the value of Recall@10 corresponding to the first advertisement recommendation model, a new initial feature set can be reselected, or the preset order can be reset;

[0086] S614, In the case where the value of Recall@10 corresponding to the second advertisement recommendation model is greater than the value of Recall@10 corresponding to the first advertisement recommendation model, determine candidate combined feature A, candidate combined feature B, and candidate combined feature C as the target combined features.

[0087] Specifically, when a large number of media information such as advertisements are placed on independent applications, websites, web pages and other platforms, the method for determining the combined features proposed in this application can be used to deliver advertisement content of interest to different object accounts. It can be understood that the target combined features obtained in this application are applied to the advertisement recommendation model to improve the recommendation accuracy of the advertisement recommendation model. Through this advertisement recommendation model, a recommendation result corresponding to the object account is generated. Further, according to the recommendation result, advertisement content of interest is recommended to the object account. Different object accounts can enter the corresponding advertisement publisher interface through the advertisement interface, achieving the purpose of improving the click-through rate and conversion rate of the advertisement. Moreover, the advertisement placement strategy can be dynamically adjusted in real time according to the recommendation effect of the advertisement content corresponding to different object accounts. Thus, the technical effect of improving the intelligence of advertisement placement is achieved.

[0088] In another exemplary embodiment, the method for determining the combined features proposed in this application can be applied to the application scenario of video recommendation, such as Figure 6 shown as follows:

[0089] S1. Extract features from the object account data and video data in independent applications, websites, web pages and other video sharing platforms to obtain an initial feature set. Among them, the object data may include but is not limited to the gender of the object account, the registration time of the object account, the registration age of the object account, etc., and the video data may include but is not limited to the video type, the video release time, etc.;

[0090] S2. Train an initial video recommendation model according to the initial feature set to obtain a first video recommendation model;

[0091] S3. Automatically group the initial features in the initial feature set according to a preset order to obtain feature group A, feature group B, and feature group C;

[0092] S4. Perform intra-group feature combination on feature group A respectively to obtain candidate combined feature A, perform intra-group feature combination on feature group B to obtain candidate combined feature B, and perform intra-group feature combination on feature group C to obtain candidate combined feature C;

[0093] S5. Train the initial video recommendation model according to the initial feature set and candidate combined feature A, candidate combined feature B, and candidate combined feature C to obtain a second video recommendation model;

[0094] S6. Input the test sample data, which includes object sample data and video sample data. Among them, the object sample data may include, but is not limited to, the gender of the object account, the registration time of the object account, the registration age of the object account, etc., and the video sample data may include, but is not limited to, the video type, the video release time, etc. Determine whether the value of GAUC corresponding to the second video recommendation model is greater than the value of GAUC corresponding to the first video recommendation model. In the case where the value of GAUC corresponding to the second video recommendation model is less than or equal to the value of GAUC corresponding to the first video recommendation model, a new initial feature set can be reselected, or the preset order can be reset.

[0095] S7. When the value of GAUC corresponding to the second video recommendation model is greater than the value of GAUC corresponding to the first video recommendation model, determine the candidate combined feature A, the candidate combined feature B, and the candidate combined feature C as the target combined features.

[0096] It can be understood that different object accounts often generate different object account data when browsing the same video. When the object account is interested in the video content, the browsing time is usually longer. In this way, the object account with a longer browsing time is more likely to follow the video creator or post comments, etc., to increase the popularity of the video creator. Thus, it brings more benefits to the video creator. For the object account, the video sharing platform will recommend more videos with similar content to the object account to increase the staying time of the object account on the platform, thereby improving the activity level of the object account on the platform.

[0097] Furthermore, the method for determining the combined features proposed in this application can be used to deliver videos of interest to different object accounts. That is, the target combined features obtained in this application are applied to the video recommendation model to improve the recommendation accuracy of the video recommendation model. Through this video recommendation model, generate recommendation results corresponding to the object account. Further, recommend video content of interest to the object account according to the recommendation results. Different object accounts can all watch videos of their own interest. In other words, use this video recommendation model to recommend more video content that meets the preferences of the object account to improve the user experience satisfaction. By adjusting the video push strategy in real time according to the video content recommendation effect corresponding to different object accounts, personalized video push is realized, thereby improving the video push effect.

[0098] In another exemplary embodiment, the method for determining the combined features proposed in this application can be applied to the application scenario of game teaming. The games here may include, but are not limited to, role-playing games, shooting games, board games, simulation business games, etc., as Figure 6 shown:

[0099] S1. Extract features from the player data and game application data in the game to obtain an initial feature set. Among them, the player data may include, but is not limited to, player gender, player registration time, player registration age, etc., and the game application data may include, but is not limited to, game type, game version, etc.;

[0100] S2. Train an initial team formation recommendation model based on the initial feature set to obtain a first team formation recommendation model;

[0101] S3. Automatically group the initial features in the initial feature set according to a preset order to obtain feature group A, feature group B, and feature group C;

[0102] S4. Perform intra-group feature combination on feature group A respectively to obtain candidate combined feature A, perform intra-group feature combination on feature group B to obtain candidate combined feature B, and perform intra-group feature combination on feature group C to obtain candidate combined feature C;

[0103] S5. Train the initial team formation recommendation model based on the initial feature set and candidate combined feature A, candidate combined feature B, and candidate combined feature C to obtain a second team formation recommendation model;

[0104] S6. Input test sample data, which includes player sample data and game application sample data. Among them, the player sample data may include, but is not limited to, player gender, player registration time, player registration age, etc., and the game application sample data may include, but is not limited to, game type, game version, etc. Determine whether the value of GAUC corresponding to the second team formation recommendation model is greater than the value of GAUC corresponding to the first team formation recommendation model. In the case where the value of GAUC corresponding to the second team formation recommendation model is less than or equal to the value of GAUC corresponding to the first team formation recommendation model, a new initial feature set can be reselected, or the preset order can be reset;

[0105] S7. When the value of GAUC corresponding to the second team formation recommendation model is greater than the value of GAUC corresponding to the first team formation recommendation model, determine candidate combined feature A, candidate combined feature B, and candidate combined feature C as target combined features.

[0106] Among them, the target combined features obtained in this application are applied to the team formation recommendation model to improve the recommendation accuracy of the team formation recommendation model. Through this team formation recommendation model, recommendation results corresponding to players are generated. The recommendation results here may include, but are not limited to, one or more relevant players with the highest cooperation degree with the player and the highest success rate of completing game tasks in this game. Furthermore, the player can choose to form a team with the relevant players determined by the recommendation results to complete game tasks. Different players can generate different recommendation results through the team formation recommendation model.

[0107] Furthermore, players can select one or more relevant players with the highest cooperation degree with themselves and the highest success rate in completing game tasks to form a team, so as to help players find suitable teammates, improve players' gaming experience and gaming performance. By recommending teammates with similar gaming interests and ability levels to players, the cooperation and competitiveness of the game team are ensured. Thus, the stickiness between the game and the players is increased, achieving the purpose of improving players' gaming experience.

[0108] Through the embodiments of the present application, the initial recommendation model is trained based on the initial feature set to obtain a first recommendation model corresponding to the first evaluation model index; the number of features in each feature group is determined according to the preset order, and the initial feature set is grouped to obtain multiple feature groups respectively matching the number of features in each feature group; when the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, the initial recommendation model is trained based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to the second evaluation model index; in the case that the first evaluation model index and the second evaluation model index indicate that the model effect of the second recommendation model is better than that of the first recommendation model, the candidate combined features are determined as the target combined features. By automatically grouping the initial features of the initial feature combination in advance, after obtaining the corresponding feature groups, intra-group feature combination is performed within each feature group, thereby obtaining multiple groups of candidate combined features, and finally the target combined features are determined according to the multiple groups of candidate combined features. The target combined features can improve the training effect of the recommendation model. The initial recommendation model is trained according to the target combined features and the initial feature combination to obtain the second recommendation model, achieving the purpose of reasonably selecting the target combined features. Using the target combined features to train the recommendation model realizes the technical effect of improving the model effect of the recommendation model, and further solves the technical problem that the determination efficiency of the combined features is low due to the complex determination method of the combined features.

[0109] In addition, by comparing the model effect of training the initial recommendation model by combining multiple groups of candidate combined features with the initial feature set with the model effect of training the initial recommendation model only using the initial feature set, that is, comparing the model effect of the second recommendation model with the model effect of the first recommendation model, it is determined that the model effect of the second recommendation model is better than the model effect of the first recommendation model, and then the target combined features are obtained for use in the training of the recommendation model, greatly saving the production time of the combined features and improving the production efficiency of the combined features.

[0110] As an alternative solution, determining the number of features in each feature group according to a preset order, grouping the above initial feature set, and obtaining a plurality of feature groups respectively matching the number of features in each feature group above includes: inputting the above initial feature set into a pre-trained target feature grouping model to determine a target probability matrix, where the number of rows of the above target probability matrix is the same as the number of features in the above initial feature set, the number of columns of the above target probability matrix is the same as the number of feature groups of the above plurality of feature groups, and the elements in the above target probability matrix are used to represent the probability from the target initial feature group to the target feature group; grouping the above initial feature set according to the above target probability matrix to obtain the above plurality of feature groups.

[0111] Optionally, in the embodiments of the present application, the above target feature grouping model may include, but is not limited to, a clustering model, a classification model, an association rule model, a neural network model, etc.

[0112] Exemplarily, Figure 7 is a schematic diagram of another alternative method for determining combined features according to the embodiments of the present application. As shown in Figure 7 In the target feature grouping model 702, after inputting the initial feature set 704, the initial feature set 704 will pass through the automatic feature grouping layer 706. In the present application, the initial feature set 704 only needs to be processed by the automatic feature grouping layer 706 to complete the operation of automatically grouping the initial features in the initial feature set 704.

[0113] Furthermore, after the above initial feature set is processed by the automatic feature grouping layer of the target feature grouping model, the above target probability matrix is obtained. That is to say, the elements in the target probability matrix represent the probability values that the features in the initial feature set are assigned to different feature groups. Assume that the above target probability matrix has P rows, and the number of features in the initial feature set is also P. And assume that the above target probability matrix has Q columns, and the number of feature groups of the above plurality of feature groups is also Q. The number of the above plurality of feature groups can be preset, and the present application does not limit the specific value of the number of the above plurality of feature groups.

[0114] Exemplarily, obtaining the above target probability matrix can be achieved through the following steps:

[0115] S1. Extract features from the object data to determine the initial feature set;

[0116] S2. Generate a plurality of feature vectors according to the features in the initial feature set, input the plurality of feature vectors into the target feature grouping model for training, and randomly generate an initial probability matrix;

[0117] S3. By training the target feature grouping model, the initial probability matrix is continuously updated, and finally the target probability matrix is generated.

[0118] Exemplarily, Figure 8 is a schematic diagram of another optional method for determining combined features according to an embodiment of the present application. As Figure 8 shown, the initial feature set 802 includes the target initial feature [A] and the target initial feature [B]. After the initial feature set 802 is processed by the automatic feature grouping layer of the target feature grouping model, the target probability matrix 804 is obtained. The target probability matrix 804 has two rows, corresponding to the two features [A] and [B] in the initial feature set 802. The target initial feature 806 represents the target initial feature [A] in the initial feature set 802, corresponding to the first row of the target probability matrix 804. And the target probability matrix 804 has two columns, corresponding to two feature groups. Among them, the target probability 810 that the target initial feature 806 is assigned to the target feature group 808 represented by the first column is 0.1, and the target probability assigned to the feature group 812 represented by the second column is 0.9.

[0119] Through the embodiment of the present application, the initial feature set is input into the pre-trained target feature grouping model. After being processed by the automatic feature grouping layer in the target feature grouping model, the target probability matrix is obtained. The target probability matrix can represent the probabilities that the initial features in the initial feature set are assigned to different feature groups. Then, multiple feature groups are determined according to the target probability matrix, achieving the technical effect of improving the efficiency of feature grouping and further realizing the reasonable grouping of the initial feature set.

[0120] As an optional solution, before inputting the initial feature set into the pre-trained target feature grouping model, the method further includes: obtaining a sample feature set and sample annotation results corresponding to each sample feature in the sample feature set, where the sample feature set has the same feature category as the initial feature set, and the sample annotation results are used to represent the true recommendation results corresponding to the sample features; inputting the sample feature set and the sample annotation results into the initial feature grouping model to be trained, obtaining a predicted recommendation result, and using the true recommendation result and the predicted recommendation result to train the initial feature grouping model to obtain the target feature grouping model.

[0121] Optionally, in the embodiments of the present application, the above sample feature set may include multiple above sample features. The above sample features may include, but are not limited to, the age feature, gender feature, and field of interest feature of a certain account. The present application does not limit the number of sample features in the sample feature set. The above sample annotation results may include, but are not limited to, 30 years old, male, engineer, etc. The sample annotation result represents the true attributes or characteristics of the sample. That is to say, through the sample annotation result, the true recommendation result corresponding to the sample feature can be obtained. For example, sample object A clicks on the advertisement link of advertisement F, and sample object B does not click on the advertisement link of advertisement F. That is, the sample annotation result corresponding to sample object A indicates that it has been clicked, while the sample annotation result corresponding to sample object B indicates that it has not been clicked.

[0122] It should be noted that the feature categories involved in the above sample feature set correspond to the feature categories of the above initial feature set. For example, the initial feature set includes the age feature and gender feature of the object, that is, there are sample features with the feature category of the object's age and sample features of the object's gender in the sample feature set. That is to say, the object attribute categories included in the sample object data used to extract the above sample features are the same as the object attribute categories included in the object data used to extract the initial features. The above feature categories may include the object's age, the object's gender, the field of interest of the object, etc.

[0123] In an exemplary embodiment, when training the above initial feature grouping model, the above sample feature set and the above sample annotation results are input to obtain a predicted recommendation result. The predicted recommendation result can be understood as the sample annotation result owned by any feature in the sample feature set predicted by the initial feature grouping model. For example, according to the predicted recommendation result of the initial feature grouping model, it is obtained that both sample object A and sample object B will click on the advertisement link of advertisement F, while the true recommendation result indicates that sample object A clicks on the advertisement link of advertisement F, and sample object B does not click on the advertisement link of advertisement F. At this time, after training the initial feature grouping model using the true recommendation result and the predicted recommendation result, a target feature grouping model will be obtained. The target feature grouping model can also be continuously updated and optimized using the true recommendation result and the predicted recommendation result, so that the target feature grouping model can be applicable to a wider range of business requirement scenarios.

[0124] Through the embodiments of the present application, after inputting the sample feature set and the sample annotation results corresponding to each sample feature in the sample feature set into the initial feature grouping model to obtain a predicted recommendation result, the initial feature grouping model is then trained using the true recommendation result and the predicted recommendation result to obtain a target feature grouping model. This target feature grouping model can continuously learn and optimize as the object data and market environment change, thereby obtaining a target feature grouping model with a more accurate grouping result and stronger adaptability, achieving the purpose of reasonably grouping the initial features in the initial feature set.

[0125] As an alternative solution, the above-mentioned steps of inputting the above-mentioned sample feature set and the above-mentioned sample annotation results into the initial feature grouping model to be trained to obtain a predicted recommendation result, and using the above-mentioned true recommendation result and the above-mentioned predicted recommendation result to train the above-mentioned initial feature grouping model to obtain the above-mentioned target feature grouping model include: inputting the above-mentioned sample feature set and the above-mentioned sample annotation results into the above-mentioned initial feature grouping model to obtain an initial probability matrix, where the number of rows and columns of the above-mentioned initial probability matrix is the same as that of the target probability matrix, and the values of each element in the above-mentioned initial probability matrix are randomly set; automatically grouping the sample features in the above-mentioned sample feature set according to the above-mentioned initial probability matrix to obtain multiple sample feature groupings, where the number of features in the above-mentioned sample feature set grouped into each feature grouping among the above-mentioned multiple feature groupings is the above-mentioned preset order; performing an encoding operation on each sample feature in the above-mentioned sample feature set respectively to determine a feature vector set, where the feature vectors in the above-mentioned feature vector set correspond one-to-one with the sample features in the above-mentioned sample feature set; determining multiple feature grouping vectors according to the above-mentioned feature vector set and the above-mentioned multiple sample feature groupings, where the above-mentioned multiple feature grouping vectors correspond one-to-one with the above-mentioned multiple sample feature groupings; inputting the above-mentioned multiple feature grouping vectors into a multi-layer perceptron to determine a predicted recommendation result, and using the above-mentioned true recommendation result and the above-mentioned predicted recommendation result to train the above-mentioned initial feature grouping model to obtain the above-mentioned target feature grouping model, where when the above-mentioned initial feature grouping model is trained into the above-mentioned target feature grouping model, the above-mentioned initial probability matrix is simultaneously trained into the above-mentioned target probability matrix.

[0126] Optionally, in the embodiments of the present application, the above-mentioned initial feature grouping model can randomly generate the probability that each sample feature is assigned to each feature grouping according to the sample feature set and the sample annotation result to obtain the above-mentioned initial probability matrix. Each element in the initial probability matrix corresponds to the probability that each sample feature is assigned to each feature grouping. For example, the element in the first row and first column of the initial probability matrix is 0.23, that is, the probability that the first sample feature is assigned to the first feature grouping is 0.23, or the element in the first row and second column of the initial probability matrix is 0.4, that is, the probability that the first sample feature is assigned to the second feature grouping is 0.4.

[0127] It should be noted that after updating the above initial probability matrix, the above target probability matrix will be obtained. That is to say, each row in the above initial probability matrix corresponds to a sample feature, and each column corresponds to a feature group. Moreover, the number of such feature groups can be set in advance. Assuming the sample feature set is [30 years old, female, male, postgraduate], and the sample feature set is divided into 6 feature groups. That is, there will be 2 sample features in each sample feature group, namely sample feature group A [30 years old, female], sample feature group B [30 years old, male], sample feature group C [30 years old, postgraduate], sample feature group D [female, male], sample feature group A [female, postgraduate], sample feature group A [male, postgraduate]. And a sample feature can be in different sample feature groups. At this time, the initial probability matrix will include 4 rows and 6 columns, corresponding to 4 sample features and 6 sample feature groups.

[0128] Furthermore, the values of all elements in the initial probability matrix corresponding to the above initial feature grouping model are randomly set. During the process of training the above initial feature grouping model with the above real recommendation results and the above predicted recommendation results, the values of the elements in the initial probability matrix will be updated. After the training of the initial feature grouping model is completed, a target feature grouping model is obtained. At this time, the initial probability matrix is updated to the target probability matrix, and all elements in the initial probability matrix are updated to the above target elements.

[0129] In an exemplary embodiment, the number of sample features included in each of the above sample feature combinations is the same. The above preset order represents the number of sample features in any one sample feature group. That is to say, assuming the preset order is P, and P is a non-negative integer. After automatically grouping the sample features in the above sample feature set according to the above initial probability matrix, the above multiple sample feature groups are obtained, and each sample feature group includes P sample features.

[0130] Specifically, the above encoding operation can be understood as obtaining the digital representation of the above sample features, that is, converting the sample features into feature vectors with a fixed length. The above feature vector set corresponds to the above initial sample feature set. For example, the initial sample feature set includes 30 sample features, and there is one sample feature [glass]. Correspondingly, the feature vector set includes 30 feature vectors, and there is one feature vector used to represent the sample feature [glass].

[0131] Optionally, in the embodiments of the present application, the above-mentioned set of feature vectors is grouped to obtain multiple grouped feature vectors corresponding to the above-mentioned multiple sample feature groups. That is to say, the grouped feature vectors include the feature vectors extracted from the corresponding sample feature groups. For example, there are a total of 20 sample feature groups, including one sample feature group [sample feature A, sample feature B]. That is, there are 20 grouped feature vectors corresponding to it, including one grouped feature vector [feature vector of sample feature A, feature vector of sample feature B].

[0132] Exemplarily, the above-mentioned multi-layer perceptron can be understood as a feed-forward neural network with multiple hidden layers, such as Figure 7 the MLP shown, which consists of a hierarchical structure composed of multiple neurons. In each layer, the neurons multiply the input signal by the corresponding weight and then process it through an activation function to generate an output signal. The grouped feature vectors are input into the multi-layer perceptron to obtain the predicted recommendation result. Finally, the initial grouped feature model is trained according to the above-mentioned true recommendation result and the predicted recommendation result to obtain the target grouped feature model.

[0133] Through the embodiments of the present application, the sample feature set and the corresponding true annotation results are input into the initial grouped feature model to obtain an initial probability matrix, and the sample feature set is grouped according to the initial probability matrix to obtain sample feature groups. Then, the sample features in each sample feature group are encoded, and after extracting the sample feature vectors, multiple groups of grouped feature vectors are generated. The multiple groups of grouped feature vectors are input into the multi-layer perceptron to generate the predicted recommendation result. In this way, the initial grouped feature model is trained according to the predicted recommendation result combined with the existing true recommendation result, so as to achieve the technical effect of improving the accuracy of feature grouping and predicted recommendation.

[0134] As an optional solution, the above-mentioned features of each feature group are respectively combined within the group to obtain the candidate combined features corresponding to each feature group, including: the features of each feature group are respectively combined within the group in the following way to obtain the candidate combined features corresponding to each feature group. Among them, the feature group for which the within-group feature combination is performed each time is regarded as the current feature group: N current features are selected from the above-mentioned current feature group, where the above-mentioned N current features are the N features with the highest probability of being grouped into the above-mentioned current feature group determined according to the above-mentioned target probability matrix, and N is a positive integer greater than or equal to 2; the above-mentioned N current features are concatenated to determine a group of candidate combined features corresponding to the above-mentioned current feature group in the above-mentioned candidate combined features.

[0135] Optionally, in the embodiments of the present application, the above feature combination may include a splicing operation on the features in the feature grouping. For example, if the feature grouping A is [43 years old, female], the corresponding feature combination may be represented by [43 years old_female] or [female_43 years old]. This feature combination represents different categories of features of the object, that is, the object is a female and the age of object A is 43 years old.

[0136] It should be noted that feature combination needs to be performed on each of the above multiple feature groupings. Among them, the feature grouping currently undergoing feature combination is the above current feature grouping. The above target probability matrix corresponds to multiple groups of feature groupings, and each group of feature groupings includes M of the above initial features, where M is a non-negative integer. N current features are determined from the current feature grouping, and the value of N can be equal to or less than the value of M. For example, the target probability matrix indicates that there are 10 groups of feature groupings, and each group of feature groupings includes 5 initial features. At this time, the current feature grouping is the second group of feature groupings, and the number of current features determined from the second group of feature groupings is less than or equal to 5 initial features.

[0137] Specifically, the feature groupings determined from each group of the above current feature groupings are spliced to obtain the above candidate combined features, and the candidate combined features correspond to the current feature grouping. For example, there are a total of 3 groups of feature groupings, and there are 3 candidate combined features corresponding to the 3 groups of feature groupings. The second feature grouping [glass, large caliber, red] is determined as the current feature grouping. The probability that the feature [glass] is assigned to the second feature grouping is 0.79, the probability that the feature [large caliber] is assigned to the second feature grouping is 0.23, and the probability that the feature [red] is assigned to the second feature grouping is 0.89. At this time, the 2 features with the largest probability values are selected from these three features as the current features, that is, the features [glass] and [red] are determined as the current features, and the corresponding second candidate combined feature is [glass_red].

[0138] Through the embodiments of the present application, the current features corresponding to multiple groups of feature groupings are obtained according to the target probability matrix, and the current features in different feature groupings are spliced to obtain multiple candidate combined features, achieving the purpose of adaptively adjusting the initial features, thereby obtaining more valuable candidate feature combinations, and applying the candidate feature combinations to the training of the initial recommendation model, so that the finally generated second recommendation model can meet the business requirements of personalized recommendation.

[0139] As an optional solution, the above obtaining multiple feature groupings respectively matching the number of features of each feature grouping includes: when the above multiple feature groupings include X feature groupings and the above initial feature set includes Y features, respectively and sequentially determining the probability M of the i-th feature grouping to the j-th feature grouping according to the above target probability matrixij , where \(0 \lt i \leq X\), \(0 \lt j \leq Y\), \(X\) is a positive integer, and \(Y\) is a positive integer greater than or equal to 2; according to the above probability \(M\) ij determine whether to group the \(i\)-th feature above into the \(j\)-th feature group.

[0140] Optionally, in the embodiment of the present application, the above target probability matrix includes elements of \(Y\) rows and \(X\) columns, indicating that there are \(Y\) features in the above initial feature set, and these \(Y\) features are evenly divided into \(X\) feature groups. Among them, the element in the \(i\)-th row and \(j\)-th column of the target probability matrix represents the probability that the \(i\)-th feature in the above initial feature set is assigned to the \(j\)-th feature group, that is, the above \(M\) ij , assuming that there are 10 features in the initial feature set, and these 10 features are evenly divided into 2 feature groups. The element in the 1st row and 2nd column of the target probability matrix represents the probability that the 1st feature in the above initial feature set is assigned to the 2nd feature group.

[0141] Furthermore, determine the above probability \(M\) according to the above target probability matrix ij , so as to judge the probability \(M\) ij whether the corresponding \(i\)-th feature meets the preset condition for dividing the feature into the \(j\)-th feature group. For example, the preset condition is to take the 2 features with the top-ranked probability values of the initial feature being assigned to the target feature group as the target initial features and assign them to the \(j\)-th feature group. Assume \(M\) ij takes the maximum value among the \(Y\) probabilities corresponding to the \(Y\) initial features in the above initial feature set being assigned to the \(j\)-th feature group. At this time, the \(i\)-th feature will be grouped into the \(j\)-th feature group.

[0142] As an optional solution, the above method further includes: obtaining training sample data, where the above training sample data represents the sample data used for training the above initial recommendation model; performing feature extraction on the above training sample data to obtain the above initial feature set; using the above initial feature set to train the above initial recommendation model to obtain the above first recommendation model; using the above initial feature set and the above candidate combined features to train the above initial recommendation model to obtain the above second recommendation model.

[0143] Optionally, in the embodiment of the present application, the above training sample data may include, but is not limited to, the attribute data of the object. It can be understood that after performing feature extraction on the above training sample data, the features in the above initial feature set are obtained.

[0144] It should be noted that multiple groups of initial feature vectors are generated according to multiple initial features in the above initial feature set, and the multiple groups of initial feature vectors are input into the initial recommendation model for training to obtain the above first recommendation model. Then, multiple groups of candidate combination feature vectors are generated according to the above candidate combination features, and the multiple groups of initial feature vectors and multiple groups of candidate combination feature vectors are jointly input into the initial recommendation model for training to obtain the above second recommendation model. That is to say, compared with the first recommendation model, the second recommendation model has a larger number of input feature vectors for training. The generation process of the first recommendation model only requires the initial features in the initial feature set and does not require multiple groups of candidate combination features.

[0145] Exemplarily, the above initial feature set is [green, 10 meters, soft], and the corresponding [green feature vector, 10-meter feature vector, soft feature vector] are input into the initial recommendation model for training to obtain the first recommendation model. At this time, the candidate combination features are [green_10 meters] and [green_soft]. That is, [green feature vector, 10-meter feature vector, soft feature vector], [green_10-meter feature vector], and [green_soft feature vector] are input into the initial recommendation model for training together to obtain the second recommendation model.

[0146] As an optional solution, the above method further includes: obtaining test sample data, where the above test sample data represents the sample data used to test the above first recommendation model and the above second recommendation model; inputting the above test sample data into the above first recommendation model to determine the above first evaluation model index; inputting the above test sample data and the above candidate combination features into the above second recommendation model to determine the above second evaluation model index.

[0147] In an exemplary embodiment, after inputting the above test sample data into the above first recommendation model, a recommendation result is obtained, and the first GAUC is calculated according to the recommendation result. The value of the first GAUC is the above first evaluation model index. After inputting the above test sample data into the above second recommendation model, a recommendation result is obtained, and the second GAUC is calculated according to the recommendation result. The value of the second GAUC is the above second evaluation model index.

[0148] In another exemplary embodiment, after inputting the above test sample data into the above first recommendation model, a recommendation result is obtained, and the first Recall@10 is calculated according to the recommendation result. The value of the first Recall@10 is the above first evaluation model index. After inputting the above test sample data into the above second recommendation model, a recommendation result is obtained, and the second Recall@10 is calculated according to the recommendation result. The value of the second Recall@10 is the above second evaluation model index.

[0149] As an alternative solution, the above method further includes: when the second evaluation model metric is greater than the first evaluation model metric, determining the candidate combined feature as the target combined feature; or when the second evaluation model metric is greater than the first evaluation model metric and the difference between the second evaluation model metric and the first evaluation model metric is greater than a preset threshold, determining the candidate combined feature as the target combined feature.

[0150] Optionally, in the embodiments of the present application, the second evaluation model metric is compared with the first evaluation model metric. For example, the value of recall@10 is used as the evaluation metric. The larger the evaluation metric, the better the recommendation effect of the push-adding model. Suppose the value of the second evaluation model metric is 5 and the value of the first evaluation model metric is 7. Since the second evaluation model metric is less than the first evaluation model metric, in this case, the candidate combined feature will not be determined as the target combined feature, that is, the recommendation effect of the second recommendation model is not as good as that of the first recommendation model.

[0151] Exemplarily, the value of GAUC is used as the evaluation metric. The larger the evaluation metric, the better the recommendation effect of the push-adding model. The present application does not specifically limit the value of the above preset threshold. Suppose the value of the second evaluation model metric is 10, the value of the first evaluation model metric is 4, and the value of the preset threshold is 2. At this time, the second evaluation model metric is greater than the first evaluation model metric, and moreover, the difference between the second evaluation model metric and the first evaluation model metric is 6, which is greater than the preset threshold of 4. Then the candidate combined feature will be determined as the target combined feature, that is, the recommendation effect of the second recommendation model is better than that of the first recommendation model.

[0152] Through the embodiments of the present application, by comparing the correlation between the first evaluation model metric and the second evaluation model metric, when the second evaluation model metric is greater than the first evaluation model metric, or when the second evaluation model metric is greater than the first evaluation model metric and the difference between the second evaluation model metric and the first evaluation model metric is greater than a preset threshold, multiple groups of candidate combined features will be determined as the target combined features, and the recommendation effect of the second recommendation model is better than that of the first recommendation model. By using a reasonable evaluation metric to evaluate the recommendation effect of the recommendation model, the technical effects of improving the recommendation accuracy and the universality of the recommendation model are achieved.

[0153] The following further explains the present application with specific examples:

[0154] Exemplarily, this application proposes a method for determining combined features based on automatic feature grouping. Specifically, this application proposes to obtain a smaller-scale set of candidate features by additionally training a feature grouping model, and perform feature combination under this set of candidate features. Compared with the existing methods for determining combined features, the method for determining combined features proposed in this application is more efficient.

[0155] It should be noted that this application proposes a learnable automatic feature grouping model. By this feature grouping model, feature groupings that may produce gains can be obtained, and feature combination can be performed within the feature groupings, thus improving the efficiency of determining combined features. Figure 9 It is a schematic diagram of another optional method for determining combined features according to an embodiment of this application. The process of the method for determining combined features based on automatic feature grouping proposed in this application is as Figure 9 shown.

[0156] In an exemplary embodiment, in application scenarios such as search, advertising, and recommendation, features are the most important part of the model, and the quality of the features determines the upper limit of the model. The combined features involved in this application are an important way to enhance the expression of features in the model and are used in models such as search advertising recommendation. In addition, combined features can also be used in models for each link, including recall, rough ranking, and fine ranking, and play an important role in the effect of the model. The method for producing combined features based on automatic feature grouping proposed in this application mainly includes two links: training an initial feature automatic grouping model to obtain a target feature grouping model, and performing in-group combination based on the grouping result obtained from the target feature grouping model and evaluating the effect of the combined features, so as to determine the combined features. The combined features can be obtained through the following steps:

[0157] S1. Train an initial feature automatic grouping model to obtain a target feature grouping model. The feature automatic grouping framework adopted in this application includes but is not limited to a feature automatic grouping model for modeling high-order feature interactions. By setting different feature groupings, the model can automatically model feature interactions at different levels. Each group is responsible for mining feature interaction information at different levels to improve the effect of the model. This application uses the intermediate grouping result produced by this feature automatic grouping model as the basis for feature grouping, and combines the features within the group to obtain new combined features.

[0158] It should be noted that the feature automatic grouping model focuses on the effect of feature interactions in the overall model, while this application focuses on the intermediate results produced by the automatic feature grouping layer in this model.

[0159] Exemplarily, as Figure 7As shown, when the data stream is input into the target feature grouping model, it goes through three parts: Automatic Feature Grouping (automatic feature grouping layer), Interaction (interaction layer), and MLP (multi-layer perceptron layer, also known as the fully connected layer) before finally reaching the output end. After the model training is completed, this application only uses the intermediate results of the Automatic Feature Grouping layer.

[0160] For example, set p to 2. For the second-order cross, select the N_2 feature groups with the best effects for crossing, where N is the number of initial features. This application does not limit the value of p. That is to say, the second order can be extended to multiple orders, and the order is the number of features participating in the grouping. Assume the features in the model are f1,...f i ...,f N The feature groups for the p-order cross are (the number of feature groups is x, and this number can be regarded as a hyperparameter). The essence of realizing automatic feature grouping lies in deciding whether to select the i-th feature f i and add it to the j-th feature group This can be achieved through, but not limited to, the idea in differentiable neural architecture search, and use reparameterization to transform the discrete optimization problem (the above automatic feature grouping problem) into a problem that can be learned through gradient descent.

[0161] Specifically, for the p-order cross, the method first randomly initializes a matrix of size N*X, where N is the number of initial features and x is the number of feature groups. The value of each element in this matrix represents the probability that the i-th feature f i is added to the j-th feature group. This probability matrix will be updated as the model is trained. After the model training is completed, a relatively accurate grouping probability can be obtained. For the j-th feature group, this application first obtains the probabilities of all features being added to this feature group. On this basis, this application takes the top-p features of the probabilities as the corresponding features within this feature group (the second order is top-2).

[0162] It should be noted that Figure 7The Interaction layer and the MLP layer in are only used when training the model. The main function of the Interaction layer is to perform interaction modeling on the features within the feature groups. Specifically, it is implemented using a feature interaction method in the form of FM, where FM represents Factorization Machines, that is, a factorization machine, a machine learning model for processing feature interactions, which can effectively capture the interaction relationships between features. The function of the MLP layer is to model by concatenating the representations of the previous high-order feature interactions to obtain the final recommendation score. Concatenate means to splice the outputs of different fully connected layers to fuse multiple feature information.

[0163] S2. Based on the grouping results obtained from the target feature grouping model, conduct within-group combination and evaluation of the effects. After the target feature grouping model trained through S1, X feature groups can be obtained. These groups usually include p features, and these p features are considered features with combined value. After obtaining these feature groups, the p features within the feature groups can be combined, and the number of candidate combined features is X. Generally speaking, this parameter will be much smaller than The combination method within the feature group is to splice the single features within the group. For example, if the single features within a feature group are ["age", "gender"], then the combined feature is "age_gender". "age_gender" will be added as a new feature to the recommendation model.

[0164] Specifically, when adding the new feature to the recommendation model, a feature vector will be randomly initialized for each feature value of the new feature. This feature vector is concatenated with the feature vectors of other existing features and input into the recommendation model for training, which is the new combined feature. After training the recommendation model for the new combined feature, the effectiveness of this combined feature can be determined by the model's performance on the test sample set. An improvement threshold for the evaluation metric can be set. Only when the improvement of the metric of the added combined feature on the evaluation set exceeds the threshold will it be considered an effective combined feature.

[0165] In an exemplary embodiment, after screening out new combined features through the method for determining combined features proposed in this application, experiments were conducted in the recommendation model. The specific experimental results are shown in the following table:

[0166] Setting Recall@10 GAUC Initial Feature 0.7319 0.7851 Initial Feature + Candidate Combined Feature 0.7439 0.7929

[0167] From the above experimental results, it can be seen that by comparing the model performance of the first recommendation model trained using only the initial features with the model performance of the second recommendation model trained by combining the initial features and the candidate combined features, the model performance of the second recommendation model has been significantly improved. In this application, such asFigure 7 The automatic feature automatic grouping learning method generated by the target feature grouping model shown is used as the basis for feature grouping. It should be noted that any solution that can achieve automatic feature grouping can be used in the process of determining the combined features of this application.

[0168] It can be understood that in the specific implementation manners of this application, data related to object information, etc. is involved. When the above embodiments of this application are applied to specific products or technologies, object permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions.

[0169] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0170] According to another aspect of the embodiments of this application, there is also provided a combined feature determination device for implementing the above combined feature determination method. As Figure 10 shown, the device includes:

[0171] An acquisition module 1002, configured to train an initial recommendation model based on an initial feature set to obtain a first recommendation model corresponding to a first evaluation model metric;

[0172] A grouping module 1004, configured to determine the number of features in each feature group according to a preset order, group the initial feature set, and obtain a plurality of feature groups respectively matching the number of features in each feature group;

[0173] A training module 1006, configured to, when the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, train the initial recommendation model based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to a second evaluation model metric;

[0174] A determination module 1008, configured to determine the candidate combined features as target combined features when the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model.

[0175] As an alternative solution, the above-mentioned device is used to group the initial feature set to obtain a plurality of feature groups respectively matching the number of features of each feature group according to a preset order in the following manner: A processing unit is configured to input the above-mentioned initial feature set into a pre-trained target feature grouping model to determine a target probability matrix, where the number of rows of the above-mentioned target probability matrix is the same as the number of features of the above-mentioned initial feature set, the number of columns of the above-mentioned target probability matrix is the same as the number of feature groups of the above-mentioned plurality of feature groups, and the elements in the above-mentioned target probability matrix are used to represent the probability from the target initial feature group to the target feature group; A grouping unit is configured to group the above-mentioned initial feature set according to the above-mentioned target probability matrix to obtain the above-mentioned plurality of feature groups.

[0176] As an alternative solution, the above-mentioned device is further configured to: before inputting the above-mentioned initial feature set into the pre-trained target feature grouping model, obtain a sample feature set and sample annotation results corresponding to each sample feature in the above-mentioned sample feature set, where the above-mentioned sample feature set has the same feature category as the above-mentioned initial feature set, and the above-mentioned sample annotation results are used to represent the true recommendation results corresponding to the above-mentioned sample features; input the above-mentioned sample feature set and the above-mentioned sample annotation results into the initial feature grouping model to be trained to obtain predicted recommendation results, and use the above-mentioned true recommendation results and the above-mentioned predicted recommendation results to train the above-mentioned initial feature grouping model to obtain the above-mentioned target feature grouping model.

[0177] As an alternative, the above-mentioned device is used to input the above-mentioned sample feature set and the above-mentioned sample annotation result into the initial feature grouping model to be trained, obtain a predicted recommendation result, and use the above-mentioned true recommendation result and the above-mentioned predicted recommendation result to train the above-mentioned initial feature grouping model to obtain the above-mentioned target feature grouping model in the following manner: Input the above-mentioned sample feature set and the above-mentioned sample annotation result into the above-mentioned initial feature grouping model to obtain an initial probability matrix, where the number of rows and columns of the above-mentioned initial probability matrix is the same as that of the above-mentioned target probability matrix, and the values of each element in the above-mentioned initial probability matrix are randomly set; Automatically group the sample features in the above-mentioned sample feature set according to the above-mentioned initial probability matrix to obtain a plurality of sample feature groups, where the number of features in the above-mentioned sample feature set grouped into each of the above-mentioned plurality of feature groups is the above-mentioned preset order; Perform an encoding operation on each sample feature in the above-mentioned sample feature set respectively to determine a feature vector set, where the feature vectors in the above-mentioned feature vector set correspond one-to-one with the sample features in the above-mentioned sample feature set; Determine a plurality of feature group vectors according to the above-mentioned feature vector set and the above-mentioned plurality of sample feature groups, where the above-mentioned plurality of feature group vectors correspond one-to-one with the above-mentioned plurality of sample feature groups; Input the above-mentioned plurality of feature group vectors into a multi-layer perceptron to determine a predicted recommendation result, and use the above-mentioned true recommendation result and the above-mentioned predicted recommendation result to train the above-mentioned initial feature grouping model to obtain the above-mentioned target feature grouping model, where when the above-mentioned initial feature grouping model is trained into the above-mentioned target feature grouping model, the above-mentioned initial probability matrix is simultaneously trained into the above-mentioned target probability matrix.

[0178] As an alternative, the above-mentioned device is used to perform intra-group feature combination on the features of each of the above-mentioned feature groups respectively, and obtain candidate combined features corresponding to each of the above-mentioned feature groups: Perform intra-group feature combination on the features of each of the above-mentioned feature groups respectively, and obtain candidate combined features corresponding to each of the above-mentioned feature groups, where the feature group for which intra-group feature combination is performed each time is regarded as the current feature group: Screen out N current features from the above-mentioned current feature group, where the above-mentioned N current features are the N features with the highest probability of being grouped into the above-mentioned current feature group determined according to the above-mentioned target probability matrix in the above-mentioned current feature group, and N is a positive integer greater than or equal to 2; Concatenate the above-mentioned N current features to determine a set of candidate combined features corresponding to the above-mentioned current feature group among the above-mentioned candidate combined features.

[0179] As an alternative, the above-mentioned device is used to obtain a plurality of feature groups respectively matching the number of features in each of the said feature groups in the following manner: when the above-mentioned plurality of feature groups include X feature groups and the above-mentioned initial feature set includes Y features, the probability M of the i-th feature group to the j-th feature group is determined sequentially according to the above-mentioned target probability matrix ij , where 0 < i ≤ X, 0 < j ≤ Y, X is a positive integer, and Y is a positive integer greater than or equal to 2; according to the above-mentioned probability M ij to determine whether to group the i-th feature to the j-th feature.

[0180] As an alternative, the above-mentioned device is further used to: obtain training sample data, where the above-mentioned training sample data represents the sample data used for training the above-mentioned initial recommendation model; perform feature extraction on the above-mentioned training sample data to obtain the above-mentioned initial feature set; use the above-mentioned initial feature set to train the above-mentioned initial recommendation model to obtain the above-mentioned first recommendation model; use the above-mentioned initial feature set and the above-mentioned candidate combined features to train the above-mentioned initial recommendation model to obtain the above-mentioned second recommendation model.

[0181] As an alternative, the above-mentioned device is further used to: obtain test sample data, where the above-mentioned test sample data represents the sample data used for testing the above-mentioned first recommendation model and the above-mentioned second recommendation model; input the above-mentioned test sample data into the above-mentioned first recommendation model to determine the above-mentioned first evaluation model index; input the above-mentioned test sample data and the above-mentioned candidate combined features into the above-mentioned second recommendation model to determine the above-mentioned second evaluation model index.

[0182] As an alternative, when the above-mentioned device satisfies the preset index condition between the first evaluation model index and the second evaluation model index, the above-mentioned candidate combined features are determined as the target combined features in the following manner: when the above-mentioned second evaluation model index is greater than the above-mentioned first evaluation model index, the above-mentioned candidate combined features are determined as the above-mentioned target combined features; or when the above-mentioned second evaluation model index is greater than the above-mentioned first evaluation model index and the difference between the above-mentioned second evaluation model index and the above-mentioned first evaluation model index is greater than a preset threshold, the above-mentioned candidate combined features are determined as the above-mentioned target combined features.

[0183] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the function of that module or unit.

[0184] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0185] According to one aspect of the present application, there is provided a computer program product, which includes a computer program.

[0186] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0187] Figure 11 A block diagram of a computer system of an electronic device for implementing the embodiments of the present application is schematically shown.

[0188] It should be noted that Figure 11 The computer system 1100 of the electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0189] As Figure 11 shown, the computer system 1100 includes a central processing unit 1101 (Central Processing Unit, CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1102 (Read-Only Memory, ROM) or a program loaded from a storage section 1108 into a random access memory 1103 (Random Access Memory, RAM). In the random access memory 1103, various programs and data required for system operations are also stored. The central processing unit 1101, the read-only memory 1102, and the random access memory 1103 are connected to each other through a bus 1104. An input / output interface 1105 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1104.

[0190] The following components are connected to the input / output interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a local area network card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read from it can be installed into the storage section 1108 as needed.

[0191] Specifically, according to an embodiment of the present application, the processes described in each method flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit 1101, various functions defined in the system of the present application are executed.

[0192] In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit 1101, various functions provided by the embodiments of the present application are executed.

[0193] According to another aspect of the embodiments of the present application, an electronic device for implementing the determination method of the above combination features is further provided. The electronic device may be Figure 1 the terminal device or server shown. This embodiment takes the electronic device as the terminal device as an example for illustration. As Figure 12 shown, the electronic device includes a memory 1202 and a processor 1204. A computer program is stored in the memory 1202, and the processor 1204 is configured to execute the steps in any one of the above method embodiments through the computer program.

[0194] Optionally, in this embodiment, the above electronic device may be at least one network device among multiple network devices in a computer network.

[0195] Optionally, in this embodiment, the above processor may be configured to execute the methods in the embodiments of the present application through a computer program.

[0196] Optionally, those of ordinary skill in the art can understand that Figure 12 the structure shown is only schematic Figure 12 and does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components (such as a network interface, etc.) than those shown Figure 12 in the figure, or have a different configuration from that shown Figure 12 in the figure.

[0197] Among them, the memory 1202 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for determining combined features in the embodiments of the present application. The processor 1204 executes various functional applications and data processing by running the software programs and modules stored in the memory 1202, that is, implements the method for determining combined features described above. The memory 1202 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 1202 may further include a memory remotely disposed relative to the processor 1204, and these remote memories can be connected to the terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and their combinations. Among them, the memory 1202 can specifically but not limitedly be used to store information such as an initial feature set and multiple groups of candidate combined features. As an example, as Figure 12 shown, the above-mentioned memory 1202 may include but are not limited to the acquisition module 1002, grouping module 1004, combination unit 1006, and determination unit 1008 in the device for determining combined features described above. In addition, it may further include but are not limited to other module units in the device for determining combined features described above, which will not be elaborated in this example.

[0198] Optionally, the above-mentioned transmission device 1206 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one instance, the transmission device 1206 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable so as to communicate with the Internet or a local area network. In one instance, the transmission device 1206 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0199] In addition, the above-mentioned electronic device further includes: a display 1208 for displaying an initial feature set and multiple groups of candidate combined features; and a connection bus 1210 for connecting each module component in the above-mentioned electronic device.

[0200] In other embodiments, the above-mentioned terminal device or server may be a node in a distributed system. Among them, the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes in the form of network communication. Among them, the nodes may form a peer-to-peer network, and any form of computing device, such as electronic devices like servers and terminals, can become a node in the blockchain system by joining the peer-to-peer network.

[0201] According to one aspect of the present application, there is provided a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the electronic device to execute the combination feature determination method provided in various optional implementation manners of the determination aspect of the above combination features.

[0202] Optionally, in this embodiment, the above computer-readable storage medium may be set to store the methods for executing the embodiments of the present application.

[0203] Optionally, in this embodiment, those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing the relevant hardware of the terminal device. The program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0204] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0205] If the integrated unit in the above embodiments is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in the storage medium and includes several instructions to enable one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0206] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0207] In several embodiments provided by the present application, it should be understood that the disclosed application program can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0208] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0210] The above is only the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for determining combined features, characterized in that, Including: Training an initial recommendation model based on an initial feature set to obtain a first recommendation model corresponding to a first evaluation model metric; Determining the number of features in each feature group according to a preset order, and grouping the initial feature set to obtain a plurality of feature groups respectively matching the number of features in each feature group; When the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, training the initial recommendation model based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to a second evaluation model metric; When the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model, determining the candidate combined features as target combined features.

2. The method according to claim 1, wherein The determining the number of features in each feature group according to a preset order, and grouping the initial feature set to obtain a plurality of feature groups respectively matching the number of features in each feature group includes: Inputting the initial feature set into a pre-trained target feature grouping model to determine a target probability matrix, where the number of rows of the target probability matrix is the same as the number of features in the initial feature set, the number of columns of the target probability matrix is the same as the number of feature groups in the plurality of feature groups, and the elements in the target probability matrix are used to represent the probability from the target initial feature group to the target feature group; Grouping the initial feature set according to the target probability matrix to obtain the plurality of feature groups.

3. The method according to claim 2, characterized in that, Before inputting the initial feature set into the pre-trained target feature grouping model, the method further includes: Obtaining a sample feature set and sample annotation results corresponding to each sample feature in the sample feature set, where the sample feature set has the same feature category as the initial feature set, and the sample annotation results are used to represent the true recommendation results corresponding to the sample features; Inputting the sample feature set and the sample annotation results into an initial feature grouping model to be trained to obtain a predicted recommendation result, and training the initial feature grouping model using the true recommendation result and the predicted recommendation result to obtain the target feature grouping model.

4. The method according to claim 3, characterized in that The inputting the sample feature set and the sample annotation results into the initial feature grouping model to be trained to obtain a predicted recommendation result, and training the initial feature grouping model using the true recommendation result and the predicted recommendation result to obtain the target feature grouping model includes: Inputting the sample feature set and the sample annotation results into the initial feature grouping model to obtain an initial probability matrix, where the number of rows and columns of the initial probability matrix is the same as that of the target probability matrix, and the values of the elements in the initial probability matrix are randomly set; Automatically grouping the sample features in the sample feature set according to the initial probability matrix to obtain a plurality of sample feature groups, where the number of features in the sample feature set grouped into each feature group in the plurality of feature groups is the preset order; Perform an encoding operation on each sample feature in the sample feature set to determine a feature vector set, where the feature vectors in the feature vector set correspond one-to-one with the sample features in the sample feature set; Determine a plurality of feature grouping vectors according to the feature vector set and the plurality of sample feature groupings, where the plurality of feature grouping vectors correspond one-to-one with the plurality of sample feature groupings; Input the plurality of feature grouping vectors into a multi-layer perceptron to determine a predicted recommendation result, and use the true recommendation result and the predicted recommendation result to train the initial feature grouping model to obtain the target feature grouping model. When the initial feature grouping model is trained into the target feature grouping model, the initial probability matrix is simultaneously trained into the target probability matrix.

5. The method according to claim 2, characterized in that The step of respectively performing intra-group feature combination on the features of each feature grouping and obtaining candidate combined features corresponding to each feature grouping includes: Respectively perform intra-group feature combination on the features of each feature grouping in the following manner to obtain candidate combined features corresponding to each feature grouping, where the feature grouping for which intra-group feature combination is performed each time is regarded as the current feature grouping: Select N current features from the current feature grouping, where the N current features are the N features with the highest probability of being grouped into the current feature grouping determined according to the target probability matrix in the current feature grouping, and N is a positive integer greater than or equal to 2; Concatenate the N current features to determine a group of candidate combined features corresponding to the current feature grouping among the candidate combined features.

6. The method according to claim 2, characterized in that The step of obtaining a plurality of feature groupings respectively matching the number of features of each feature grouping includes: In the case that the multiple feature groups include X feature groups and the initial feature set includes Y features, the probability M of the i-th feature group to the j-th feature group is determined successively according to the target probability matrix, ij where 0 < i ≤ X, 0 < j ≤ Y, and X is a positive integer. Y is a positive integer greater than or equal to 2; According to the probability M ij Determine whether to group the i-th feature into the j-th feature group.

7. The method according to claim 1, characterized in that The method further includes: Obtain training sample data, where the training sample data represents the sample data used for training the initial recommendation model; Extract features from the training sample data to obtain the initial feature set; Train the initial recommendation model using the initial feature set to obtain the first recommendation model; Train the initial recommendation model using the initial feature set and the candidate combined features to obtain the second recommendation model.

8. The method according to claim 7, characterized in that The method further includes: Obtain test sample data, where the test sample data represents the sample data used for testing the first recommendation model and the second recommendation model; Input the test sample data into the first recommendation model to determine the first evaluation model metric; Input the test sample data and the candidate combined features into the second recommendation model to determine the second evaluation model metric.

9. The method according to claim 8, wherein The method further includes: In the case where the second evaluation model metric is greater than the first evaluation model metric, determine the candidate combined features as the target combined features; or In the case where the second evaluation model metric is greater than the first evaluation model metric, and the difference between the second evaluation model metric and the first evaluation model metric is greater than a preset threshold, the candidate combined feature is determined as the target combined feature.

10. An apparatus for determining a combined feature, characterized in that Including: An acquisition module, configured to train an initial recommendation model based on an initial feature set to obtain a first recommendation model corresponding to a first evaluation model metric; A grouping module, configured to determine the number of features in each feature group according to a preset order, group the initial feature set, and obtain a plurality of feature groups respectively matching the number of features in each feature group; A training module, configured to, in the case where the features in each feature group are respectively combined within the group and candidate combined features corresponding to each feature group are obtained, train the initial recommendation model based on the initial feature set and the candidate combined features to obtain a second recommendation model corresponding to a second evaluation model metric; A determination module, configured to determine the candidate combined feature as the target combined feature in the case where the first evaluation model metric and the second evaluation model metric indicate that the model effect of the second recommendation model is better than that of the first recommendation model.

11. The device according to claim 10, characterized in that, The grouping module includes: A processing unit, configured to input the initial feature set into a pre-trained target feature grouping model to determine a target probability matrix, where the number of rows of the target probability matrix is the same as the number of features in the initial feature set, the number of columns of the target probability matrix is the same as the number of feature groups of the plurality of feature groups, and the elements in the target probability matrix are used to represent the probability from the target initial feature group to the target feature group; A grouping unit, configured to group the initial feature set according to the target probability matrix to obtain the plurality of feature groups.

12. The device according to claim 11, characterized in that, The apparatus is further configured to: Before inputting the initial feature set into the pre-trained target feature grouping model, obtain a sample feature set and sample annotation results corresponding to each sample feature in the sample feature set, where the sample feature set has the same feature category as the initial feature set, and the sample annotation results are used to represent the true recommendation results corresponding to the sample features; Input the sample feature set and the sample annotation results into the initial feature grouping model to be trained to obtain predicted recommendation results, and use the true recommendation results and the predicted recommendation results to train the initial feature grouping model to obtain the target feature grouping model.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, where the computer program can be executed by an electronic device to execute the method according to any one of claims 1 to 9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

15. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 9 through the computer program.

Images