Method and system for generating combined features of machine learning samples

Abstract

Provided is a method and system for generating combined features of machine learning samples. The method comprises: obtaining historical data records, wherein the historical data records include multiple attribute information; according to a search strategy, iteratively performing feature combination between at least one feature generated based on multiple attribute information to generate candidate combined features, and selecting target combined features as combined features of machine learning samples, wherein for each round of iteration, pre-ranking of importance is performed on each candidate combined feature set; a portion of candidate combined features is screened out according to the pre-ranking result to form a candidate combined feature pool; re-ranking of importance is performed on each candidate combined feature in the candidate combined feature pool; and at least one candidate combined feature with higher importance is selected as the target combined feature according to the re-ranking result. Automatic feature combination can be effectively realized while using fewer computing resources, thereby improving model effects.

Description

[0001] This application is a divisional application of a patent application with application date of September 8, 2017, application number 201710803930.4, and titled “Method and system for generating combined features of machine learning samples”. Technical Field

[0002] The present invention generally relates to the field of artificial intelligence, and more specifically, to a method and system for generating combined features of machine learning samples. Background Art

[0003] With the emergence of massive data, artificial intelligence technology has developed rapidly. In order to extract value from large amounts of data, it is necessary to generate samples suitable for machine learning based on data records.

[0004] Here, each data record can be regarded as a description of an event or object, corresponding to an example or sample. In the data record, various items reflecting the performance or nature of the event or object in a certain aspect are included, and these items can be called "attributes".

[0005] How to convert the various attributes of the original data records into the features of the machine learning samples will have a great impact on the effect of the machine learning model. In fact, the prediction effect of the machine learning model is related to the selection of the model, the available data and the extraction of features. In other words, the prediction effect of the model can be improved by improving the feature extraction method. On the contrary, if the feature extraction is inappropriate, the prediction effect will deteriorate.

[0006] However, in the process of determining the feature extraction method, technicians are often required to not only master the knowledge of machine learning, but also have a deep understanding of the actual prediction problem. Prediction problems are often combined with different practical experiences in different industries, which makes it difficult to achieve satisfactory results. In particular, when combining different features, on the one hand, it is difficult to grasp which features to combine from the perspective of prediction effect, and on the other hand, from the perspective of computational efficiency, it is also difficult to effectively screen out specific combinations. In summary, it is difficult to automatically combine features in the prior art. Summary of the invention

[0007] The exemplary embodiments of the present invention are intended to overcome the defect in the prior art that it is difficult to automatically combine the features of machine learning samples.

[0008] According to an exemplary embodiment of the present invention, there is provided a method for generating combined features of machine learning samples, comprising: (A) acquiring historical data records, wherein the historical data records include multiple attribute information; and (B) according to a search strategy, iteratively performing feature combinations between at least one feature generated based on the multiple attribute information to generate candidate combined features, and selecting target combined features from the generated candidate combined features as combined features of machine learning samples, wherein for each round of iteration, pre-sorting the importance of each candidate combined feature in a candidate combined feature set; screening a portion of the candidate combined features from the candidate combined feature set according to the pre-sorting result to form a candidate combined feature pool; re-sorting the importance of each candidate combined feature in the candidate combined feature pool; and selecting at least one candidate combined feature with higher importance from the candidate combined feature pool as the target combined feature according to the re-sorting result.

[0009] Optionally, in the method, pre-sorting is performed based on a first number of historical data records, and re-sorting is performed based on a second number of historical data records, and the second number is not less than the first number.

[0010] Optionally, in the method, candidate combination features with higher importance are screened out from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool.

[0011] Optionally, in the method, the candidate combined feature set includes candidate combined features generated in the current round of iteration; or, the candidate combined feature set includes candidate combined features generated in the current round of iteration and candidate combined features generated in previous rounds of iteration that are not selected as target combined features.

[0012] Optionally, in the method, candidate combination features for the next round of iteration are generated by combining the target combination features selected in the current round of iteration with the at least one feature; or, candidate combination features for the next round of iteration are generated by performing pairwise combinations between the target combination features selected in the current round of iteration and the previous round of iteration.

[0013] Optionally, in the method, pre-sorting is performed by the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted single feature machine learning model, wherein the pre-sorted single feature machine learning model corresponds to each candidate combination feature.

[0014] Optionally, in the method, pre-sorting is performed by the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted overall machine learning model, wherein the pre-sorted overall machine learning model corresponds to the pre-sorted basic feature subset and each candidate combination feature.

[0015] Optionally, in the method, pre-sorting is performed by the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted composite machine learning model, wherein the pre-sorted composite machine learning model includes a pre-sorted basic sub-model and a pre-sorted additional sub-model based on a lifting framework, wherein the pre-sorted basic sub-model corresponds to a pre-sorted basic feature subset, and the pre-sorted additional sub-model corresponds to each candidate combination feature.

[0016] Optionally, in the method, the pre-sorted basic feature subset includes target combination features selected before the current round of iterations.

[0017] Optionally, in the method, re-ranking is performed by the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking single feature machine learning model, wherein the re-ranking single feature machine learning model corresponds to each candidate combination feature.

[0018] Optionally, in the method, re-ranking is performed by the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking overall machine learning model, wherein the re-ranking composite machine learning model corresponds to the re-ranking basic feature subset and each candidate combination feature.

[0019] Optionally, in the method, re-ranking is performed by the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking composite machine learning model, wherein the re-ranking composite machine learning model includes a re-ranking basic sub-model and a re-ranking additional sub-model based on a lifting framework, wherein the re-ranking basic sub-model corresponds to a re-ranking basic feature subset, and the re-ranking additional sub-model corresponds to each candidate combination feature.

[0020] Optionally, in the method, the re-ordered basic feature subset includes the target combination feature selected before the current round of iteration.

[0021] Optionally, in the method, step (B) further includes: for each round of iteration, checking whether the selected target combination features are suitable as combination features of machine learning samples.

[0022] Optionally, in the method, in step (B), the effect change of a machine learning model based on the verified target combination features after the selected target combination features are introduced is used to verify whether the selected target combination features are suitable as combination features of machine learning samples.

[0023] Optionally, in the method, when the test result is that the selected target combination features are suitable as the combination features of the machine learning samples, the selected target combination features are used as the combination features of the machine learning samples, and the next round of iteration is performed; when the test result is that the selected target combination features are not suitable as the combination features of the machine learning samples, another part of the candidate combination features are screened out from the candidate combination feature set according to the pre-sorting result to form a new candidate combination feature pool.

[0024] According to another exemplary embodiment of the present invention, a computer-readable medium for generating combined features of machine learning samples is provided, wherein a computer program for executing the method as described above is recorded on the computer-readable medium.

[0025] According to another exemplary embodiment of the present invention, there is provided a computing device for generating combined features of machine learning samples, comprising a storage component and a processor, wherein a set of computer executable instructions is stored in the storage component, and when the set of computer executable instructions is executed by the processor, the method as described above is performed.

[0026] According to another exemplary embodiment of the present invention, there is provided a system for generating combined features of machine learning samples, comprising: a data record acquisition device for acquiring historical data records, wherein the historical data records include multiple attribute information; and a feature combination device for iteratively combining features between at least one feature generated based on the multiple attribute information according to a search strategy to generate candidate combined features, and selecting target combined features from the generated candidate combined features as combined features of machine learning samples, wherein for each round of iteration, the feature combination device pre-sorts the importance of each candidate combined feature in a candidate combined feature set, selects a part of the candidate combined features from the candidate combined feature set according to the pre-sorting result to form a candidate combined feature pool, re-sorts the importance of each candidate combined feature in the candidate combined feature pool, and selects at least one candidate combined feature with higher importance from the candidate combined feature pool as the target combined feature according to the re-sorting result.

[0027] Optionally, in the system, the feature combination device performs pre-sorting based on a first number of historical data records, and re-sorting based on a second number of historical data records, and the second number is not less than the first number.

[0028] Optionally, in the system, the feature combination device selects candidate combination features with higher importance from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool.

[0029] Optionally, in the system, the candidate combined feature set includes candidate combined features generated in the current round of iteration; or, the candidate combined feature set includes candidate combined features generated in the current round of iteration and candidate combined features generated in previous rounds of iteration that are not selected as target combined features.

[0030] Optionally, in the system, the feature combination device generates candidate combination features for the next round of iteration by combining the target combination features selected in the current round of iteration with the at least one feature; or, the feature combination device generates candidate combination features for the next round of iteration by performing pairwise combinations between the target combination features selected in the current round of iteration and the previous round of iteration.

[0031] Optionally, in the system, the feature combination device performs pre-sorting through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted single feature machine learning model, wherein the pre-sorted single feature machine learning model corresponds to each candidate combination feature.

[0032] Optionally, in the system, the feature combination device performs pre-sorting by the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted overall machine learning model, wherein the pre-sorted overall machine learning model corresponds to the pre-sorted basic feature subset and each candidate combination feature.

[0033] Optionally, in the system, the feature combination device performs pre-sorting by the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorting composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorting composite machine learning model, wherein the pre-sorting composite machine learning model includes a pre-sorting basic sub-model and a pre-sorting additional sub-model based on a lifting framework, wherein the pre-sorting basic sub-model corresponds to a pre-sorting basic feature subset, and the pre-sorting additional sub-model corresponds to each candidate combination feature.

[0034] Optionally, in the system, the pre-sorted basic feature subset includes target combination features selected before the current round of iterations.

[0035] Optionally, in the system, the feature combination device performs re-ranking through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking single feature machine learning model, wherein the re-ranking single feature machine learning model corresponds to each candidate combination feature.

[0036] Optionally, in the system, the feature combination device performs re-ranking through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking overall machine learning model, wherein the re-ranking composite machine learning model corresponds to the re-ranking basic feature subset and each candidate combination feature.

[0037] Optionally, in the system, the feature combination device performs re-ranking by the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking composite machine learning model, wherein the re-ranking composite machine learning model includes a re-ranking basic sub-model and a re-ranking additional sub-model based on a lifting framework, wherein the re-ranking basic sub-model corresponds to a re-ranking basic feature subset, and the re-ranking additional sub-model corresponds to each candidate combination feature.

[0038] Optionally, in the system, the re-ordered basic feature subset includes the target combination feature selected before the current round of iteration.

[0039] Optionally, in the system, the feature combination device also checks, for each round of iteration, whether the selected target combination features are suitable as combination features of machine learning samples.

[0040] Optionally, in the system, the feature combination device uses the change in effect of a machine learning model based on the verified target combination features after the selected target combination features are introduced to verify whether the selected target combination features are suitable as combination features of machine learning samples.

[0041] Optionally, in the system, when the test result is that the selected target combination features are suitable as the combination features of the machine learning samples, the feature combination device uses the selected target combination features as the combination features of the machine learning samples and performs the next round of iteration; when the test result is that the selected target combination features are not suitable as the combination features of the machine learning samples, the feature combination device filters out another part of the candidate combination features from the candidate combination feature set according to the pre-sorting result to form a new candidate combination feature pool.

[0042] In the method and system for generating combined features of machine learning samples according to an exemplary embodiment of the present invention, a part of the combined features generated in each round of iteration is screened out through pre-sorting and re-sorting in a specific manner to finally form a combined feature set of the machine learning samples, thereby effectively realizing automatic feature combination while using fewer computing resources, thereby improving the effect of the machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] These and / or other aspects and advantages of the present invention will become more clear and easier to understand from the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, in which:

[0044] Figure 1 A block diagram showing a system for generating combined features of machine learning samples according to an exemplary embodiment of the present invention;

[0045] Figure 2 A block diagram showing a feature combination device according to an exemplary embodiment of the present invention;

[0046] Figure 3 A block diagram showing a feature combination device according to another exemplary embodiment of the present invention;

[0047] Figure 4 A block diagram showing a system for training a machine learning model according to an exemplary embodiment of the present invention;

[0048] Figure 5 A flowchart showing a method for generating a combined feature of a machine learning sample according to an exemplary embodiment of the present invention;

[0049] Figure 6 An example of a search tree for iteratively generating combined features according to an exemplary embodiment of the present invention is shown;

[0050] Figure 7 A flowchart illustrating a method for training a machine learning model according to an exemplary embodiment of the present invention; and

[0051] Figure 8 A flow chart of a method for generating combined features of machine learning samples according to another exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the present invention, exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0053] In an exemplary embodiment of the present invention, automatic feature combination is performed in the following manner: unit features that can be combined are generated based on the attribute information of data records, feature combinations that serve as candidate combination features are generated in an iterative manner, and in each round of iteration, a part of target combination features are screened out from the candidate combination features through pre-sorting and re-sorting in a specific manner to form a combination feature set of machine learning samples.

[0054] Here, machine learning is the inevitable product of the development of artificial intelligence research to a certain stage, and it is committed to improving the performance of the system itself by using experience through computational means. In a computer system, "experience" usually exists in the form of "data". Through a machine learning algorithm, a "model" can be generated from the data. In other words, by providing the experience data to the machine learning algorithm, a model can be generated based on these experience data. When faced with a new situation, the model will provide a corresponding judgment, that is, a prediction result. Whether training a machine learning model or using a trained machine learning model for prediction, the data needs to be converted into machine learning samples including various features. Machine learning can be implemented in the form of "supervised learning", "unsupervised learning" or "semi-supervised learning". It should be noted that the exemplary embodiments of the present invention do not impose specific restrictions on specific machine learning algorithms. In addition, it should also be noted that in the process of training and applying the model, other means such as statistical algorithms can also be combined.

[0055] Figure 1 A block diagram of a system for generating combined features of machine learning samples according to an exemplary embodiment of the present invention is shown. Figure 1 The system shown includes a data record acquisition device 100 and a feature combination device 200 .

[0056] Specifically, the data record acquisition device 100 is used to acquire historical data records, wherein the historical data records include a plurality of attribute information. Here, as an example, the data record acquisition device 100 may acquire historical data records that have been marked for use in supervised machine learning.

[0057] The above historical data records may be data generated online, data generated and stored in advance, or data received from the outside through an input device or a transmission medium. These data may involve attribute information of individuals, enterprises or organizations, such as identity, education, occupation, assets, contact information, liabilities, income, profit, taxation, etc. Alternatively, these data may also involve attribute information of business-related items, such as information about the transaction amount, transaction parties, subject matter, transaction location, etc. of a sales contract. It should be noted that the attribute information content mentioned in the exemplary embodiments of the present invention may involve the performance or nature of any object or transaction in a certain aspect, and is not limited to defining or describing individuals, objects, organizations, units, institutions, projects, events, etc.

[0058] The data record acquisition device 100 can acquire structured or unstructured data from different sources, such as text data or numerical data. The acquired data records can be used to form machine learning samples and participate in the training / testing process of the machine learning model. These data can come from the entity that expects to obtain the model prediction results, for example, from the bank, enterprise, school, etc. that expects to obtain the prediction results; these data can also come from outside the above entities, for example, from data providers, the Internet (for example, social networking sites), mobile operators, APP operators, express companies, credit institutions, etc. Optionally, the above internal data and external data can be used in combination to form machine learning samples with more information.

[0059] The above data can be input into the data record acquisition device 100 through an input device, or automatically generated by the data record acquisition device 100 according to existing data, or can be obtained by the data record acquisition device 100 from a network (e.g., a storage medium on the network (e.g., a data warehouse)). In addition, an intermediate data exchange device such as a server can help the data record acquisition device 100 to obtain corresponding data from an external data source. Here, the acquired data can be converted into a format that is easy to process by a data conversion module such as a text analysis module in the data record acquisition device 100.

[0060] The feature combination device 200 is used to iteratively perform feature combination between at least one feature generated based on the multiple attribute information according to the search strategy to generate candidate combination features, and select a target combination feature from the generated candidate combination features as the combination feature of the machine learning sample. Specifically, for each round of iteration, the feature combination device 200 pre-sorts the importance of each candidate combination feature in the candidate combination feature set, selects a part of the candidate combination features from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool, re-sorts the importance of each candidate combination feature in the candidate combination feature pool, and selects at least one candidate combination feature with higher importance from the candidate combination feature pool according to the re-sorting result as the target combination feature.

[0061] Here, the feature combination device 200 may first generate a feature that can be combined based on multiple attribute information recorded in the historical data (the feature can be regarded as the smallest unit that can be combined with the feature). In this process, the feature combination device 200 may adopt any appropriate feature processing method to obtain features that are easy to combine with each other. Here, a single feature can be regarded as a first-order feature. According to an exemplary embodiment of the present invention, a higher-order feature combination such as a second-order or third-order feature combination may be performed to generate corresponding candidate combination features, where "order" represents the number of single features involved in the combination.

[0062] As an example, the at least one feature generated by the feature combination device 200 can be at least one continuous feature, wherein the feature combination device 200 generates the continuous feature by processing at least one continuous value attribute information and / or discrete value attribute information among the multiple attribute information.

[0063] Specifically, based on at least a portion of the attribute information of the historical data record, a corresponding continuous feature can be generated. Here, a continuous feature is a feature relative to a discrete feature (e.g., a categorical feature), and its value can be a numerical value with a certain continuity, such as distance, age, amount, etc. In contrast, as an example, the value of a discrete feature does not have continuity, for example, it can be a feature with unordered classification such as "from Beijing", "from Shanghai" or "from Tianjin", "male", "female", etc.

[0064] For example, a certain continuous value attribute information in the historical data record can be directly used as the corresponding continuous feature, for example, the attribute information such as distance, age, amount, etc. can be directly used as the corresponding continuous feature. That is to say, each of the continuous features can be formed by the continuous value attribute information itself among the multiple attribute information. Alternatively, it is also possible to process certain attribute information (for example, continuous value attributes and / or discrete value attribute information) in the historical data record to obtain the corresponding continuous feature, for example, the ratio of height to weight is used as the corresponding continuous feature. In particular, the continuous feature can be formed by continuously transforming the discrete value attribute information among the multiple attribute information. As an example, the continuous transformation can indicate statistics on the values ​​of the discrete value attribute information. For example, the continuous feature can indicate the statistical information of certain discrete value attribute information about the prediction target of the machine learning model. For example, in the example of predicting the purchase probability, the discrete value attribute information of the seller merchant number can be transformed into the probability statistical feature of the historical purchase behavior of the corresponding seller merchant code.

[0065] The continuous features described above may be combined by methods such as arithmetic operations to serve as candidate combined features according to an exemplary embodiment of the present invention.

[0066] As another example, the at least one feature generated by the feature combination device 200 may be at least one discrete feature, wherein the feature combination device 200 generates the discrete feature by processing at least one continuous value attribute information and / or discrete value attribute information among the multiple attribute information.

[0067] Specifically, based on at least a portion of the attribute information of the historical data record, corresponding discrete features can be generated. For example, a certain discrete value attribute information in the historical data record can be directly used as a corresponding continuous feature, that is, each discrete feature can be formed by the discrete value attribute information itself among the multiple attribute information. Alternatively, certain attribute information (for example, continuous value attribute and / or discrete value attribute information) in the historical data record can be processed to obtain the corresponding discrete feature.

[0068] Here, the corresponding discrete features can be obtained by discretizing continuous features (for example, continuous value attribute information itself or continuous features formed by continuous transformation of discrete value attribute information). Preferably, when discretizing continuous features, the feature combination device 200 can perform at least one binning operation for each continuous feature to generate a discrete feature composed of at least one binning feature, wherein each binning operation corresponds to one binning feature.

[0069] Specifically, for continuous features, the feature combination device 200 can perform at least one binning operation, so as to simultaneously obtain multiple discrete features that characterize certain attributes of the original data record from different angles, scales / levels.

[0070] As an example, the combined features of the machine learning samples can be generated in an iterative manner according to the search strategy for the combined features. To this end, the feature combination device 200 can generate candidate combined features step by step in an iterative manner. As an example, in each round of iteration, the feature combination device 200 can combine new candidate combined features according to the pre-set search strategy. Here, the feature combination device 200 can pre-rank the importance of each candidate combined feature currently involved in the screening, and filter out a part of the candidate combined features according to the pre-ranking result, for example, a part of the candidate combined features with higher importance (for example, among 100 candidate combined features, the first to tenth important features can be filtered out), a part of the candidate combined features arranged in intervals of importance (for example, among 100 candidate combined features, the first important feature, the eleventh important feature, the twenty-first important feature..., the ninety-first important feature can be filtered out), etc. The filtered candidate combined features can constitute a candidate combined feature pool, so that the feature combination device 200 can further filter out target combined features with higher importance by re-ranking. After the target combined features are determined as described above, the next round of iteration can be performed to further obtain new target combined features.

[0071] It should be noted that the record acquisition device 100 and the feature combination device 200 can be configured as individual units composed of software, hardware and / or firmware, and some or all of these units can be integrated into one or work together to perform specific functions.

[0072] As an example, the following will refer to Figure 2 2 is a block diagram of a feature combination device 200 according to an exemplary embodiment of the present invention. Figure 2 The feature combination device 200 may include a candidate combination feature generating unit 210 , a pre-sorting unit 220 and a re-sorting unit 230 .

[0073] Here, the candidate combination feature generating unit 210 is used to generate candidate combination features for each round of iteration according to the search strategy.

[0074] The candidate combined feature generation unit 210 may first generate combinable features based on the attribute information of the historical data records in the first round of iterations. As described above, the combinable features may be regarded as the smallest unit that can be combined, which may be either continuous features or discrete features.

[0075] As an example, the combinable feature may indicate a discrete feature, in which case the corresponding combinable discrete feature may be obtained by discretizing the continuous feature (e.g., the continuous value attribute information itself). Preferably, when discretizing the continuous feature, the candidate combined feature generation unit 210 may perform at least one binning operation for each continuous feature to generate a discrete feature composed of at least one binning feature, wherein each binning operation corresponds to one binning feature.

[0076] Specifically, corresponding continuous features can be generated for at least a portion of the attribute information of the historical data record. For example, a certain continuous value attribute information in the historical data record can be directly used as the corresponding continuous feature, for example, the attribute information such as distance, age, amount, etc. can be directly used as the corresponding continuous feature. That is to say, each of the continuous features can be formed by the continuous value attribute information itself among the multiple attribute information. Alternatively, it is also possible to process certain attribute information (for example, continuous value attributes and / or discrete value attribute information) in the historical data record to obtain the corresponding continuous feature, for example, the ratio of height to weight is used as the corresponding continuous feature. In particular, the continuous feature can be formed by continuously transforming the discrete value attribute information among the multiple attribute information. As an example, the continuous transformation can indicate statistics on the values ​​of the discrete value attribute information. For example, the continuous feature can indicate the statistical information of certain discrete value attribute information about the prediction target of the machine learning model. For example, in the example of predicting the purchase probability, the discrete value attribute information of the seller merchant number can be transformed into the probability statistical feature of the historical purchase behavior of the corresponding seller merchant code.

[0077] In addition, in addition to the continuous features that will be binned, the candidate combined feature generation unit 210 may also generate other discrete features. As an alternative, the above features may also be generated by other feature generation devices (not shown). According to an exemplary embodiment of the present invention, the above features may be arbitrarily combined, wherein the continuous features have been converted into bin group features when combined.

[0078] For each continuous feature, the candidate combined feature generation unit 210 may perform at least one binning operation, so as to simultaneously obtain multiple discrete features that characterize certain attributes of the original data record from different angles, scales / levels.

[0079] Here, binning operation refers to a specific way of discretizing continuous features, that is, dividing the value range of continuous features into multiple intervals (that is, multiple boxes), and determining the corresponding binning feature values ​​based on the divided boxes. Binning operations can be roughly divided into supervised binning and unsupervised binning, and these two types each include some specific binning methods. For example, supervised binning includes minimum entropy binning, minimum description length binning, etc., while unsupervised binning includes equal width binning, equal depth binning, binning based on k-means clustering, etc. In each binning method, corresponding binning parameters can be set, such as width, depth, etc. It should be noted that according to an exemplary embodiment of the present invention, the binning operation performed by the candidate combined feature generation unit 210 does not limit the type of binning method, nor does it limit the parameters of the binning operation, and the specific representation method of the corresponding generated binning features is also not limited.

[0080] The binning operations performed by the candidate combination feature generation unit 210 may differ in binning methods and / or binning parameters. For example, the at least one binning operation may be a binning operation of the same type but with different operation parameters (e.g., depth, width, etc.), or may be a binning operation of different types. Accordingly, each binning operation may obtain a binning feature, and these binning features together constitute a binning group feature, which may reflect different binning operations, thereby improving the effectiveness of machine learning materials and providing a good basis for the training / prediction of machine learning models.

[0081] The above shows the process of discretizing continuous features in the first round of iteration. However, it should be understood that according to an exemplary embodiment of the present invention, the continuous features can be discretized only for the first round of iteration to obtain discrete features that are always used for combination in the subsequent rounds at one time; or the discretization process can be re-performed for subsequent iterations (for example, for each round of iterations) to obtain discrete features corresponding to the relevant subsequent iterations.

[0082] As an example, the at least one binning operation may be selected from a predetermined number of binning operations for each iteration or for all iterations, wherein the importance of the binning feature corresponding to the selected binning operation is not less than the importance of the binning feature corresponding to the unselected binning operation. Here, the candidate combined feature generation unit 210 may measure the importance of each binning feature by any means of determining feature importance.

[0083] As another example, the features that can be combined may indicate continuous features. In this case, the candidate combined feature generation unit 210 may directly use any continuous features generated as described above as features to be combined; or, the candidate combined feature generation unit 210 may continuously transform any discrete features generated as described above to obtain features to be combined.

[0084] That is, according to an exemplary embodiment of the present invention, for each continuous feature, after obtaining at least one corresponding binning feature by performing at least one binning operation, a feature corresponding to the continuous feature can be obtained by taking each binning feature as a component element, and the feature can be regarded as a set of binning features, and combined with continuous features and / or discrete features. Here, it should be understood that the continuous feature is discretized and placed in a corresponding specific box due to the execution of the binning operation. However, it should be noted that according to an exemplary embodiment of the present invention, in the converted binning feature, each dimension can indicate whether a discrete value of the continuous feature is assigned to the box (for example, "0" or "1"), or a specific continuous value (for example, a characteristic value, an average value, an intermediate value, a boundary value, a normalized value, etc.). Accordingly, when the discrete values ​​of each dimension (for example, for classification problems) or continuous values ​​(for example, for regression problems) are specifically applied in machine learning, a combination between discrete values ​​(for example, Cartesian products, etc.) or a combination between continuous values ​​(for example, arithmetic operation combinations, etc.) can be performed.

[0085] In subsequent iterations, the candidate combined feature generation unit 210 may generate new candidate combined features according to the search strategy. Here, the search strategy may be intended to prune the search tree for the combined features to control the number of candidate combined features generated in each iteration. For example, the candidate combined feature generation unit 210 may generate new candidate combined features in each iteration based only on the target combined features selected in the previous iteration.

[0086] The pre-sorting unit 220 is used to pre-sort the importance of each candidate combination feature in the candidate combination feature set for each round of iteration. Here, the candidate combination feature set may include candidate combination features generated in one or more rounds of iteration, for example, candidate combination features generated in the current round of iteration or in conjunction with several previous rounds of iteration. The pre-sorting unit 220 may measure the importance of each candidate combination feature in the candidate combination feature set by any means of judging feature importance. Through pre-sorting, the importance order of each candidate combination feature can be known, and on this basis, the pre-sorting unit 220 can screen out a part of the candidate combination features to form a candidate combination feature pool, where the screened candidate combination features can be expressed as a certain consistency in terms of predictive effect, so that only features with higher importance can be screened out as target combination features of machine learning samples.

[0087] Accordingly, the re-ranking unit 230 is used to re-rank the importance of each candidate combination feature in the candidate combination feature pool, and select at least one candidate combination feature with higher importance from the candidate combination feature pool as the target combination feature according to the re-ranking result. Here, the re-ranking unit 230 can use any means of judging the importance of features to measure the importance of each candidate combination feature in the candidate combination feature pool. For example, the re-ranking unit 230 can measure the importance of the candidate combination features in the candidate combination feature pool in the same manner as the pre-ranking unit 220, but it makes a more accurate judgment based on a larger number of data records. The re-ranking unit 230 can select a predetermined number of the most important candidate combination features in the candidate combination feature pool as the target combination features. Here, the target combination features can be directly used as the combination features of the machine learning samples, or the target combination features can be further verified to determine whether to use them as the combination features of the machine learning samples. As an example, if no suitable target combination feature is screened out in the current candidate combination feature pool or it is necessary to continue to screen suitable target combination features in the current round of iteration, a new candidate combination feature pool can be re-determined according to the pre-sorting result, for example, among the 100 candidate combination features, the eleventh to twentieth important features can be screened out again; or, among the 100 candidate combination features, the second most important feature, the twelfth most important feature, the twenty-second most important feature..., the ninety-second most important feature can be screened out again. On the other hand, at the end of the screening process of the current round of iteration, the next round of iteration can be performed to generate new candidate combination features.

[0088] exist Figure 2 In the feature combination device 200 shown, the candidate combination feature generation unit 210, the pre-sorting unit 220 and the re-sorting unit 230 are all involved in determining the importance of features. Accordingly, as an optional method, the above three units can share some operation parameters or results to save resources.

[0089] Figure 3 A block diagram of a feature combination device 200 according to another exemplary embodiment of the present invention is shown. Figure 3 In the feature combination device 200 shown, in addition to the candidate combination feature generation unit 210, the pre-sorting unit 220 and the re-sorting unit 230, it further includes a verification unit 240, which is used to verify whether the target combination feature selected by the re-sorting unit 230 is suitable as a combination feature of a machine learning sample for each round of iteration.

[0090] Here, the candidate combination feature generating unit 210, the pre-sorting unit 220 and the re-sorting unit 230 can be based on the reference Figure 2 The operation is performed in the manner described above, and the details will not be repeated here. In addition, the target combination features selected each time by the re-ranking unit 230 will not be directly used as the combination features of the machine learning samples, but will be verified by the verification unit 240. As an example, the verification unit 240 can verify whether the selected target combination features are suitable as the combination features of the machine learning samples by integrating them into the actual machine learning model that will perform predictions for the prediction problem. For example, the verification unit 240 can introduce the target combination features to be verified into the machine learning model based on the verified combination features, and verify whether the target combination features to be verified are suitable as the combination features of the machine learning samples by measuring the changes in the effect of the model.

[0091] Figure 1 (Combined with Figure 2 and Figure 3 ) is intended to generate combined features of machine learning samples. The system can exist independently. Here, it should be noted that the way in which the system obtains data records is not limited. That is, as an example, the data record acquisition device 100 can be a device capable of receiving and processing data records, or it can be a device that only provides prepared data records. In addition, the above system can also be integrated into the model training system as a component to complete feature processing.

[0092] Figure 4 A block diagram of a training system for a machine learning model according to an exemplary embodiment of the present invention is shown. Figure 4 In the system shown, in addition to the above-mentioned data record acquisition device 100 and feature combination device 200, it also includes a machine learning sample generation device 300 and a machine learning model training device 400.

[0093] Specifically, in Figure 4 In the system shown in FIG. 1 , the data record acquisition device 100 and the feature combination device 200 can be configured as follows: Figures 1 to 3The operation is performed in the manner shown, where, as an example, the data record acquisition device 100 can acquire the historical data records that have been marked.

[0094] In addition, the machine learning sample generating device 300 is used to generate a machine learning sample that includes at least a part of the generated combined features. That is, in the machine learning sample generated by the machine learning sample generating device 300, a part or all of the combined features generated by the feature combining device 200 are included. In addition, as an optional method, the machine learning sample may also include any other features generated based on the attribute information of the data record, for example, features obtained by performing feature processing on the attribute information of the data record, etc. As an example, these other features may be generated by the feature combining device 200, or by other devices.

[0095] Specifically, the machine learning sample generating device 300 can generate a machine learning training sample. In particular, as an example, in the case of supervised learning, the machine learning training sample generated by the machine learning sample generating device 300 may include two parts: a feature and a label.

[0096] The machine learning model training device 400 is used to train the machine learning model based on the machine learning training samples. Here, the machine learning model training device 400 can use any appropriate machine learning algorithm (e.g., log probability regression) to learn an appropriate machine learning model from the machine learning training samples. As an example, the machine learning model training device 400 can use a machine learning algorithm that is the same as or similar to the model used by the combined feature generation device 200 to measure the importance of related features.

[0097] In the above example, a relatively stable machine learning model with good prediction effect can be trained.

[0098] The following combination Figure 5 1 is a flowchart of a method for generating a combined feature of a machine learning sample according to an exemplary embodiment of the present invention. Here, as an example, Figure 5 The method shown can be Figure 1 The system and its device shown in the figure can be implemented by the computer program in software or by a computing device with a specific configuration. Figure 5 For the convenience of description, assume that Figure 5 The method shown is Figure 1 The system shown is implemented and assumes Figure 1 The feature combination device 200 may have Figure 2 The structure shown.

[0099] As shown in the figure, in step S100, the data record acquisition device 100 acquires historical data records, wherein the historical data records include a plurality of attribute information.

[0100] Here, as an example, the data record acquisition device 100 can collect data manually, semi-automatically or fully automatically, or process the collected raw data so that the processed data records have an appropriate format or form. As an example, the data record acquisition device 100 can collect historical data in batches.

[0101] Here, the data record acquisition device 100 can receive data records manually input by a user through an input device (e.g., a workstation). In addition, the data record acquisition device 100 can systematically retrieve data records from a data source in a fully automatic manner, for example, by systematically requesting a data source and obtaining the requested data from a response through a timer mechanism implemented in software, firmware, hardware, or a combination thereof. The data source may include one or more databases or other servers. The fully automatic data acquisition method can be implemented via an internal network and / or an external network, which may include transmitting encrypted data through the Internet. In the case where the server, database, network, etc. are configured to communicate with each other, data acquisition can be performed automatically without human intervention, but it should be noted that certain user input operations may still exist in this manner. The semi-automatic method is between the manual method and the fully automatic method. The difference between the semi-automatic method and the fully automatic method is that a trigger mechanism activated by a user replaces, for example, a timer mechanism. In this case, a request to extract data is generated only when a specific user input is received. Each time data is acquired, preferably, the captured data can be stored in a non-volatile memory. As an example, a data warehouse can be used to store raw data collected during acquisition and processed data.

[0102] The above-mentioned acquired data records may come from the same or different data sources, that is, each data record may also be the result of splicing different data records. For example, in addition to acquiring the information data record filled in by the customer when applying to the bank for a credit card (including attribute information fields such as income, education, position, and asset status), as an example, the data record acquisition device 100 may also acquire other data records of the customer in the bank, such as loan records, daily transaction data, etc. These acquired data records may be spliced ​​into a complete data record. In addition, the data record acquisition device 100 may also acquire data from other private or public sources, such as data from data providers, data from the Internet (e.g., social networking sites), data from mobile operators, data from APP operators, data from express companies, data from credit institutions, and the like.

[0103] Optionally, the data record acquisition device 100 can store and / or process the collected data with the help of a hardware cluster (such as a Hadoop cluster, a Spark cluster, etc.), for example, storage, classification, and other offline operations. In addition, the data record acquisition device 100 can also perform online stream processing on the collected data.

[0104] As an example, the data record acquisition device 100 may include a data conversion module such as a text analysis module. Accordingly, in step S100, the data record acquisition device 100 may convert unstructured data such as text into more usable structured data for further processing or reference in the future. Text-based data may include emails, documents, web pages, graphics, spreadsheets, call center logs, transaction reports, etc.

[0105] In the step after obtaining the historical data records, the feature combination device 200 iteratively combines features between at least one feature generated based on the multiple attribute information according to the search strategy to generate candidate combination features, and selects target combination features from the generated candidate combination features as combination features of machine learning samples, wherein, for each round of iteration, the feature combination device 200 pre-sorts the importance of each candidate combination feature in the candidate combination feature set, selects a part of the candidate combination features from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool, re-sorts the importance of each candidate combination feature in the candidate combination feature pool, and selects at least one candidate combination feature with higher importance from the candidate combination feature pool as the target combination feature according to the re-sorting result.

[0106] The various steps involved in the above processing will be described in detail below. First, for the first round of iteration, in step S205, the candidate combined feature generation unit 210 generates at least one feature based on the attribute information of the historical data record. Here, the candidate combined feature generation unit 210 can use any appropriate feature processing method to obtain continuous features or discrete features that are easy to combine with each other.

[0107] As an example, the at least one feature generated by the candidate combined feature generation unit 210 may be at least one continuous feature. For example, the candidate combined feature generation unit 210 may generate at least one continuous feature based on the attribute information recorded in the historical data. In this process, in particular, the candidate combined feature generation unit 210 may continuously transform the discrete features to obtain continuous features that are convenient for combination. Here, the discrete features may be generated in any manner.

[0108] As another example, the at least one feature generated by the candidate combined feature generation unit 210 may be at least one discrete feature. For example, the candidate combined feature generation unit 210 may generate at least one discrete feature and / or at least one continuous feature based on the attribute information recorded in the historical data, and convert the generated continuous feature into a discrete feature.

[0109] Specifically, corresponding continuous features can be generated for at least a portion of the attribute information of the historical data record. According to an exemplary embodiment of the present invention, each continuous feature needs to be converted into a discrete feature when combined with other features. In addition, the candidate combined feature generation unit 210 can also generate discrete features, for example, directly using a certain discrete value attribute information in the historical data record as a discrete feature, or obtaining a discrete feature by feature processing the attribute information. As an optional method, the above features can also be generated by other feature generation devices (not shown).

[0110] Here, the continuous features may be discretized in any appropriate manner. Preferably, the candidate combined feature generation unit 210 may perform at least one binning operation for each continuous feature to generate a discrete feature composed of at least one binning feature, wherein each binning operation corresponds to a binning feature. The discrete features composed of the above binning features may replace the original continuous features and participate in the automatic combination between discrete features.

[0111] Here, the candidate combined feature generating unit 210 may perform a binning operation according to various binning methods and / or binning parameters.

[0112] Taking unsupervised equal-width binning as an example, assuming that the value interval of the continuous feature is [0,100], and the corresponding binning parameter (i.e., width) is 50, then 2 boxes can be divided. In this case, the continuous feature with a value of 61.5 corresponds to the second box. If the labels of these two boxes are 0 and 1, the box corresponding to the continuous feature is labeled 1. Alternatively, assuming that the bin width is 10, 10 boxes can be divided. In this case, the continuous feature with a value of 61.5 corresponds to the 7th box. If the labels of these ten boxes are 0 to 9, the box corresponding to the continuous feature is labeled 6. Alternatively, assuming that the bin width is 2, 50 boxes can be divided. In this case, the continuous feature with a value of 61.5 corresponds to the 31st box. If the labels of these fifty boxes are 0 to 49, the box corresponding to the continuous feature is labeled 30.

[0113] After the continuous feature is mapped to multiple boxes, the corresponding feature value can be any custom value. Here, the binning feature can indicate which box the continuous feature is divided into according to the corresponding binning operation. That is, the binning operation is performed to generate a multi-dimensional binning feature corresponding to each continuous feature, where, as an example, each dimension can indicate whether the corresponding continuous feature is divided into the corresponding box. For example, "1" indicates that the continuous feature is divided into the corresponding box, and "0" indicates that the continuous feature is not divided into the corresponding box. Accordingly, in the above example, assuming that 10 boxes are divided, the binning feature can be a feature of 10 dimensions, and the binning feature corresponding to the continuous feature with a value of 61.5 can be expressed as [0,0,0,0,0,0,1,0,0,0].

[0114] The above shows an example of obtaining discrete features by performing binning operations on continuous features. It should be noted that according to an exemplary embodiment of the present invention, binning features that can be used as continuous features can also be obtained by setting the values ​​of relevant dimensions in the binning features. Specifically, in the multi-dimensional binning features obtained by performing binning operations on continuous features, each dimension can indicate the feature value of the corresponding continuous feature divided into the corresponding box. Accordingly, in the above example, the binning feature corresponding to the continuous feature with a value of 61.5 can be expressed as [0, 0, 0, 0, 0, 61.5, 0, 0, 0]; or, each dimension indicates the average value of the feature values ​​of all continuous features divided into the corresponding box; or, each dimension indicates the middle value of the feature values ​​of all continuous features divided into the corresponding box; or, each dimension indicates the boundary value of the feature values ​​of all continuous features divided into the corresponding box, where the boundary value can be an upper boundary value or a lower boundary value. In addition, the values ​​of the binning features can also be normalized to facilitate the execution of operations. Assume that the j-th value of the i-th continuous feature that performs the binning operation is , its binning feature can be expressed as (BinID, ) The label of the box to which the continuous feature is assigned. The value range of this label is 0, 1, ..., B-1, where B is the total number of boxes. for The normalized values ​​of the above features (BinID, ) indicates that the feature value of the dimension corresponding to the box labeled BinID in the bin feature is , the feature values ​​of the remaining dimensions are 0.

[0115] in, It can be expressed as follows:

[0116] ,

[0117] in, is the maximum value of the i-th continuous feature, is the minimum value of the i-th continuous feature, and

[0118] ,in, The floor operator symbol.

[0119] Taking unsupervised equal-width binning as an example, assuming that the value interval of the continuous feature is [0,100]. When the bin width is 50, according to the above calculation formula, the continuous feature with a value of 61.5 can correspond to the bin feature (1,0.23). When the bin width is 10, according to the above calculation formula, the continuous feature with a value of 61.5 can correspond to the bin feature (6,0.15).

[0120] Here, in order to obtain the above features (BinID, ), according to the above calculation formula, for each Values ​​for BinID and Alternatively, a mapping table of value ranges for each BinID may be generated in advance, and the BinID corresponding to the continuous feature may be obtained by searching the data table.

[0121] In addition, as an example, before performing the binning operation, the noise in the data record can be reduced by removing possible outliers in the data sample. In this way, the effectiveness of machine learning using binning features can be further improved.

[0122] Specifically, an additional outlier box can be set so that the continuous features with outlier values ​​are divided into the outlier box. For example, for a continuous feature with a value interval of [0,1000], a certain number of samples can be selected for pre-binning. For example, equal-width binning is first performed according to a bin width of 10, and then the number of samples in each box is recorded. For boxes with a small number of samples (for example, less than a threshold), they can be merged into at least one outlier box. As an example, if the number of samples in the boxes at both ends is small, the boxes with fewer samples can be merged into outlier boxes, and the remaining boxes can be retained. Assuming that the number of samples in boxes 0-10 is small, boxes 0-10 can be merged into outlier boxes, so that the continuous features with values ​​​​of [0,100] are uniformly divided into outlier boxes.

[0123] According to an exemplary embodiment of the present invention, the at least one binning operation may be a binning operation with the same binning method but different binning parameters; or, the at least one binning operation may be a binning operation with different binning methods.

[0124] The binning methods here include various binning methods under supervised binning and / or unsupervised binning. For example, supervised binning includes minimum entropy binning, minimum description length binning, etc., while unsupervised binning includes equal width binning, equal depth binning, binning based on k-means clustering, etc.

[0125] As an example, at least one binning operation may correspond to equal-width binning operations of different widths. That is, the binning method used is the same but the granularity of the division is different, which enables the generated binning features to better characterize the regularity of the original data records, thereby being more conducive to the training and prediction of the machine learning model. In particular, the different widths used by at least one binning operation may numerically constitute a geometric progression, for example, the binning operation may perform equal-width binning according to the widths of values ​​2, 4, 8, 16, etc. Alternatively, the different widths used by at least one binning operation may numerically constitute an arithmetic progression, for example, the binning operation may perform equal-width binning according to the widths of values ​​2, 4, 6, 8, etc.

[0126] As another example, at least one binning operation may correspond to equal-depth binning operations of different depths. That is, the binning operation uses the same binning method but different granularity of division, which enables the generated binning features to better characterize the regularity of the original data records, thereby being more conducive to the training and prediction of the machine learning model. In particular, the different depths used in the binning operation may constitute a geometric progression in terms of values. For example, the binning operation may perform equal-depth binning according to depths of values ​​10, 100, 1000, 10000, etc. Alternatively, the different depths used in the binning operation may constitute an arithmetic progression in terms of values. For example, the binning operation may perform equal-depth binning according to depths of values ​​10, 20, 30, 40, etc.

[0127] For each continuous feature, after obtaining at least one corresponding binning feature by performing a binning operation, a discrete feature corresponding to the continuous feature can be obtained by taking each binning feature as a component element. The discrete feature can be regarded as a set of binning features.

[0128] As described above, according to an exemplary embodiment of the present invention, it is necessary to perform at least one binning operation on the continuous features. Here, the at least one binning operation can be determined by any appropriate means, for example, it can be determined by the experience of a technician or business person, or it can be automatically determined by technical means. As an example, the specific binning operation method can be effectively determined based on the importance of the binning features.

[0129] Accordingly, the candidate combined feature generation unit 210 may select the at least one binning operation from a predetermined number of binning operations, so that the importance of the binning feature corresponding to the selected binning operation is not less than the importance of the binning feature corresponding to the unselected binning operation. In this way, the effect of machine learning can be ensured while reducing the size of the combined feature space.

[0130] Specifically, the predetermined number of binning operations may indicate a plurality of binning operations that differ in binning methods and / or binning parameters. Here, by performing each binning operation, a corresponding binning feature may be obtained, and accordingly, the candidate combined feature generation unit 210 may determine the importance of these binning features, and then select the binning operation corresponding to the more important binning feature as at least one binning operation to be performed by the candidate combined feature generation unit 210.

[0131] Here, the candidate combined feature generation unit 210 may adopt any appropriate method to automatically determine the importance of the binning features.

[0132] For example, the candidate combination feature generation unit 210 can obtain a binning single feature machine learning model for each binning feature among the binning features corresponding to the predetermined number of binning operations, determine the importance of each binning feature based on the effect of each binning single feature machine learning model, and select at least one binning operation based on the importance of each binning feature, wherein the binning single feature machine learning model corresponds to each binning feature.

[0133] As an example, assume that for a continuous feature F, there are a predetermined number M (M is an integer greater than 1) of binning operations, corresponding to M binning features ,in, Accordingly, the candidate combination feature generation unit 210 can use at least a portion of the historical data records to construct M bin-by-bin feature machine learning models (where each bin-by-bin feature machine learning model is based on a corresponding single bin feature To predict the machine learning problem), then measure the effects of these M binning single-feature machine learning models on the same test data set (for example, AUC (area under the ROC (Receiver Operating Characteristic) curve, MAE (mean absolute error), etc.), and determine at least one binning operation to be performed based on the ranking of the effects.

[0134] For another example, the candidate combined feature generation unit 210 may obtain a binning overall machine learning model for each binning feature among the binning features corresponding to the predetermined number of binning operations, determine the importance of each binning feature based on the effect of each binning overall machine learning model, and select the at least one binning operation based on the importance of each binning feature, wherein the binning overall machine learning model corresponds to the binning basic feature subset and each binning feature. As an example, the binning overall machine learning model here can be a logarithmic probability regression (LR) model; accordingly, the sample of the binning overall machine learning model consists of the binning basic feature subset and each binning feature.

[0135] As an example, assume that for a continuous feature F, there are a predetermined number M of binning operations, corresponding to M binning features Accordingly, the candidate combination feature generation unit 210 can use at least a portion of the historical data records to construct M bin-by-bin overall machine learning models (wherein the sample features of each bin-by-bin overall machine learning model include a fixed bin basic feature subset and a corresponding bin feature ), and then measure the effect of the M binned overall machine learning model on the same test data set (for example, AUC, MAE, etc.), and determine at least one binning operation to be performed based on the ranking of the effects.

[0136] For another example, the candidate combined feature generation unit 210 may obtain a binning composite machine learning model for each binning feature among the binning features corresponding to the predetermined number of binning operations, determine the importance of each binning feature based on the effects of each binning composite machine learning model, and select at least one binning operation based on the importance of each binning feature, wherein the binning composite machine learning model includes a binning basic sub-model and a binning additional sub-model based on a boosting framework (e.g., a gradient boosting framework), wherein the binning basic sub-model corresponds to a binning basic feature subset, and the binning additional sub-model corresponds to each binning feature.

[0137] As an example, assume that for a continuous feature F, there are a predetermined number M of binning operations, corresponding to M binning features Accordingly, the candidate combined feature generation unit 210 can use at least a portion of the historical data records to construct M binning composite machine learning models (where each binning composite machine learning model is based on a fixed binning basic feature subset and a corresponding binning feature , predict the machine learning problem according to the lifting framework), then measure the effect of the M binning composite machine learning models on the same test data set (for example, AUC, MAE, etc.), and determine at least one binning operation to be performed based on the ranking of the effect. Preferably, in order to further improve the operation efficiency and reduce resource consumption, the candidate combination feature generation unit 210 can be used to fix the basic sub-model of binning, respectively for each binning feature Train bin-added sub-models to build composite machine learning models for each bin.

[0138] According to an exemplary embodiment of the present invention, the binning basic feature subset can be fixedly applied to the binning basic sub-models in all relevant binning overall machine learning models or binning composite machine learning models. Here, for the first round of iterations, the binning basic feature subset can be empty; or, any feature generated based on the attribute information of the historical data record can be used as the binning basic feature, for example, a part of the attribute information or all the attribute information of the historical data record can be directly used as the binning basic feature. In addition, as an example, an actual machine learning problem can be considered, and relatively important or basic features can be determined as binning basic features based on estimation or according to the designation of business personnel.

[0139] After the unit features for generating the combined features are generated as above, in step S210, the candidate combined features generating unit 210 generates the candidate combined features for each round of iteration according to the search strategy. Here, in the case of combining continuous features, the combination can be performed by means such as arithmetic operations to serve as the candidate combined features according to the exemplary embodiment of the present invention; in the case of combining discrete features, the combination can be performed by means such as Cartesian products to serve as the candidate combined features according to the exemplary embodiment of the present invention. However, it should be noted that the combination method is not limited thereto, and any method that can combine two or more continuous or discrete features with each other can be applied to the exemplary embodiment of the present invention.

[0140] Here, for the first round of iterations, the candidate combined feature generating unit 210 may directly use each feature generated in step S205 as a candidate combined feature.

[0141] Next, in step S220, the pre-sorting unit 220 pre-sorts the importance of each candidate combination feature in the candidate combination feature set for the first round of iteration. Here, the candidate combination feature set may include candidate combination features that need to be pre-sorted in the current round of iteration, and these candidate combination features may be at least a part of the candidate combination features that have been generated. As an example, the candidate combination feature set in the first round of iteration may include all features generated in step S205.

[0142] Here, the pre-sorting unit 220 may use any means of determining feature importance to measure the importance of each candidate combination feature in the candidate combination feature set.

[0143] For example, the pre-sorting unit 220 can obtain a pre-sorting single feature machine learning model for each candidate combination feature in the candidate combination feature set, and determine the importance of each candidate combination feature based on the effect of each pre-sorting single feature machine learning model, wherein the pre-sorting single feature machine learning model corresponds to each candidate combination feature.

[0144] As an example, assume that the candidate combination feature set includes N (N is an integer greater than 1) candidate combination features ,in, Accordingly, the pre-sorting unit 220 may use at least a portion of the historical data records to construct N pre-sorted single-feature machine learning models (where each pre-sorted single-feature machine learning model is based on a corresponding single candidate combination feature To make predictions for machine learning problems), then measure the effects of these N pre-sorted single-feature machine learning models on the same test dataset (for example, AUC, MAE, etc.), and determine the order of importance of each candidate combination feature in the candidate combination feature set based on the ranking of the effects.

[0145] For another example, the pre-sorting unit 220 may obtain a pre-sorted overall machine learning model for each candidate combination feature in the candidate combination feature set, and determine the importance of each candidate combination feature based on the effect of each pre-sorted overall machine learning model, wherein the pre-sorted overall machine learning model corresponds to the pre-sorted basic feature subset and each candidate combination feature. As an example, the pre-sorted overall machine learning model here can be an LR model; accordingly, the sample of the pre-sorted overall machine learning model consists of the pre-sorted basic feature subset and each candidate combination feature.

[0146] As an example, assume that the candidate combination feature set includes N candidate combination features Accordingly, the pre-sorting unit 220 may use at least a portion of the historical data records to construct N pre-sorted overall machine learning models (wherein the sample features of each pre-sorted overall machine learning model include a fixed pre-sorted basic feature subset and a corresponding candidate combination feature ), and then measure the effects of these N pre-ranked overall machine learning models on the same test data set (for example, AUC, MAE, etc.), and determine the importance order of each candidate combination feature in the candidate combination feature set based on the ranking of the effects.

[0147] For another example, the pre-sorting unit 220 may obtain a pre-sorting compound machine learning model for each candidate combination feature in the candidate combination feature set, and determine the importance of each candidate combination feature based on the effect of each pre-sorting compound machine learning model, wherein the pre-sorting compound machine learning model includes a pre-sorting basic sub-model and a pre-sorting additional sub-model based on a boosting framework (e.g., a gradient boosting framework), wherein the pre-sorting basic sub-model corresponds to a pre-sorting basic feature subset, and the pre-sorting additional sub-model corresponds to each of the candidate combination features.

[0148] As an example, assume that the candidate combination feature set includes N candidate combination features Accordingly, the pre-sorting unit 220 may use at least a portion of the historical data records to construct N pre-sorted composite machine learning models (where each pre-sorted composite machine learning model is based on a fixed pre-sorted basic feature subset and a corresponding candidate combination feature , predict the machine learning problem according to the lifting framework), then measure the effect of the N pre-sorted composite machine learning models on the same test data set (for example, AUC, MAE, etc.), and determine the importance order of each candidate combination feature in the candidate combination feature set based on the ranking of the effect. Preferably, in order to further improve the computing efficiency and reduce resource consumption, the pre-sorting unit 220 can be fixed in the case of pre-sorting basic sub-models, respectively for each candidate combination feature Train pre-sort additional sub-models to build individual pre-sort composite machine learning models.

[0149] According to an exemplary embodiment of the present invention, the pre-sorted basic feature subset can be fixedly applied to all relevant pre-sorted overall machine learning models or pre-sorted basic sub-models in the pre-sorted composite machine learning model. Here, for the first round of iterations, the pre-sorted basic feature subset can be empty; or, any feature generated based on the attribute information of the historical data record can be used as the pre-sorted basic feature. For example, part of the attribute information or all of the attribute information of the historical data record can be directly used as the pre-sorted basic feature. In addition, as an example, an actual machine learning problem can be considered, and relatively important or basic features can be determined as pre-sorted basic features based on estimation or according to the designation of business personnel.

[0150] After determining the importance order of each candidate combination feature in the candidate combination feature set by pre-sorting, the pre-sorting unit 220 may screen out at least a portion of the candidate combination features based on the sorting result to form a candidate combination feature pool. As described above, important candidate combination features that are consistent in predictive effect may be preferentially screened to form a candidate combination feature pool, so as to effectively determine the combination features that may eventually constitute a machine learning sample. For example, the pre-sorting unit 220 may screen out candidate combination features with higher importance from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool.

[0151] Assuming that the candidate combination feature set of the first round of iteration includes 1000 features as candidate combination features, the pre-sorting unit 220 can select the 10 most important features in the pre-sorting results to form a candidate combination feature pool.

[0152] Next, in step S230, the re-ranking unit 230 re-ranks the importance of each candidate combination feature in the candidate combination feature pool, and selects at least one candidate combination feature with higher importance from the candidate combination feature pool as the target combination feature according to the re-ranking result.

[0153] Here, the re-ranking unit 230 may use any means of determining feature importance to measure the importance of each candidate combination feature in the candidate combination feature pool.

[0154] For example, the re-ranking unit 230 can obtain a re-ranking single feature machine learning model for each candidate combination feature in the candidate combination feature pool, and determine the importance of each candidate combination feature based on the effect of each re-ranking single feature machine learning model, wherein the re-ranking single feature machine learning model corresponds to each candidate combination feature.

[0155] As an example, assume that the candidate combination feature pool includes 10 candidate combination features. Accordingly, the re-ranking unit 230 can use at least a portion of the historical data records to construct 10 re-ranked single feature machine learning models (wherein each re-ranked single feature machine learning model is based on a corresponding single candidate combination feature to make predictions for the machine learning problem), and then measure the effects of the 10 re-ranked single feature machine learning models on the same test data set (e.g., AUC, MAE, etc.), and determine the order of importance of each candidate combination feature in the candidate combination feature pool based on the ranking of the effects.

[0156] For another example, the re-ranking unit 230 may obtain a re-ranking overall machine learning model for each candidate combination feature in the candidate combination feature pool, and determine the importance of each candidate combination feature based on the effect of each re-ranking overall machine learning model, wherein the re-ranking composite machine learning model corresponds to the re-ranking basic feature subset and each candidate combination feature. As an example, the re-ranking overall machine learning model here can be an LR model; accordingly, the sample of the re-ranking overall machine learning model consists of the re-ranking basic feature subset and each candidate combination feature.

[0157] As an example, assuming that the candidate combination feature pool includes 10 candidate combination features, accordingly, the re-ranking unit 230 can use at least a portion of the historical data records to construct 10 re-ranking overall machine learning models (wherein the sample features of each re-ranking overall machine learning model include a fixed re-ranking basic feature subset and corresponding candidate combination features), and then measure the effects of these 10 re-ranking overall machine learning models on the same test data set (for example, AUC, MAE, etc.), and determine the order of importance of each candidate combination feature in the candidate combination feature pool based on the ranking of the effects.

[0158] For another example, the re-ranking unit 230 may obtain a re-ranking compound machine learning model for each candidate combination feature in the candidate combination feature pool, and determine the importance of each candidate combination feature based on the effect of each re-ranking compound machine learning model, wherein the re-ranking compound machine learning model includes a re-ranking basic sub-model and a re-ranking additional sub-model based on a boosting framework (e.g., a gradient boosting framework), wherein the re-ranking basic sub-model corresponds to a re-ranking basic feature subset, and the re-ranking additional sub-model corresponds to each of the candidate combination features.

[0159] As an example, assuming that the candidate combination feature pool includes 10 candidate combination features, accordingly, the re-ranking unit 230 can use at least a portion of the historical data records to construct 10 re-ranking compound machine learning models (wherein each re-ranking compound machine learning model is based on a fixed re-ranking basic feature subset and the corresponding candidate combination features, and predicts the machine learning problem according to the lifting framework), and then measure the effects of these 10 re-ranking compound machine learning models on the same test data set (for example, AUC, MAE, etc.), and determine the importance order of each candidate combination feature in the candidate combination feature pool based on the ranking of the effects. Preferably, in order to further improve computing efficiency and reduce resource consumption, the re-ranking unit 230 can construct each re-ranking compound machine learning model by training a re-ranking additional sub-model for each candidate combination feature separately under the condition of a fixed re-ranking basic sub-model.

[0160] According to an exemplary embodiment of the present invention, the re-ranking basic feature subset can be fixedly applied to all relevant re-ranking overall machine learning models or re-ranking basic sub-models in the composite machine learning model. Here, for the first round of iterations, the re-ranking basic feature subset can be empty; or, any feature generated based on the attribute information of the historical data record can be used as the re-ranking basic feature. For example, part of the attribute information or all of the attribute information of the historical data record can be directly used as the re-ranking basic feature. In addition, as an example, an actual machine learning problem can be considered, and relatively important or basic features can be determined as re-ranking basic features based on estimation or according to the designation of business personnel.

[0161] After determining the importance order of each candidate combination feature in the candidate combination feature pool by re-ranking, the re-ranking unit 230 may select at least one relatively important candidate combination feature from the candidate combination feature pool based on the ranking result as the target combination feature. Assuming that the candidate combination feature pool of the first round of iteration includes 10 features as candidate combination features, the re-ranking unit 230 may select the most important feature in the re-ranking result as the target combination feature.

[0162] According to exemplary embodiments of the present invention, computing resources may be further effectively controlled by sharing the same model part.

[0163] As an example, when the candidate combined feature generation unit 210, the pre-sorting unit 220 and / or the re-sorting unit 230 respectively sort the importance of relevant features based on their respective lifting framework composite machine learning models, for example, a common basic sub-model part can be trained based on relatively more historical data records (e.g., the full amount of historical data records), and this part can be used as a fixed model part and used as a binning basic sub-model in the binning composite machine learning model, a pre-sorting basic sub-model in the pre-sorting composite machine learning model, and / or a re-sorting basic sub-model in the re-sorting composite machine learning model. Further, in the case of sharing the basic sub-model, the binning additional sub-model, the pre-sorting additional sub-model and / or the re-sorting additional sub-model corresponding to each feature whose importance is to be determined can be trained in parallel, so that multiple models can be trained simultaneously through only one reading operation of the historical data record.

[0164] In addition, according to an exemplary embodiment of the present invention, the effect of the combined features may be further ensured by controlling the sample training set size, sample training order and / or sample training set quality of the relevant model part.

[0165] As an example, the pre-sorting unit 220 may train a pre-sorting single-feature machine learning model based on relatively fewer historical data records, while the re-sorting unit 230 may train a re-sorting single-feature machine learning model based on relatively more historical data records; or, the pre-sorting unit 220 may train a pre-sorting overall machine learning model based on relatively fewer historical data records, while the re-sorting unit 230 may train a re-sorting overall machine learning model based on relatively more historical data records; or, the pre-sorting unit 220 may train a pre-sorting additional sub-model based on relatively fewer historical data records, while the re-sorting unit 230 may train a re-sorting additional sub-model based on relatively more historical data records. Here, the historical data records used by the re-sorting unit 230 may include at least a portion of the historical data records used by the pre-sorting unit 220, or the historical data records used by the re-sorting unit 230 may not include any historical data records used by the sorting unit 220. In addition to the difference in the size of the sample training set, the pre-sorting unit 220 and the re-sorting unit 230 may use the same set of historical data records, but the order of the two during training is different. It can be seen that the feature combination device 200 can perform pre-sorting based on the first number of historical data records, and re-sorting based on the second number of historical data records, and the second number is not less than the first number. In addition, the pre-sorting unit 220 can also use a sample training set with different quality from the re-sorting unit 230. For example, the pre-sorting unit 220 can use a sample training set with lower quality, while the re-sorting unit 230 can use a sample training set with higher quality. In this way, even if the re-sorting unit 230 uses a smaller sample training set, the effect of the re-sorting related model can be ensured.

[0166] However, it should be noted that the exemplary embodiments of the present invention are not limited thereto, but any manner may be used to construct respective basic sub-models, and any appropriate training data set may be used.

[0167] After the target combination features are screened out in the first round of iteration, in step S235, it is determined whether the conditions for terminating the iteration are met. Here, any conditions required to terminate the iteration can be preset, such as the number of target combination features obtained, the number of iterations performed, etc. When the iteration termination conditions are met, the generation process of the combination features can be terminated; otherwise, the method can return to step S205 or step S210 to perform the next round of iteration.

[0168] As an example, assuming that a binning operation is involved in the process of generating unit features, for example, at least one binning operation performed when converting a continuous feature into a discrete feature (or a binned continuous feature) is selected from a predetermined number of binning operations for each round of iteration, the method needs to return to step S205 so that the candidate combined feature generation unit 210 reselects the at least one binning operation for the second round of iteration.

[0169] Here, the candidate combination feature generation unit 210 may convert the continuous features into corresponding bin features again in various ways similar to the first round, so as to perform subsequent feature combination or feature continuous transformation.

[0170] In particular, in the case where a binning operation is selected for each round of iteration, assuming that the candidate combined feature generation unit 210 uses a binning overall machine learning model or a binning composite machine learning model to measure the importance of the binning features, the target combined features selected in each round of iteration can be added to the binning basic feature subset as a new feature, that is, the binning basic feature subset can include the target combined features selected before the current round of iteration. Here, the binning basic feature subset based on the binning basic sub-model can be updated with the iteration of generating the target combined features. Specifically, the target combined features selected in the first round of iteration can be added to the binning basic feature subset of the first round of iteration to form the binning basic feature subset of the second round of iteration.

[0171] After the candidate combined feature generation unit 210 retrieves the binning feature converted from the continuous feature as described above, the method may proceed to step S210. In addition, as an example, if no binning operation is involved in the process of generating the unit feature, for example, when at least one binning operation performed to convert the continuous feature into a discrete feature is selected from a predetermined number of binning operations at one time for all rounds of iteration, the method may directly start from step S210 to perform the next round of iteration without performing step S205 again.

[0172] Specifically, in step S210, the candidate combination feature generation unit 210 may generate candidate combination features for the second round of iteration according to the search strategy. As an example, in the first round of iteration, the first-order features are selected as the target combination features, and accordingly, in the second round of iteration, the candidate combination feature generation unit 210 may obtain second-order or higher-order candidate combination features by performing Cartesian product combination or arithmetic operation combination on the features.

[0173] For example, the following will describe the combination of the candidate combination feature generation unit 210 Figure 6 An example of iteratively generating combined features by using a search tree as shown. The search tree may be based on a heuristic search strategy such as a beam search, where a layer of the search tree may correspond to a feature combination of a certain order.

[0174] Reference Figure 6 For the convenience of description, it is assumed that the unit features that can be combined include feature A, feature B, feature C, feature D and feature E. As an example, feature A, feature B, feature C can be discrete features formed by the discrete value attribute information of the historical data record itself, and feature D and feature E can be discrete features converted from continuous features through corresponding binning operations in each round of iteration. However, it should be understood that the unit features combined according to the exemplary embodiment of the present invention can be either discrete features or continuous features. Therefore, the following description takes discrete features as an example in order to explain the exemplary embodiment of the present invention, and is by no means a limitation. Those skilled in the art can fully understand how to combine and sort continuous features in a similar manner.

[0175] According to the search strategy, the importance of the features under re-ranking can be used as an indicator to sort the nodes of the search tree, and then select a part of the nodes to continue to expand in the next layer. For example, assuming that in the first round of iterations, the two nodes of feature B and feature E as first-order features are finally selected as target combination features, then in the second round of iterations, the candidate combination feature generation unit 210 can generate features BA, feature BC, feature BD, feature BE, feature EA, feature EB, feature EC, and feature ED as second-order combination features based on feature B and feature E. Here, as an example, only the combination features that change in order (for example, feature BE and feature EB) can be regarded as the same features, so that only one of them is retained through deduplication processing. As described above, the candidate combination feature generation unit 210 can generate candidate combination features for the next round of iteration by combining the target combination feature selected in the current round of iteration with at least one feature generated based on multiple attribute information recorded based on historical data. Accordingly, assuming that feature BC and feature EA are selected in the second round of iterations, it can be as follows Figure 6 As shown, the iteration continues in the above manner until a specific cutoff condition is met, such as an order limit, etc. Here, the nodes selected in each layer (shown by solid lines) can be used as target combination features for subsequent processing, for example, as the final sample features or for further verification, while the remaining features (shown by dotted lines) are pruned.

[0176] The above shows an example in which the candidate combination feature generating unit 210 generates candidate combination features step by step. In this example, the candidate combination features generated in the second round of iterations include feature BA, feature BC, feature BD, feature BE, feature EA, feature EB, feature EC, and feature ED.

[0177] Accordingly, in step S220, the pre-sorting unit 220 pre-sorts the importance of each candidate combination feature in the candidate combination feature set for the second round of iteration. Here, the candidate combination feature set may include candidate combination features that need to be pre-sorted in the current round of iteration. As an example, the candidate combination feature set may include candidate combination features generated in the current round of iteration, for example, feature BA, feature BC, feature BD, feature BE, feature EA, feature EB, feature EC, feature ED generated in the second round of iteration; as another example, the candidate combination feature set may not only include candidate combination features generated in the current round of iteration, but may also further include candidate combination features generated in the previous round of iteration that are not selected as target combination features, for example, feature BA, feature BC, feature BD, feature BE, feature EA, feature EB, feature EC, feature ED generated in the second round of iteration together with non-target candidate features generated in the first round of iteration, that is, feature A, feature C and feature D. In this way, the candidate combination features can be more comprehensively measured under the premise of ensuring computational efficiency. It should be noted that according to an exemplary embodiment of the present invention, a portion of all candidate combination features generated in the current round and / or previous round iterations may be selected to enter the candidate combination feature set, rather than necessarily using all currently existing candidate combination features.

[0178] Here, the pre-ranking unit 220 may rank the importance of each candidate combination feature in the candidate combination feature set in various ways similar to the first round.

[0179] In particular, when the pre-sorting unit 220 uses a pre-sorting overall machine learning model or a pre-sorting composite machine learning model to measure the importance of candidate combination features, the target combination features selected in each round of iteration can be added to the pre-sorted basic feature subset as new continuous features or discrete features, that is, the pre-sorted basic feature subset can include the target combination features selected before the current round of iteration. Here, the pre-sorted basic feature subset based on the pre-sorted basic sub-model can be updated with the iteration of generating the target combination features. Specifically, the target combination features selected in the first round of iteration can be added to the pre-sorted basic feature subset of the first round of iteration to form the pre-sorted basic feature subset of the second round of iteration.

[0180] After the pre-sorting unit 220 obtains a new candidate combination feature pool through pre-sorting processing in step S220, in step S230, the re-sorting unit 230 re-sorts the importance of each candidate combination feature in the candidate combination feature pool, and selects at least one candidate combination feature with higher importance from the candidate combination feature pool as the target combination feature according to the re-sorting result.

[0181] Here, the re-ranking unit 230 may rank the importance of each candidate combination feature in the candidate combination feature pool in various ways similar to the first round.

[0182] In particular, when the re-ranking unit 230 uses a re-ranking overall machine learning model or a re-ranking composite machine learning model to measure the importance of candidate combination features, the target combination features selected in each round of iteration can be added to the re-ranking basic feature subset as a new continuous feature or discrete feature, that is, the re-ranking basic feature subset can include the target combination features selected before the current round of iteration. Here, the re-ranking basic feature subset based on the re-ranking basic sub-model can be updated with the iteration of generating the target combination features. Specifically, the target combination features selected in the first round of iteration can be added to the re-ranking basic feature subset of the first round of iteration to form the re-ranking basic feature subset of the second round of iteration.

[0183] Here, in addition to using Figure 6 In addition to generating candidate combination features step by step in a manner, according to an exemplary embodiment of the present invention, candidate combination features can also be generated more effectively in each round of iteration. Specifically, in step S210, the candidate combination feature generation unit 210 can generate candidate combination features for the next round of iteration by performing pairwise combinations between target combination features selected in the current round of iteration and the previous round of iteration. In this way, valuable combinations can be mined more centrally.

[0184] According to an exemplary embodiment of the present invention, when the iteration termination condition is met, the sum of the target combination features selected in each round of iteration can be used as the combination feature set of the machine learning sample. In particular, when the binned basic feature subsets, pre-sorted basic feature subsets and / or re-sorted basic feature subsets are updated with the target combination features selected in each round, the combination features in the above subsets in the last round of iteration can be used as the combination feature set of the machine learning sample.

[0185] Figure 7 A flow chart of a method for training a machine learning model according to an exemplary embodiment of the present invention is shown. Figure 7 In the method shown, in addition to the above-mentioned step S100, step S205, step S210, step S220, step S230 and step S235, the method also includes step S300 and step S400.

[0186] Specifically, in Figure 7 In the method shown, step S100, step S205, step S210, step S220, step S230 and step S235 can be Figure 5 The corresponding steps shown are similar and the details will not be repeated here.

[0187] In addition, in step S300, the machine learning sample generating device 300 may generate a machine learning training sample including at least a part of the generated combined features. In the case of supervised learning, the machine learning training sample may include two parts: features and labels.

[0188] In step S400, the machine learning model can be trained by the machine learning model training device 400 based on the machine learning training sample. Here, the machine learning model training device 400 can use an appropriate machine learning algorithm to learn an appropriate machine learning model from the machine learning training sample. As an example, the appropriate machine learning algorithm can be the same as or different from the machine learning algorithm based on the binning single feature machine learning model, the pre-sorting single feature machine learning model, the re-sorting single feature machine learning model, the binning overall machine learning model, the pre-sorting overall machine learning model, the re-sorting overall machine learning model, the binning composite machine learning model (or the binning basic sub-model or the binning additional sub-model), the pre-sorting composite machine learning model (or the pre-sorting basic sub-model or the pre-sorting additional sub-model), or the re-sorting composite machine learning model (or the re-sorting basic sub-model or the re-sorting additional sub-model).

[0189] After the machine learning model is trained, the trained machine learning model can be used to make predictions.

[0190] According to an exemplary embodiment of the present invention, in order to further enhance the effectiveness of the target combination features, the target combination features may be further verified, and only the verified target combination features may be used as combination features of machine learning samples. Figure 8 A flowchart of a method for generating combined features of machine learning samples according to another exemplary embodiment of the present invention is shown. In this example, for each round of iteration, it can also be checked whether the selected target combined features are suitable as combined features of machine learning samples.

[0191] Reference Figure 8 , step S100, step S205, step S210, step S220 and step S230 and Figure 5 The corresponding steps shown are similar and the details will not be repeated here.

[0192] In addition, after obtaining the target combination features in step S230, the method proceeds to step S240. In step S240, the verification unit 240 can verify whether the target combination features obtained in step S230 are suitable as combination features of machine learning samples.

[0193] As an example, the verification unit 240 may use the change in the effect of the machine learning model after the introduction of the selected target combination features for verification, wherein the sample features of the machine learning model include the target combination features that have passed the test. Specifically, the verification unit 240 may construct a machine learning model based on the target combination features that have passed the test. For example, the sample of the machine learning model may include at least those target combination features that have passed the test before, and may further include other features. Here, the machine learning model may be based on a similar feature subset to the binning basic feature sub-model, the pre-sorting basic feature sub-model and / or the re-sorting basic feature sub-model, and may be trained based on more historical data records. Optionally, the machine learning model is not based on a boosting framework, thereby more accurately verifying whether the selected target combination features are truly helpful in performing predictions for machine learning problems.

[0194] Here, the verification unit 240 can determine whether the change in the model effect meets the requirements (for example, the effect enhancement meets expectations or the effect weakening is acceptable) after the above-mentioned machine learning model introduces the newly selected target combination features in this round of iteration. Specifically, the verification unit 240 can determine whether the model effect is enhanced (for example, whether the enhancement of the model effect reaches a predetermined enhancement degree); or, the verification unit 240 can determine whether the model effect is only slightly weakened (for example, whether the weakening of the model effect is lower than a predetermined weakening degree, in which case the weakening of the model effect can be ignored). When the effect change of the model meets the requirements, it can be determined that the selected target combination features are suitable as combination features of machine learning samples.

[0195] Accordingly, when the test result is that the selected target combination features are suitable as the combination features of machine learning samples, the selected target combination features can be used as the combination features of machine learning samples, and the next round of iterations can be performed; when the test result is that the selected target combination features are not suitable as the combination features of machine learning samples, another part of the candidate combination features can be screened out from the candidate combination feature set according to the pre-sorting result to form a new candidate combination feature pool.

[0196] As an example, the verification unit 240 may determine in step S240 whether the machine learning model performs better on the same data test set after the newly selected target combination feature is introduced. If it is determined that the newly selected target combination feature brings a better prediction effect, it means that the corresponding target combination feature can be used as a combination feature of the machine learning sample, and the method proceeds to step S235 to determine whether the iteration termination condition is met.

[0197] If the iteration termination condition is met, the method ends, and all target combination features that have passed the test can be used as the combination features finally adopted by the machine learning sample. Otherwise, the method returns to step S205 (or step S210) to perform the next round of iteration.

[0198] If it is determined that the newly selected target combination feature does not bring a better prediction effect, the method proceeds to step S245 to determine whether to continue screening other target combination features in this round of iteration. Here, if the screening termination condition is met (for example, a predetermined number of target combination features have been tested in this round of iteration, all target combination features have been tested in this round of iteration, etc.), the method executes step S235. Otherwise, the method may return to step S220, in which the pre-sorting unit 220 reconstructs the candidate combination feature pool so that the target combination feature is re-screened by the re-sorting unit 230. Here, the pre-sorting unit 220 may re-construct a new candidate combination feature pool according to the pre-sorting result. For example, if the first to tenth most important features have been screened before, the eleventh to twentieth most important features may continue to be screened, and so on.

[0199] It should be noted that the above inspection steps can also be applied to Figure 7 The method shown will not be described in detail here.

[0200] Figures 1 to 4 The devices and units shown and their units may be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these devices or units may correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. In addition, one or more functions implemented by these devices or units may also be uniformly executed by components in physical entity devices (e.g., processors, clients, or servers, etc.).

[0201] Reference above Figures 1 to 8A method and system for generating a combined feature of a machine learning sample according to an exemplary embodiment of the present invention and a corresponding machine learning model training method and system are described. It should be understood that the above method can be implemented by a program recorded on a computer readable medium. For example, according to an exemplary embodiment of the present invention, a computer readable medium for generating a combined feature of a machine learning sample can be provided, wherein a computer program for performing the following method steps is recorded on the computer readable medium: (A) obtaining a historical data record, wherein the historical data record includes a plurality of attribute information; and (B) according to a search strategy, iteratively performing feature combination between at least one feature generated based on the plurality of attribute information to generate a candidate combined feature, and selecting a target combined feature from the generated candidate combined features as the combined feature of the machine learning sample, wherein for each round of iteration, the importance of each candidate combined feature in the candidate combined feature set is pre-sorted; according to the pre-sorting result, a part of the candidate combined features are screened from the candidate combined feature set to form a candidate combined feature pool; the importance of each candidate combined feature in the candidate combined feature pool is re-sorted; according to the re-sorting result, at least one candidate combined feature with higher importance is selected from the candidate combined feature pool as the target combined feature.

[0202] The computer program in the computer-readable medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the computer program can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been referred to in Figures 1 to 8 The description has been made, and will not be repeated here to avoid repetition.

[0203] It should be noted that the combined feature generation system and the machine learning model training system according to the exemplary embodiments of the present invention can completely rely on the operation of the computer program to realize the corresponding functions, that is, each device corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (for example, lib library) to realize the corresponding function.

[0204] on the other hand, Figures 1 to 4 The various devices or units shown may also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segment for performing the corresponding operation may be stored in a computer-readable medium such as a storage medium, so that the processor may perform the corresponding operation by reading and running the corresponding program code or code segment.

[0205] For example, an exemplary embodiment of the present invention can also be implemented as a computing device, which includes a storage component and a processor, wherein a set of computer-executable instructions is stored in the storage component, and when the set of computer-executable instructions is executed by the processor, a combined feature generation method or a machine learning model training method is executed.

[0206] Specifically, the computing device may be deployed in a server or client, or may be deployed on a node device in a distributed network environment. In addition, the computing device may be a PC, a tablet device, a personal digital assistant, a smart phone, a web application, or other device capable of executing the above instruction set.

[0207] Here, the computing device is not necessarily a single computing device, but may also be any device or circuit collection that can execute the above instructions (or instruction sets) individually or jointly. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0208] In the computing device, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. By way of example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0209] According to the exemplary embodiments of the present invention, some operations described in the combined feature generation method and the machine learning model training method can be implemented by software, some operations can be implemented by hardware, and in addition, these operations can also be implemented by a combination of software and hardware.

[0210] The processor may execute instructions or codes stored in one of the storage components, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0211] The storage component may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage component may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage component and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the storage component.

[0212] In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the computing device may be connected to each other via a bus and / or a network.

[0213] The operations involved in the combined feature generation method and the corresponding machine learning model training method according to the exemplary embodiment of the present invention can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logical device or operated according to non-exact boundaries.

[0214] For example, as described above, a computing device for generating combined features of machine learning samples according to an exemplary embodiment of the present invention may include a storage component and a processor, wherein a computer executable instruction set is stored in the storage component, and when the computer executable instruction set is executed by the processor, the following steps are performed: (A) obtaining historical data records, wherein the historical data records include multiple attribute information; and (B) according to a search strategy, iteratively performing feature combinations between at least one feature generated based on the multiple attribute information to generate candidate combined features, and selecting target combined features from the generated candidate combined features as the combined features of the machine learning samples, wherein for each round of iteration, the importance of each candidate combined feature in the candidate combined feature set is pre-sorted; according to the pre-sorting result, a part of the candidate combined features are screened out from the candidate combined feature set to form a candidate combined feature pool; the importance of each candidate combined feature in the candidate combined feature pool is re-sorted; and according to the re-sorting result, at least one candidate combined feature with higher importance is selected from the candidate combined feature pool as the target combined feature.

[0215] The above describes various exemplary embodiments of the present invention, it should be understood that the above description is only exemplary, not exhaustive, and the present invention is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present invention, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present invention should be based on the scope of the claims.

Claims

1. A method for generating machine learning samples, comprising: (A) obtaining a historical data record, wherein the historical data record is text data or numerical data including a plurality of attribute information, and the historical data record includes a plurality of samples related to at least one of the following items: an individual, an object, an organization, a unit, an institution, a project, or an event; (B) according to the search strategy, iteratively combining features between at least one feature generated based on the plurality of attribute information to generate candidate combination features, and selecting a target combination feature from the generated candidate combination features, wherein the target combination feature is used as a combination feature of a machine learning sample, and the search strategy is used to prune a search tree for the combination feature to control the number of candidate combination features generated in each round of iteration; (C) generating the machine learning sample whose combination feature is the target combination feature based on the historical data record and the target combination feature, wherein the machine learning sample is used to train a machine learning model, and prediction is performed according to the trained machine learning model; Among them, for each round of iteration, the importance of each candidate combination feature in the candidate combination feature set is pre-sorted; according to the pre-sorting result, a part of the candidate combination features are screened from the candidate combination feature set to form a candidate combination feature pool; the importance of each candidate combination feature in the candidate combination feature pool is re-sorted; according to the re-sorting result, at least one candidate combination feature with higher importance is selected from the candidate combination feature pool as the target combination feature; the importance of each candidate combination feature is determined by the prediction effect of the machine learning model obtained by the corresponding training of each candidate combination feature.

2. The method of claim 1, wherein: The historical data records are pre-sorted based on a first quantity, and the historical data records are re-sorted based on a second quantity, and the second quantity is not less than the first quantity.

3. The method of claim 1, wherein: According to the pre-sorting results, candidate combination features with higher importance are screened out from the candidate combination feature set to form a candidate combination feature pool.

4. The method of claim 1, wherein: The candidate combined feature set includes the candidate combined features generated in the current round of iteration; or, the candidate combined feature set includes the candidate combined features generated in the current round of iteration and the candidate combined features generated in the previous round of iteration that are not selected as the target combined features.

5. The method of claim 1, wherein: The candidate combination features for the next round of iteration are generated by combining the target combination features selected in the current round of iteration with the at least one feature; or, the candidate combination features for the next round of iteration are generated by performing pairwise combinations between the target combination features selected in the current round of iteration and the previous round of iteration.

6. The method of claim 1, wherein: Pre-sorting is performed through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted single feature machine learning model, wherein the pre-sorted single feature machine learning model corresponds to each candidate combination feature.

7. The method of claim 1, wherein: Pre-sorting is performed through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted overall machine learning model, wherein the pre-sorted overall machine learning model corresponds to the pre-sorted basic feature subset and each of the candidate combination features.

8. The method of claim 1, wherein: Pre-sorting is performed through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorting composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorting composite machine learning model, wherein the pre-sorting composite machine learning model includes a pre-sorting basic sub-model and a pre-sorting additional sub-model based on the lifting framework, wherein the pre-sorting basic sub-model corresponds to the pre-sorting basic feature subset, and the pre-sorting additional sub-model corresponds to each of the candidate combination features.

9. The method according to claim 7 or 8, wherein: The pre-sorted basic feature subset includes the target combination features selected before the current round iteration.

10. The method of claim 1, wherein: The re-ranking is performed through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking single feature machine learning model, wherein the re-ranking single feature machine learning model corresponds to each candidate combination feature.

11. The method of claim 1, wherein: The re-ranking is performed through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking overall machine learning model, wherein the re-ranking composite machine learning model corresponds to the re-ranking basic feature subset and each candidate combination feature.

12. The method of claim 1, wherein: The reranking is performed through the following processing: for each candidate combination feature in the candidate combination feature pool, a reranking composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each reranking composite machine learning model, wherein the reranking composite machine learning model includes a reranking basic sub-model and a reranking additional sub-model based on the lifting framework, wherein the reranking basic sub-model corresponds to a reranking basic feature subset, and the reranking additional sub-model corresponds to each of the candidate combination features.

13. The method according to claim 11 or 12, wherein: The re-ranked basic feature subset includes the target combination features selected before the current round of iterations.

14. The method of claim 1, wherein: Step (B) also includes: for each round of iteration, checking whether the selected target combination features are suitable as combination features of machine learning samples.

15. The method of claim 14, wherein: In step (B), the effect change of the machine learning model based on the verified target combination features after the introduction of the selected target combination features is used to verify whether the selected target combination features are suitable as combination features of machine learning samples.

16. The method of claim 15, wherein: When the test result shows that the selected target combination features are suitable as the combination features of the machine learning samples, the selected target combination features are used as the combination features of the machine learning samples, and the next round of iterations is performed; when the test result shows that the selected target combination features are not suitable as the combination features of the machine learning samples, another part of the candidate combination features are screened out from the candidate combination feature set according to the pre-sorting result to form a new candidate combination feature pool.

17. A computer-readable medium for generating machine learning samples, wherein: A computer program for executing the method according to any one of claims 1 to 16 is recorded on the computer readable medium.

18. A computing device for generating machine learning samples, comprising a storage component and a processor, wherein: A computer executable instruction set is stored in the storage component. When the computer executable instruction set is executed by the processor, the method according to any one of claims 1 to 16 is executed.

19. A system for generating machine learning samples, comprising: A data record acquisition device, used to acquire historical data records, wherein the historical data records are text data or numerical data including a plurality of attribute information, and the historical data records include a plurality of samples involving at least one of the following items: an individual, an object, an organization, a unit, an institution, a project, an event; A feature combination device, used for iteratively combining features between at least one feature generated based on the plurality of attribute information to generate candidate combination features according to a search strategy, and selecting a target combination feature from the generated candidate combination features, wherein the target combination feature is used as a combination feature of a machine learning sample, and the search strategy is used for pruning a search tree for the combination feature to control the number of candidate combination features generated in each round of iteration; and A sample generating device, for generating the machine learning sample whose combined feature is the target combined feature based on the historical data record and the target combined feature, wherein the machine learning sample is used to train the machine learning model and / or to make predictions based on the machine learning model obtained through training; Among them, for each round of iteration, the feature combination device pre-sorts the importance of each candidate combination feature in the candidate combination feature set, selects a part of the candidate combination features from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool, re-sorts the importance of each candidate combination feature in the candidate combination feature pool, and selects at least one candidate combination feature with higher importance from the candidate combination feature pool as the target combination feature according to the re-sorting result; the importance of each candidate combination feature is determined by the prediction effect of the machine learning model obtained by the corresponding training of each candidate combination feature.

20. The system of claim 19, wherein: The feature combination device performs pre-sorting based on a first number of historical data records, and re-sorting based on a second number of historical data records, and the second number is not less than the first number.

21. The system of claim 19, wherein: The feature combination device selects candidate combination features with higher importance from the candidate combination feature set according to the pre-sorting result to form a candidate combination feature pool.

22. The system of claim 19, wherein: The candidate combined feature set includes the candidate combined features generated in the current round of iteration; or, the candidate combined feature set includes the candidate combined features generated in the current round of iteration and the candidate combined features generated in the previous round of iteration that are not selected as the target combined features.

23. The system of claim 19, wherein: The feature combination device generates candidate combination features for the next round of iteration by combining the target combination features selected in the current round of iteration with the at least one feature; or, the feature combination device generates candidate combination features for the next round of iteration by performing pairwise combinations between the target combination features selected in the current round of iteration and the previous round of iteration.

24. The system of claim 19, wherein: The feature combination device performs pre-sorting through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorted single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorted single feature machine learning model, wherein the pre-sorted single feature machine learning model corresponds to each candidate combination feature.

25. The system of claim 19, wherein: The feature combination device performs pre-sorting through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorting overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorting overall machine learning model, wherein the pre-sorting overall machine learning model corresponds to the pre-sorting basic feature subset and each of the candidate combination features.

26. The system of claim 19, wherein: The feature combination device performs pre-sorting through the following processing: for each candidate combination feature in the candidate combination feature set, a pre-sorting composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each pre-sorting composite machine learning model, wherein the pre-sorting composite machine learning model includes a pre-sorting basic sub-model and a pre-sorting additional sub-model based on a lifting framework, wherein the pre-sorting basic sub-model corresponds to a pre-sorting basic feature subset, and the pre-sorting additional sub-model corresponds to each of the candidate combination features.

27. The system of claim 25 or 26, wherein: The pre-sorted basic feature subset includes the target combination features selected before the current round iteration.

28. The system of claim 19, wherein: The feature combination device performs re-ranking through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking single feature machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking single feature machine learning model, wherein the re-ranking single feature machine learning model corresponds to each candidate combination feature.

29. The system of claim 19, wherein: The feature combination device performs re-ranking through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking overall machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking overall machine learning model, wherein the re-ranking composite machine learning model corresponds to the re-ranking basic feature subset and each candidate combination feature.

30. The system of claim 19, wherein: The feature combination device performs re-ranking through the following processing: for each candidate combination feature in the candidate combination feature pool, a re-ranking composite machine learning model is obtained, and the importance of each candidate combination feature is determined based on the effect of each re-ranking composite machine learning model, wherein the re-ranking composite machine learning model includes a re-ranking basic sub-model and a re-ranking additional sub-model based on a lifting framework, wherein the re-ranking basic sub-model corresponds to a re-ranking basic feature subset, and the re-ranking additional sub-model corresponds to each of the candidate combination features.

31. The system of claim 29 or 30, wherein: The re-ranked basic feature subset includes the target combination features selected before the current round of iteration.

32. The system of claim 19, wherein: The feature combination device also checks, for each round of iteration, whether the selected target combination features are suitable as combination features of machine learning samples.

33. The system of claim 32, wherein: The feature combination device uses the change in effect of the machine learning model based on the target combination features that have passed the test after the selected target combination features are introduced to test whether the selected target combination features are suitable as combination features of machine learning samples.

34. The system of claim 33, wherein: When the test result shows that the selected target combination features are suitable as the combination features of the machine learning samples, the feature combination device uses the selected target combination features as the combination features of the machine learning samples and performs the next round of iterations; when the test result shows that the selected target combination features are not suitable as the combination features of the machine learning samples, the feature combination device selects another part of the candidate combination features from the candidate combination feature set according to the pre-sorting result to form a new candidate combination feature pool.

Images