Method for obtaining automatic feature engineering for structured data in time sequence arrangement

Abstract

The invention discloses a method for obtaining automatic feature engineering for structured data in time sequence arrangement. The method comprises the following steps: obtaining table data X with a time sequence relationship between samples; calculating the action probability of different feature conversion operations of each feature by using a strategy network; sampling a plurality of automatic feature engineering plans; generating a plurality of tables containing expansion features; the pre-selection evaluation module is used for evaluating pre-selection evaluation indexes of pre-selection machine learning on the tables containing the extension features; weighting the pre-selected evaluation indexes on all the tables to calculate rewards; iteratively performing a sampling feature engineering plan and a strategy network optimization process; and outputting the feature engineering plan with the best evaluation index obtained in the iteration process.

Description

Technical Field

[0001] The present invention relates to the fields of computer and machine learning, and in particular to a method for obtaining automatic feature engineering for structured data arranged in time series. Background Art

[0002] Over the past few decades, various research tasks around structured data (also called tabular data) have proliferated, significantly driving the advancement of some domain-related tasks such as fraud detection, risk management, advertising recommendations, marketing technology, etc. A common feature of them is that they involve a complete machine learning pipeline built on top of structured data.

[0003] Among them, automated feature engineering plays a vital role in the complete machine learning pipeline on structured data. Automated feature engineering automatically identifies and constructs the most valuable features from raw data through algorithms, greatly reducing the time and effort required for manual feature selection and construction. Through automated feature selection and construction, information in structured data can be captured more accurately, thereby improving the accuracy and generalization ability of machine learning algorithms. Good features are the basis of a complete machine learning pipeline, and automated feature engineering helps improve the performance of machine learning algorithms on various evaluation indicators.

[0004] However, previous automated feature engineering has focused on structured data with no clear relationship between samples. By analyzing the complex relationship between features, different features in a sample are used to generate new extended features, thereby improving various evaluation indicators of machine learning algorithms on new feature sets. The present invention focuses on feature engineering operations on structured data with temporal relationships between samples, specifically structured data sampled at certain time intervals. Such data samples are arranged in chronological order, and different samples are spaced at the same time interval. For such data, it is necessary to extract information from historical samples to construct new features in order to better improve the performance of machine learning algorithms on various evaluation indicators.

[0005] In view of the shortcomings of the prior art, the present invention proposes an automatic feature engineering method for time series structured data. While using different feature combinations to form new features, the features of historical samples are used to construct time series and generate features containing historical information. The present invention is aimed at structured data with clear time series relationships between samples. Specifically, different samples in these data sets are sampling information separated by the same time interval. Summary of the invention

[0006] The automated feature engineering (Automated Feature Engineering, AutoFE) of the present invention aims to perform this process by automated means. Given a data set D containing original features X and target variables Y, as well as a pre-selected learning algorithm L and performance evaluation index E, the goal of automated feature engineering is to find a feature conversion operation plan T that converts the original feature set X into a new feature set through a series of feature conversion operations. Make the learning algorithm L use the new feature set The model trained above reaches the maximum value in the performance evaluation indicator E.

[0007] The present invention is implemented by the following technical solution: a method for obtaining automatic feature engineering for structured data arranged in time series is provided, the method comprising the following steps:

[0008] (1) Obtaining table data X with a time series relationship between samples; where X is a two-dimensional table, which is composed of different feature columns in the vertical direction and different samples in the horizontal direction, and whose adjacent sampling moments are separated by the same time;

[0009] (2) Using the automatic feature engineering strategy network to calculate the action probability of feature conversion operations for each feature;

[0010] (3) sampling multiple automatic feature engineering plans according to the action probabilities of different feature conversion operations; using the automatic feature engineering plans to generate several tables containing extended features;

[0011] (4) Evaluate the pre-selected evaluation indicators of the pre-selected machine learning on each table containing extended features, record the automatic feature engineering plan that generates the table containing extended features with the best pre-selected evaluation indicators; weight the pre-selected evaluation indicators on all tables to calculate the reward;

[0012] (5) Iteratively use the optimized policy network to sample feature engineering plans, calculate rewards, record the feature engineering plan with the best pre-selected evaluation index, and optimize the policy network until multiple iterations fail to obtain a feature engineering plan with a better pre-selected evaluation index or the number of iterations is reached;

[0013] (6) Using the weighted average of the evaluation indicators of the pre-selected machine learning algorithm on the validation set containing the extended feature table as a reward, the proximal optimization strategy PPO is used to optimize the policy network so that the policy network gives a better feature transformation operation set based on the evaluation indicators of the pre-selected machine learning algorithm. Repeat iterative steps (1) to (5) until the feature engineering plan with the best evaluation indicator obtained in the iterative process is output.

[0014] Specifically, the feature conversion operation in step (2) includes a feature conversion function and input features of the conversion function.

[0015] Furthermore, the policy network in step (2) includes an inter-group policy network and a temporal policy network, specifically:

[0016] The inter-group feature strategy network is used to calculate the probability of an inter-group feature conversion operation on each feature, wherein the inter-group feature conversion operation refers to a feature conversion operation that uses an inter-group feature conversion function to act on the feature and several other features as input features;

[0017] The timing feature strategy network is used to calculate the probability of a timing feature conversion operation; the timing feature conversion operation refers to a timing feature conversion operation on the characteristics of a timing feature conversion function, specifically treating samples of a fixed-length time window on the characteristics on which the timing feature conversion operation acts as a time series, and using a timing feature function that extracts a feature value from a time series to extract the timing feature of the feature.

[0018] Furthermore, the inter-group feature strategy network adopts the strategy network of the FETCH automatic feature engineering framework, including a dense layer, a multi-head attention layer, an action decoding layer and a softmax layer; the dense layer is a fully convolutional network, which converts the 1xm feature vector in the structured data into a fixed-length feature vector to reduce the dimension of the feature vector, and uses the full convolutional network with the same weight to reduce the dimension of the feature vectors of different features to ensure the effectiveness of the dimensionality reduction operation for the new extended features; the multi-head attention layer extracts the relationship between the table features, and first tries to calculate the attention matrix using the feature vector after the dimensionality reduction of the dense layer, mines the relationship between the features in the table, encodes the relationship between the feature columns, and uses the attention matrix to linearly transform the feature vector group to obtain the feature vector that can reflect the relationship with other features; and then uses an attention layer including an action decoding layer and a softmax layer to obtain the feature vector obtained by converting it into the probability of a specific feature conversion operation.

[0019] Furthermore, the timing strategy network is a timing information extraction module for extracting sample information added before the dense layer of the strategy network of the FETCH framework, including a TCN layer, a dense layer, a multi-head attention layer, an action decoding layer and a softmax layer; the module for extracting timing information is a timing convolutional network TCN module, that is, a TCN layer, and TCN uses causal convolution to perform convolution operations only on historical data to ensure that the output at any given time point only depends on the information before that time point; TCN uses dilated convolution that skips part of the input, allowing the model to increase the receptive field and expand the range of input sequences that the model can observe while keeping the number of parameters unchanged.

[0020] Furthermore, the inter-group feature conversion operation on the feature refers to a feature conversion operation using the inter-group feature conversion function, using the feature or the feature and other features as input features; the feature conversion function includes a unary feature conversion function and a binary feature conversion function;

[0021] The feature conversion functions between tuples include: tanh, square, inverse, log, sqrt, sigmoid value conversion functions;

[0022] The feature conversion functions between two tuples include: addition, subtraction, multiplication, division, and combine arithmetic functions;

[0023] For an inter-group feature conversion operation on a feature, if its feature conversion function is a unigram inter-group feature conversion function, then the feature is the input feature of the conversion function; if its feature conversion function is a binary inter-group feature conversion function that requires two input features, then the feature is used as the first input feature of the conversion function, and the other features are used as the second input feature of the conversion function. Different second input features correspond to different feature conversion operations, that is, a binary inter-group conversion feature function has n-1 inter-group conversion operations on one feature.

[0024] Furthermore, the time series feature conversion operation on the feature refers to an operation of taking samples of a fixed-length time window on the feature as a time series, and extracting time series features using a time series feature function that extracts a feature value from a time series; the time series feature conversion function is all the time series feature functions in the EfficientFCParameters class in the python library tsfresh0.20.1, and there are 784 alternative time series feature conversion functions.

[0025] Furthermore, the sampling of step (3) is specifically as follows: the sampling method selected is a polynomial sampling method, and during the sampling process, multiple feature conversion operation sets (t1, t3, ... t e ), which is used to fully explore the feature space, where e is the number of feature transformation operation sets for each sampling, and one feature transformation operation set includes an inter-group feature transformation operation and a temporal feature transformation operation on each feature sampling.

[0026] Furthermore, the step (4) is specifically as follows: replace table X with a table containing extended features, continue the operations from step (2) to step (4) to obtain a table containing second-order extended features, iterate to obtain a table containing higher-order extended features, and set the highest order of extended features; the higher-order extended features refer to features that use feature conversion operations on extended features, wherein extended features generated by using original features through feature conversion operations are first-order extended features, and features generated by using i-order extended features through feature conversion operations are called i+1-order features; in the process of iteratively generating tables containing high-order extended features, each table containing extended features only samples and selects one feature conversion operation set, so that each time a higher-order feature is generated, a total of e feature sets are still sampled and selected.

[0027] Furthermore, in step (6), when preselecting a feature transformation operation set with a better evaluation index of a machine learning algorithm, for classification tasks, the preselected machine learning algorithm is selected from linear support vector machine, logistic regression, random forest, gradient boosting tree XGBOOST, and light gradient boosting tree LightGBM; for regression tasks, the preselected machine learning algorithm is selected from linear support vector machine, linear regression LinearRegression, random forest, gradient boosting tree, and light gradient boosting tree; the preselected evaluation index E on the classification task is F1-score, and its calculation formula is as follows:

[0028]

[0029] Among them, TP represents the number of true positive examples, that is, the number of instances correctly predicted by the model as positive categories; FP represents the number of false positive examples, that is, the number of instances incorrectly predicted by the model as positive categories; FN represents the number of false negative examples, that is, the number of instances incorrectly predicted by the model as negative categories;

[0030] Then, substitute these values ​​into the formula for the F1 score to calculate the performance metric of the model;

[0031] The pre-selected evaluation index for the regression task is 1-relative absolute error rae, and its calculation formula is as follows:

[0032]

[0033] in, represents the predicted value of the label, and y represents the actual value of the label. Represents the mean of the actual values ​​of the labels.

[0034] The beneficial effects of this method are as follows:

[0035] The present invention breaks through the assumption that the arrangement of structured data samples does not affect the prediction of structured data in previous automatic feature engineering of structured data, and focuses on structured data arranged in time series, in which the sample order contains important information. While using different features to construct new features for structured data arranged in time series, historical sample data is used to construct time series features. This solves the problem that previous feature engineering on structured data did not consider the impression of sample data on prediction performance, and that automatic feature engineering on time series data lacked the mutual influence of different features. Using such a method to construct new features on structured data arranged in time series helps to grasp the relationship with historical data while grasping the relationship between different features, thereby improving the effect of each pre-selected machine learning indicator on the pre-selected machine learning algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the process of the present invention;

[0037] Figure 2 A schematic diagram of the automatic feature engineering technology framework used in the present invention;

[0038] Figure 3 This is a schematic diagram of the inter-group strategy network of the present invention;

[0039] Figure 4 This is a schematic diagram of the timing strategy network in this aspect. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. It should be noted that the features in the following embodiments and implementation methods can be combined with each other without conflict.

[0041] Automated Feature Engineering (AutoFE) aims to perform this process by automated means. Given a dataset D containing original features X and target variables Y, as well as a pre-selected learning algorithm L and performance evaluation indicator E, the goal of automated feature engineering is to find a feature transformation operation plan T that transforms the original feature set X into a new feature set through a series of feature transformation operations. Make the learning algorithm L used in the new feature set The model trained above reaches the maximum value in the performance evaluation index E. Figure 1 As shown, the present invention provides a method for obtaining automatic feature engineering for structured data arranged in time series, which is mainly divided into the following steps:

[0042] Step 1: Obtain table data X with a time series relationship between samples, where X is a two-dimensional m×n table with (c1, c2, …c n ) different feature columns, where c i is the i-th feature, n is the number of features. m ) Different sample compositions, s i The samples sampled at each i-th moment, m is the number of samples. Adjacent sampling moments are separated by the same time.

[0043] Step 2: If Figure 2 As shown, the present invention uses an automatic feature engineering framework to extract features from time series data and calculate the reward of the extracted features. The automatic feature engineering framework includes a strategy network, sampling actions, feature extraction, and downstream machine learning models. The specific operations of extracting features and calculating rewards using the automatic feature engineering framework are as follows:

[0044] Step 2.1. Use the automatic feature engineering strategy network to calculate the action probability of different feature conversion operations for each feature, use the inter-group feature strategy network to calculate the probability of each inter-group feature conversion operation on each feature, and use the time series feature strategy network to calculate the probability of each time series feature conversion operation on each feature.

[0045] Specifically, the feature conversion operation includes a feature conversion function and an input feature of the conversion function;

[0046] Specifically, the inter-group feature conversion operation on a feature refers to a feature conversion operation using the inter-group feature conversion function, using the feature or the feature and other features as input features. The feature conversion function includes a unary feature conversion function and a binary feature conversion function;

[0047] The feature conversion functions between tuples include: tanh, square, inverse, log, sqrt, sigmoid and other value conversion functions;

[0048] The feature conversion functions between two tuples include: addition, subtraction, multiplication, division, combine and other combination arithmetic functions;

[0049] Specifically, for an inter-group feature conversion operation on a feature, if its feature conversion function is a one-tuple inter-group feature conversion function, then the feature is the input feature of the conversion function; if its feature conversion function is a two-tuple inter-group feature conversion function that requires two input features, then the feature is used as the first input feature of the conversion function, and other features are used as the second input features of the conversion function. Different second input features correspond to different feature conversion operations, that is, a two-tuple inter-group conversion feature function has n-1 inter-group conversion operations on one feature;

[0050] Specifically, the time series feature conversion operation on a feature refers to an operation of treating samples of a fixed-length time window on the feature as a time series, and extracting the time series feature using a time series feature function that extracts a feature value from the time series;

[0051] The candidate time series feature conversion functions of this method are all the time series feature functions in the EfficientFCParameters class in the Python library tsfresh0.20.1, with a total of 784 candidate time series feature conversion functions.

[0052] Step 2.2: Sampling operation obtains multiple feature transformation operation sets. The sampling method selected in this method is the polynomial sampling method. During the sampling process, multiple feature transformation operation sets (t1, t3, ... t e ) is used to fully explore the feature space, and e is the number of feature transformation operation sets for each sampling. One feature transformation operation set includes an inter-group feature transformation operation and a temporal feature transformation operation on each feature sample.

[0053] Step 2.3: Extract extended features based on the sampled feature operation set. Each feature conversion operation set selected in step 3 will generate a table containing extended features. Each feature engineering operation in the feature conversion operation set will generate a feature for each sample. When using the inter-group feature engineering operation to generate the extended features of a sample, the input features of the component feature engineering operation on the sample are directly used, and the feature conversion function of the inter-group feature engineering operation is called to generate new extended features. When using a time series feature engineering operation to generate the extended features of a sample, the input features of the time series feature engineering operation on the sample and a continuous historical sample of the sample are used to construct a time series, and the time feature conversion function is used to calculate an extended feature of the sample.

[0054] Step 2.4: Table X containing the extended features 1,1 ,…X 1,e Substituting table X, continue steps 2.1 to 2.3 to obtain table X containing the second-order expanded features 2,1 ,…X2,e , iteratively obtains a table X containing higher-order extended features i,1 , …X p,q , …X k,e , k is the highest order of the set extended features; the high-order extended features refer to the features using feature conversion operations on the extended features, wherein the extended features generated by the original features through feature conversion operations are first-order extended features, and the features generated by the i-order extended features through feature conversion operations are called i+1-order features; in the process of iteratively generating tables containing high-order extended features, each table containing extended features only samples and selects one feature conversion operation set, so that each time a higher-order feature is generated, a total of e feature sets are still sampled and selected.

[0055] Step 2.5: Calculate the reward of the multi-order extended feature table obtained from steps 2.1 to 2.5 on the downstream machine learning model. Use the extended feature table X generated by the feature conversion operation obtained in step 5 1,1 , …X p,q , …X k,e The training set is trained on the downstream machine learning model, and the pre-selected evaluation indicators are used to evaluate the effect of the machine learning algorithm on the test set, and the evaluation indicators of the validation set containing the extended feature table and the feature conversion operation set sequence that generates them are recorded; the extended feature table X is generated p,q The automatic feature engineering plan refers to the iterative generation of X in step 5. p,1 To X p,q The feature engineering plan consists of the feature transformation operation set selected at the time. By operating the feature sets in the feature engineering plan in sequence, the table X can be transformed into X p,q .

[0056] Step 3: Use the weighted average of the evaluation indicators of the pre-selected machine learning algorithm on the validation set containing the extended feature table as the reward, and use the proximal optimization strategy (PPO) to optimize the policy network so that the policy network can give a better feature transformation operation set based on the evaluation indicators of the pre-selected machine learning algorithm;

[0057] Among them, for classification tasks, the pre-selected machine learning algorithm L of this method is selected from linear support vector machine, logistic regression, random forest, gradient boosting tree (XGBOOST), and light gradient boosting tree (LightGBM); for regression tasks, the pre-selected machine learning algorithm L of this method is selected from linear support vector machine, linear regression (LinearRegression), random forest, gradient boosting tree, and light gradient boosting tree;

[0058] The pre-selected evaluation index E of this method on the classification task is F1-score, and its calculation formula is as follows:

[0059]

[0060] Here, TP stands for the number of true positives, i.e., the number of instances correctly predicted by the model as positive. FP stands for the number of false positives, i.e., the number of instances incorrectly predicted by the model as positive. FN stands for the number of false negatives, i.e., the number of instances incorrectly predicted by the model as negative.

[0061] Then, by plugging these values ​​into the formula for the F1 score, we can calculate the performance metric of the model;

[0062] The pre-selected evaluation index for the regression task of this method is 1-relative absolute error (rae), and its calculation formula is as follows:

[0063]

[0064] in, represents the predicted value of the label, and y represents the actual value of the label. Represents the mean of the actual values ​​of the labels.

[0065] Step 4: Repeat steps 2 to 4 until the set number of iterations is reached or multiple iterations fail to find a better feature engineering plan with the pre-selected evaluation indicator of the pre-selected machine learning algorithm. The best feature engineering plan with the pre-selected evaluation indicator of the pre-selected machine learning algorithm in the iterative process is given as a sequence of feature transformation operations found by automatic feature engineering. A new structured data containing the same features can use the feature engineering plan to generate extended features to improve the value of the pre-selected evaluation indicator of the pre-selected machine learning algorithm.

[0066] Further, if Figure 3 As shown in the figure, the strategy network used to calculate the probability of inter-group feature conversion operations in step 2.1 uses the same strategy network as the FETCH framework, which consists of four parts: dense layer, attention layer, action decoding layer, and softmax layer. The dense layer is a fully convolutional network. The 1xm feature vector in the structured data is converted into a fixed-length feature vector through the fully convolutional network to reduce the dimension of the feature vector. At the same time, the feature vectors of different features are reduced in dimension using the fully convolutional network with the same weight to ensure the effectiveness of the dimensionality reduction operation for the new extended features; the multi-head attention layer extracts the relationship between the table features. It first tries to calculate the attention matrix using the feature vector after the dimensionality reduction of the dense layer, mines the complex mutual relationship between the features in the table, and encodes the relationship between the feature columns. Then, the feature vector group is linearly transformed with the help of the attention matrix to obtain the feature vector that can reflect the relationship between other features; then, a layer including an action decoding layer and a softmax layer is used to convert the feature vector obtained by the attention layer into the probability of a specific feature conversion operation.

[0067] Further, if Figure 4 As shown, the policy network used to calculate the probability of time series feature conversion in step 2.1 uses the policy network of the FETCH framework, and adds a time series information extraction module before the dense layer of the policy network of the FETCH framework. The module used in this method to extract time series information is the temporal convolutional network (TCN) module. TCN uses causal convolution to perform convolution operations only on historical data, ensuring that the output at any given time point depends only on the information before that time point. TCN uses dilated convolutions that skip part of the input, allowing the model to increase the receptive field while keeping the number of parameters unchanged, expanding the range of input sequences that the model can observe.

[0068] The following examples use the same hyperparameters of the policy network, where the dimension of the feature vector after dimensionality reduction is 128, the convolution kernel size of the TCN is 2, the dimension of the Q, K, V vectors in the multi-head attention mechanism is 32 for a total of 6 heads, and the dimension of the feature vector after the multi-head attention layer is 64. The maximum number of search iterations is set to 300, and the number of iterations of the policy network using the proximal optimization strategy (PPO) is 10 per iteration. At the same time, the Adam optimizer is used to train the agent. Each time a higher-order extended feature is generated, the number of sampled feature conversion operation sets is 4, and the highest order of the sampled extended features is 3. The learning rate is 0.001.

[0069] The embodiment of the present invention uses the Room Occupancy Estimation dataset for feature engineering operations, which is a structured dataset containing 16 initial features: <S1_Temp,S2_Temp,S3_Temp,S4_Temp,S1_Light,S2_Light,S3_Light,S4_Light,S1_Sound,S2_Sound,S3_Sound,S4_Sound,S5_CO2,S5_CO2_Slope,S6_PIR,S7_PIR> A classification task, the prediction target is the number of people in the room (Room_Occupancy_Count).

[0070] In the embodiment of the present invention, the following steps are performed:

[0071] (A) The policy network is used to iterate K times to sample feature transformation operation sets for the data set. Each iteration process summarizes and samples E feature transformation operation sets, and obtains E tables containing extended features. A total of K×E tables containing extended features and an automatic feature plan for generating these tables from the original tables are obtained.

[0072] (B) Use the LightGBM algorithm to calculate the F1-score on these datasets, calculate the reward based on the calculation results, record the feature engineering plan with high F1-score, and use the PPO algorithm to optimize the policy network.

[0073] (C) Iterate the process of using the optimized policy network to sample feature engineering plans, calculate rewards, record feature engineering plans with high F1-scores, and optimize the policy network until 10 iterations fail to find a feature engineering plan with a higher F1-score or the number of iterations reaches 300.

[0074] The partial feature subset of the new feature set formed in this embodiment is that the constructed feature subset can improve the F1-score of the LightGBM algorithm in the classification task from 0.925 to 0.988.

[0075] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0076] It will be appreciated that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for automatic feature engineering of structured data arranged in time series, characterized in that: The method comprises the following steps: (1) Obtaining table data X with a time series relationship between samples; where X is a two-dimensional table, which is composed of different feature columns in the vertical direction and different samples in the horizontal direction, and the adjacent sampling moments are separated by the same time; (2) Using the automatic feature engineering strategy network to calculate the action probability of feature conversion operations for each feature; (3) sampling multiple automatic feature engineering plans according to the action probabilities of different feature conversion operations; using the automatic feature engineering plans to generate several tables containing extended features; (4) Evaluate the pre-selected evaluation indicators of the pre-selected machine learning on each table containing extended features, record the automatic feature engineering plan that generates the table containing extended features with the best pre-selected evaluation indicators; weight the pre-selected evaluation indicators on all tables to calculate the reward; (5) Iteratively use the optimized policy network to sample feature engineering plans, calculate rewards, record the feature engineering plan with the best pre-selected evaluation index, and optimize the policy network until multiple iterations fail to obtain a feature engineering plan with a better pre-selected evaluation index or the number of iterations is reached; (6) Using the weighted average of the evaluation indicators of the pre-selected machine learning algorithm on the validation set containing the extended feature table as a reward, the proximal optimization strategy PPO is used to optimize the policy network so that the policy network gives a better feature transformation operation set based on the evaluation indicators of the pre-selected machine learning algorithm. Repeat iterative steps (1) to (5) until the feature engineering plan with the best evaluation indicator obtained in the iterative process is output.

2. The method for automatic feature engineering of time-series structured data according to claim 1, characterized in that: The feature conversion operation in step (2) includes a feature conversion function and input features of the conversion function.

3. The method for automatic feature engineering of time-series structured data according to claim 1, characterized in that: The policy network in step (2) includes an inter-group policy network and a temporal policy network, specifically: The inter-group feature strategy network is used to calculate the probability of an inter-group feature conversion operation on each feature, wherein the inter-group feature conversion operation refers to a feature conversion operation that uses an inter-group feature conversion function to act on the feature and several other features as input features; The timing feature strategy network is used to calculate the probability of a timing feature conversion operation; the timing feature conversion operation refers to a timing feature conversion operation on the characteristics of a timing feature conversion function, specifically treating samples of a fixed-length time window on the characteristics on which the timing feature conversion operation acts as a time series, and using a timing feature function that extracts a feature value from a time series to extract the timing feature of the feature.

4. The method for automatic feature engineering of time-series structured data according to claim 3, characterized in that: The inter-group feature strategy network adopts the strategy network of the FETCH automatic feature engineering framework, including a dense layer, a multi-head attention layer, an action decoding layer and a softmax layer; the dense layer is a fully convolutional network, which converts the 1xm feature vector in the structured data into a fixed-length feature vector to reduce the dimension of the feature vector, and uses the full convolutional network with the same weight to reduce the dimension of the feature vectors of different features to ensure the effectiveness of the dimensionality reduction operation for the new extended features; the multi-head attention layer extracts the relationship between the table features, and first tries to calculate the attention matrix using the feature vector after the dimensionality reduction of the dense layer, mines the relationship between the features in the table, encodes the relationship between the feature columns, and uses the attention matrix to linearly transform the feature vector group to obtain the feature vector that can reflect the relationship between other features; and then uses an attention layer including an action decoding layer and a softmax layer to obtain the feature vector converted into the probability of a specific feature conversion operation.

5. The method for automatic feature engineering for time-series structured data according to claim 3, characterized in that: The temporal strategy network is a temporal information extraction module for extracting sample information added before the dense layer of the strategy network of the FETCH framework, including a TCN layer, a dense layer, a multi-head attention layer, an action decoding layer and a softmax layer; the module for extracting temporal information is a temporal convolutional network TCN module, that is, a TCN layer. TCN uses causal convolution to perform convolution operations only on historical data, ensuring that the output at any given time point only depends on the information before that time point; TCN uses dilated convolution that skips part of the input, allowing the model to increase the receptive field and expand the range of input sequences that the model can observe while keeping the number of parameters unchanged.

6. The method for automatic feature engineering of time-series structured data according to claim 3, characterized in that: The inter-group feature conversion operation on the feature refers to a feature conversion operation using an inter-group feature conversion function, using the feature or the feature and other features as input features; the feature conversion function includes a unary feature conversion function and a binary feature conversion function; The feature conversion functions between tuples include: tanh, square, inverse, log, sqrt, sigmoid value conversion functions; The feature conversion functions between two tuples include: addition, subtraction, multiplication, division, and combine arithmetic functions; For an inter-group feature conversion operation on a feature, if its feature conversion function is a unigram inter-group feature conversion function, then the feature is the input feature of the conversion function; if its feature conversion function is a binary inter-group feature conversion function that requires two input features, then the feature is used as the first input feature of the conversion function, and the other features are used as the second input feature of the conversion function. Different second input features correspond to different feature conversion operations, that is, a binary inter-group conversion feature function has n-1 inter-group conversion operations on one feature.

7. The method for automatic feature engineering of time-series structured data according to claim 3, characterized in that: The time series feature conversion operation on the feature refers to the operation of extracting time series features by taking samples of a fixed-length time window on the feature as a time series and using a time series feature function that extracts a feature value from a time series; the time series feature conversion function is all the time series feature functions in the EfficientFCParameters class in the python library tsfresh0.20.1, and there are 784 alternative time series feature conversion functions.

8. The method for automatic feature engineering of time-series structured data according to claim 1, characterized in that: The sampling of step (3) is specifically as follows: the sampling method selected is a polynomial sampling method, and during the sampling process, multiple feature conversion operation sets (t1, t3, ... t e ), which is used to fully explore the feature space, where e is the number of feature transformation operation sets for each sampling, and one feature transformation operation set includes an inter-group feature transformation operation and a temporal feature transformation operation on each feature sample.

9. The method for automatic feature engineering of time-series structured data according to claim 1, characterized in that: The step (4) is specifically as follows: replace table X with a table containing extended features, continue the operations from step (2) to step (4) to obtain a table containing second-order extended features, iterate to obtain a table containing higher-order extended features, and set the highest order of extended features; the higher-order extended features refer to features using feature conversion operations on extended features, wherein extended features generated by using original features through feature conversion operations are first-order extended features, and features generated by using i-order extended features through feature conversion operations are called i+1-order features; in the process of iteratively generating tables containing high-order extended features, each table containing extended features only samples and selects one feature conversion operation set, so that each time a higher-order feature is generated, a total of e feature sets are still sampled and selected.

10. The method for automatic feature engineering of time-series structured data according to claim 1, characterized in that: In the step (6), when preselecting a feature conversion operation set with a better evaluation index of a machine learning algorithm, for the classification task, the preselected machine learning algorithm is selected from a linear support vector machine, logistic regression, random forest, gradient boosting tree XGBOOST, and light gradient boosting tree LightGBM; for the regression task, the preselected machine learning algorithm is selected from a linear support vector machine, linear regression LinearRegression, random forest, gradient boosting tree, and light gradient boosting tree; the preselected evaluation index E on the classification task is F1-score, and its calculation formula is as follows: Among them, TP represents the number of true positive examples, that is, the number of instances correctly predicted by the model as positive categories; FP represents the number of false positive examples, that is, the number of instances incorrectly predicted by the model as positive categories; FN represents the number of false negative examples, that is, the number of instances incorrectly predicted by the model as negative categories; Then, substitute these values ​​into the formula for the F1 score to calculate the performance metric of the model; The pre-selected evaluation index for the regression task is 1-relative absolute error rae, and its calculation formula is as follows: in, represents the predicted value of the label, and y represents the actual value of the label. Represents the mean of the actual values ​​of the labels.

Images