Visual machine learning feature extraction system and method based on feature computation graph

Abstract

The present invention provides a visual machine learning feature extraction system and method based on a feature calculation graph. First, each column of the structured original data set, the feature transformation function, and the output items are abstracted into node controls and drawn on a graphical interface; then, the user drags the node controls to build a feature engineering processing flow according to their own needs; finally, the data processing module reads the feature calculation graph built by the user, uses depth-first search to traverse each node to extract features, and completes the feature conversion. The present invention abstracts the flowchart of the machine learning feature extraction stage into a feature calculation graph, so that users can independently build data processing flows by dragging controls, reducing the difficulty of feature extraction tasks, and facilitating modification and debugging, which helps to reduce the mental burden of data processing personnel and allows non-professionals to participate in related projects.

Description

Technical Field

[0001] The present invention belongs to the field of feature engineering for machine learning, and specifically relates to a visual machine learning feature extraction system and method based on feature computation graphs. Background Art

[0002] Feature engineering is a crucial component of machine learning. This process, known as feature extraction, converts raw data into features that can be directly fed into machine learning models. Feature engineering primarily involves using data transformation functions to transform and aggregate data, generating new features. These new features, generated by data transformation, exhibit better optimization properties when distributed in high-dimensional space. Therefore, using these transformed features can help train models with higher accuracy than using the original features.

[0003] Data transformation is a key step in feature extraction. This process converts one or more input feature columns into one or more new feature columns according to a specific calculation method. Therefore, data transformation operations are also called feature transformation operators. Generally speaking, feature transformation operators can be divided into two main categories based on parameter rules. The first category is metaparameters, which are values that need to be set initially, such as the base for logarithmic transformations and the threshold for binarization. The second category is statistical parameters related to the specific data set, such as the maximum and mean for normalization transformations and the mapping table for one-hot encoding. For operators with only metaparameters, a single pass is sufficient to obtain the transformed data. However, for operators with statistical parameters, the first pass only calculates the statistical parameters, requiring a second pass to complete the data transformation. The first pass is generally called fit, and the second pass is called transform.

[0004] In feature engineering, data engineers write code or call external libraries to implement feature transformation operators. They then use these operators to orchestrate code so that the raw data is processed layer by layer to produce new features that meet their expectations. However, this process generally requires data processing professionals proficient in code writing, and the code flow is cumbersome, reducing production efficiency. To address these difficulties, this method proposes a visual feature extraction solution based on feature calculation graphs. This method constructs the feature extraction calculation graph by dragging controls on the interface, and extracts features through a depth-first search of the calculation graph in the back-end processing. This method makes the feature extraction process more intuitive and easy to operate. Summary of the Invention

[0005] The purpose of this method is to guide users to build a feature extraction process calculation graph through visualization, solving the problem of having to write code to process data in the past.

[0006] The present invention provides a visual machine learning feature extraction system based on a feature calculation graph, comprising a visual front-end interface and a data processing module. The visual front-end interface includes three node controls for building a feature calculation graph, namely, an input node, an operator node, and an output node. The data processing module is used to process user data sets using the feature calculation graph. Each input node corresponds to a column of elements in the user data set. Operator nodes have a variety of different algorithm types. Operator nodes of different algorithm types contain corresponding feature transformation functions and their parameters for transforming the input data.

[0007] Preferably, the operator nodes are classified by algorithm type, and include data scaling nodes, discretization nodes, vectorization nodes, function transformation nodes, and dimension reduction nodes. The data scaling node performs standardization or normalization operations on the input data according to preset parameters. The discretization node performs equidistant bucketing, equal-frequency bucketing, or binarization on the input data according to preset parameters; wherein, the parameters required for equidistant bucketing and equal-frequency bucketing are the number of buckets, and the parameter required for binarization is the threshold. The part of the input data greater than or equal to the threshold will be uniformly transformed into 1, and the part less than the threshold will be uniformly transformed into 0. The vectorization node performs one-hot encoding or word vector transformation on the input data according to preset parameters. wherein, the word vector transformation corresponds to a word vector library. The function transformation node performs logarithmic transformation and exponential transformation on the input data according to preset parameters, wherein the parameter required for logarithmic transformation is the base; the parameter required for exponential transformation is the exponent. The Dimensionality Reduction node reduces the dimensionality of the input data using principal component analysis or tSNE. The parameter required for principal component analysis is the number of principal components, while the parameter required for tSNE is the target dimension.

[0008] Preferably, in the initial visual front-end interface, multiple Input nodes consisting of column names and one Output node are presented, and an operator toolbox is presented on the visual front-end interface; the operator toolbox arranges various operator nodes in a classified display manner.

[0009] Preferably, the data type of the Input node is a numerical type or a categorical type; the input data type of the operator node is the same as the data type of the Input node;

[0010] Preferably, the feature computation graph is composed of an Input node, one or more Operator nodes, and an Output node. Each Input node is connected to an Output node via one or more Operator nodes. Each Input node corresponds to a column of data in the user dataset.

[0011] Preferably, the user data set has an automatically generated ID column as a row identifier.

[0012] Preferably, an Input node has one or more output terminals for connecting to one or more subsequent nodes. An Output node has one or more input terminals. An Operator node has one or more input terminals and one or more output terminals. The input terminals of an Operator node are connected to the corresponding Input node or the operator node of the previous level, and the output terminals are connected to the operator node of the next level or the Output node.

[0013] The method for using the feature computation graph-based visual machine learning feature extraction system to perform dataset feature analysis is as follows:

[0014] Step 1. The user uses the visual front-end interface to build a feature calculation graph by dragging different nodes according to the feature extraction target. Each Input node is connected to the same Output node through one or more operator nodes; the type of each operator node corresponds to the features that need to be extracted from the user dataset, and is selected by the user according to needs. The user dataset adds an ID column before inputting each Input node; the number of Input nodes corresponds to the number of columns in the user dataset; each Input node inputs the ID column of the user dataset and a corresponding column of elements. For an operator node or Output node with multiple child nodes, its input data is the data obtained by merging the output data of each child node by column. The child node represents each operator node or Input node at the previous level of the current Output node or operator node.

[0015] Step 2: The data processing module uses the graph depth-first search method to calculate the user data set, visiting each node from the output node forward. The specific process is as follows:

[0016] 2-1. Starting from the Output node, the fit method of each child node is called sequentially until all Input nodes have been fit. The fit method calculates the statistics corresponding to the node's input data. When the fit method is executed, the transform method of the child node is called. The transform method performs the data transformation corresponding to the node itself.

[0017] 2-2. Starting from the Output node, call the transform method of the child nodes step by step, so that the Output node outputs the result data set after feature extraction.

[0018] Preferably, a JSON object is constructed upon completion of the feature computation graph and upon completion of the fit method for each node. These JSON objects are merged to form a single JSON file. After step 2, if feature extraction is required for a new user dataset, the fit method for each node is directly completed using the JSON file, with only the transform method being executed.

[0019] The beneficial effects of the present invention are as follows:

[0020] 1. The present invention provides three node controls for building a feature calculation graph, allowing data processing personnel to quickly and easily build a feature calculation graph corresponding to the data transformation process according to the needs of feature extraction, and use the data processing module to automatically read the feature calculation graph and perform feature extraction on the user data set, freeing data processing personnel from code writing and allowing them to focus on data exploration.

[0021] 2. The present invention lowers the threshold for feature engineering processing, allowing non-professionals to participate in feature extraction projects more easily.

[0022] 3. The present invention provides a visual feature engineering solution, making the data flow more intuitive. During the debugging process, it is very convenient to modify the feature calculation graph, thereby improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of three types of nodes and their input and output terminals used in the present invention;

[0024] Figure 2 A schematic diagram of a visual interface for building a feature calculation graph for the present invention;

[0025] Figure 3 This is a typical feature calculation graph structure and its traversal process;

[0026] Figure 4 This is a flow chart of the present invention performing feature calculation on a feature calculation graph. DETAILED DESCRIPTION

[0027] The present invention will be further described below with reference to the accompanying drawings.

[0028] like Figure 1 、 2As shown in Figure 3, a visual machine learning feature extraction system based on a feature calculation graph includes a visual front-end interface and a data processing module. The visual front-end interface contains three types of node controls: Input nodes, operator nodes, and Output nodes, allowing users to build feature calculation graphs by dragging different nodes. The data processing module is used to process user datasets based on the feature calculation graph built by the user. The feature calculation graph is composed of an Input node, one or more operator nodes, and an Output node. The Input node is connected to the Output node through one or more operator nodes. Each operator node is arranged between the Input node and the Output node in series, parallel, or a combination of series and parallel. The Input node is used to input the user dataset. The user dataset is structured data that the user needs to process, and the program automatically generates an ID column as a row identifier to provide an identifier for the merging process after column splitting.

[0029] In a feature computation graph, an Input node represents a column in a structured data table and has one or more outputs, indicating that it can be connected to one or more subsequent nodes. This is similar to the reuse of a column to generate features in traditional data processing methods. An Operator node represents a feature transformation function and has one or more inputs and one or more outputs. Multi-inputs have two semantics: if the feature transformation function accepts only single-value inputs, then multi-inputs apply the same function to each column input value, and the output is a vector value. If the feature transformation function accepts multiple inputs, then multi-inputs represent an aggregate transformation operation, and the output can be a vector or scalar. Similar to the Input node, the output of an Operator node can be connected to one or more subsequent nodes, allowing the output data to be reused. The Output node is the terminal node of the entire feature computation graph. Data generated by all other nodes connected to the Output node are combined here to form feature data that can be directly input into machine learning algorithms.

[0030] like Figure 2 As shown in the figure, the visualization front-end displays multiple Input nodes consisting of column names, followed by a final Output node. On the right side of the interface is an operator toolbox consisting of various feature transformation operators, arranged in categories for easy searching. Users select appropriate operators from the operator toolbox to transform the data, ultimately generating a feature calculation graph, also known as a feature calculation graph.

[0031] Each Input node corresponds to a column of elements in the user data set, identified by a column name. The column name needs to be entered when constructing the object. The user data set can be fully represented by a group of Input nodes. In terms of connection rules, the Input node only has an output terminal and can be connected to multiple subsequent operator nodes or Output nodes at the same time. The data type contained in the Input node is numerical or categorical; the Output node is the final node of the data flow and can obtain the input data for the algorithm model. In terms of connection rules, the Output node only has an input terminal and is connected to one or more preceding Input nodes or operator nodes.

[0032] Operator nodes have a variety of different algorithm types. Operator nodes of different algorithm types contain corresponding feature transformation functions and their parameters to transform the input data. In terms of connection rules, an operator node has one or more input terminals and one or more output terminals; the input terminal of an operator node is connected to the corresponding Input node or the operator node of the previous level, and the output terminal is connected to the operator node of the next level or the Output node. The input data type of the operator node must be the same as the data type of the Input node;

[0033] Operator nodes are categorized by algorithm type into data scaling nodes, discretization nodes, vectorization nodes, function transformation nodes, and dimensionality reduction nodes. Data scaling nodes perform standardization or normalization operations on input data based on preset parameters. Normalization operations require setting two parameters: a maximum value and a minimum value. By default, 0 is used as the minimum value and 1 as the maximum value.

[0034] The discretization node performs equidistant bucketing, equal-frequency bucketing or binarization on the input data according to preset parameters; wherein, the parameter required to be set for equidistant bucketing and equal-frequency bucketing is the number of buckets, and the parameter required to be set for binarization is the threshold. The part of the input data greater than or equal to the threshold will be uniformly converted to 1, and the part less than the threshold will be uniformly converted to 0.

[0035] The vectorization node performs one-hot encoding or word vector transformation on the input data according to preset parameters. The word vector transformation corresponds to a word vector library.

[0036] The function transformation node performs logarithmic transformation and exponential transformation on the input data according to preset parameters, wherein the parameter required to be set for the logarithmic transformation is the base; the parameter required to be set for the exponential transformation is the exponent.

[0037] The dimensionality reduction node reduces the dimensionality of the input data by principal component analysis or tSNE (i.e., high-dimensional data dimensionality reduction visualization); wherein the parameter required to be set for principal component analysis is the number of principal components; the parameter required to be set for tSNE is the target dimension.

[0038] Operator nodes are categorized into two main types based on the computational steps involved: global-dependent and independent. For global-dependent operator nodes, the data transformation parameters depend on the entire column of data, requiring two consecutive passes through the column. The first pass, called fit, computes statistics for the column; the second pass, called transform, performs the actual transformation. If the transformation operator doesn't depend on the entire column of data, then only a single pass is required to directly execute the transform task.

[0039] The method for using the feature computation graph-based visual machine learning feature extraction system to perform dataset feature analysis is as follows:

[0040] Step 1: Users use the visual front-end interface to build a feature calculation graph by dragging different nodes based on the features they need to extract from their dataset. Each input node is connected to the same output node through one or more operator nodes. The type of each operator node corresponds to the features to be extracted from the user dataset, and the user selects the operator node based on their needs.

[0041] The user dataset adds an ID column before each Input node. The number of Input nodes corresponds to the number of columns in the user dataset. Each Input node takes the ID column and a corresponding column of elements from the user dataset. For operator nodes or Output nodes with multiple child nodes, the input data is the column-wise merge of the output data from each child node. Child nodes represent the operator nodes or Input nodes immediately preceding the current Output node or operator node.

[0042] Step 2: The data processing module uses the graph depth-first search method to calculate the user data set, visiting each node from the output node forward. The specific process is as follows:

[0043] 2-1. Starting from the Output node, the fit method of the child nodes is called step by step until the fit method is called on each Input node. The fit method calculates the statistics corresponding to the input data. Its input parameter is the original data set, and the return value is the node object itself. The purpose of the fit method is to calculate the statistical parameters of the data column. When the fit method is executed, the transform method of the child node is called. The transform method is the process of performing the data transformation corresponding to the node itself. Its input parameter is the original data set. It is responsible for performing the actual data transformation action and returns the transformed data set. The feature calculation graph is converted into a json object and stored on disk using a persistence method (called the persist method). Each time a node completes the fit method, a json object is constructed. Each json object is merged to form a total json file.

[0044] For the Output node, its fit method first calls the fit method of all child nodes, and then records the return value of each child node into an array. The corresponding JSON object contains the name of the child node and the return value of the child node;

[0045] For the data scaling node that performs standardization, its fit method first calls the fit and transform methods of all child nodes, then merges the output columns of each child node, and then calculates the mean and variance of the current data and records them.

[0046] For discretization nodes that perform equidistant or equifrequency bucketing, their fit method first calls the fit and transform methods of all child nodes and merges the output columns. It then determines the separation point based on the number of buckets and records it.

[0047] For a vectorized node that performs one-hot encoding, its fit method first calls the fit and transform methods of all child nodes and merges the output columns. It then calculates the encoding output dimensions and encoding table for each column and records them.

[0048] For a dimensionality reduction node that performs principal component analysis, its fit method first calls the fit and transform methods of all child nodes and merges the output columns, then executes the principal component analysis algorithm and records the transformation matrix.

[0049] For Input nodes, its fit method has no effect and directly returns its input data.

[0050] 2-2. Starting from the Output node, call the transform method of each child node one by one, so that the Output node outputs the result dataset after the required feature extraction. This completes the user-specified feature extraction task. The output result dataset can be directly input into the machine learning algorithm for training.

[0051] For the Output operator, its transform method calls the transform methods of all child nodes, and then merges the output columns to return as the result dataset.

[0052] For operator nodes that perform normalization, equal-distance bucketing, equal-frequency bucketing, one-hot encoding, principal component analysis, and tSNE, their transform methods first call the transform methods of all child nodes and merge the output columns. They then execute their own corresponding data transformation functions and return the resulting dataset.

[0053] For the Input node, its transform method extracts the id column and the specified column from the original dataset as output.

[0054] Step 3. When feature extraction is required for a new user dataset, use the JSON file obtained in step 2 to directly complete the fit method of each node. By only performing the transform method, the new user dataset can be completely transformed to obtain a new result dataset, thereby improving the efficiency of feature extraction.

[0055] Combine Figure 4 The data transformation process of the present invention is further explained as follows: Feature calculation Figure 1 It is a user-configured feature calculation scheme. During model training, the statistical parameters of each operator node are calculated in the fit phase to obtain the feature calculation Figure 2 , and then go through the transform stage to get the transformed feature data. During model inference, only the feature calculation of the saved state is needed. Figure 2 , transform the inference data to obtain the data input model for prediction.

Claims

1. A visual machine learning feature extraction system based on feature computation graph, characterized by: It includes a visual front-end interface and a data processing module. The visual front-end interface contains three node controls for building feature calculation graphs: Input nodes, operator nodes, and Output nodes. The data processing module is used to process user data sets using feature calculation graphs. Each Input node corresponds to a column of elements in the user data set. Operator nodes have a variety of different algorithm types. Operator nodes of different algorithm types contain corresponding feature transformation functions and their parameters, which are used to transform input data. In the initial visual front-end interface, multiple Input nodes consisting of column names and one Output node are displayed, and the operator toolbox is displayed on the visual front-end interface. The operator toolbox arranges various operator nodes in a classified manner. The feature calculation graph is composed of an Input node, one or more operator nodes, and an Output node. Each Input node is connected to the Output node through one or more operator nodes; each Input node corresponds to a column of data in the user dataset.

2. A visual machine learning feature extraction system based on feature computation graph according to claim 1, characterized in that: Classified by algorithm type, the operator nodes include data scaling nodes, discretization nodes, vectorization nodes, function transformation nodes and dimensionality reduction nodes; the data scaling node performs standardization or normalization operations on the input data according to preset parameters; the discretization node performs equidistant bucketing, equal-frequency bucketing or binarization on the input data according to preset parameters; wherein, the parameter required to be set for equidistant bucketing and equal-frequency bucketing is the number of buckets, and the parameter required to be set for binarization is the threshold, and the part of the input data greater than or equal to the threshold will be uniformly transformed into 1, and the part less than the threshold will be uniformly transformed into 0; the vectorization node performs one-hot encoding or word vector transformation on the input data according to preset parameters; wherein, the word vector transformation corresponds to a word vector library; the function transformation node performs logarithmic transformation and exponential transformation on the input data according to preset parameters, wherein the parameter required to be set for logarithmic transformation is the base; the parameter required to be set for exponential transformation is the exponent; the dimensionality reduction node reduces the dimensionality of the input data through principal component analysis or tSNE; wherein, the parameter required to be set for principal component analysis is the number of principal components; the parameter required to be set for tSNE is the target dimension.

3. The visual machine learning feature extraction system based on feature computation graph according to claim 1, characterized in that: The data type of the Input node is a numeric type or a categorical type; the input data type of the operator node is the same as the data type of the Input node.

4. The visual machine learning feature extraction system based on feature computation graph according to claim 1, characterized in that: The user data set has an automatically generated ID column as a row identifier.

5. The visual machine learning feature extraction system based on feature computation graph according to claim 1, characterized in that: An input node has one or more output terminals for connecting to one or more subsequent nodes; an output node has one or more input terminals; an operator node has one or more input terminals and one or more output terminals; the input terminal of an operator node is connected to the corresponding input node or the operator node of the previous level, and the output terminal is connected to the operator node of the next level or the output node.

6. A visual machine learning feature extraction method based on feature computation graph, characterized by: Step 1: The user uses the visual machine learning feature extraction system according to claim 1 to build a feature calculation graph by dragging different nodes according to the feature extraction target; Each Input node is connected to the same Output node through one or more Operator nodes. The type of each Operator node corresponds to the features that need to be extracted from the user dataset and is selected by the user according to their needs. The user dataset adds an ID column before each Input node. The number of Input nodes corresponds to the number of columns in the user dataset. Each Input node inputs the ID column of the user dataset and a corresponding column of elements. For an operator node or Output node with multiple child nodes, its input data is the data obtained by merging the output data of each child node by column. The child nodes represent the operator nodes or Input nodes at the previous level of the current Output node or operator node. Step 2: The data processing module uses the graph depth-first search method to calculate the user data set, visiting each node from the output node forward. The specific process is as follows: 2-1. Starting from the Output node, the fit method of the child nodes is called step by step until the fit method is called on each Input node. The fit method represents the process of calculating the statistics corresponding to the input data of the node. When the fit method is executed, the transform method of the child node is called. The transform method represents the process of performing the data transformation corresponding to the node itself. 2-2. Starting from the Output node, call the transform method of the child nodes step by step, so that the Output node outputs the result data set after feature extraction.

7. The method for visual machine learning feature extraction based on feature computation graph according to claim 6, characterized in that: When the feature calculation graph is constructed and each node completes the fit method, a JSON object is constructed; each JSON object is merged to form a total JSON file; after step 2 is executed, if feature extraction is required for a new user dataset, the JSON file is used to directly complete the fit method part of each node, and only the transform method is performed.

Images