Feature processing method, generation method and device based on feature engineering platform

Abstract

The present disclosure provides a feature processing method based on a feature engineering platform, and relates to the fields of artificial intelligence, machine learning and data processing. The specific implementation scheme is as follows: N feature engineering nodes are displayed on a visual interface of the feature engineering platform, and a first object list is displayed, wherein a plurality of objects included in the first object list each correspond to a plurality of data sequences; the i+1th object list is displayed by: in response to an operation of selecting at least one object from the i th object list, taking the at least one object as an input object of the i th feature engineering node, determining an output object of the i th feature engineering node; determining the i+1th object list according to the output object of the i th feature engineering node; displaying the i+1th object list, i=1,......,N-1; and determining a data structure of feature engineering of the plurality of data sequences according to the N th object list. The present disclosure also provides a feature generation method and device, an electronic device and a storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of machine learning and data processing technology. It can be applied to artificial intelligence development platforms, end-to-end enterprise-level artificial intelligence platforms, machine learning platforms, etc. More specifically, this disclosure provides a feature processing method based on a feature engineering platform, a feature generation method based on a feature engineering platform, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] Currently, machine learning is developing towards greater ease of use, lower technical barriers, and lower development costs. For structured data modeling, feature engineering is extremely time-consuming yet crucial, often being the most critical step in determining model performance. Summary of the Invention

[0003] This disclosure provides a feature processing method, a feature generation method, apparatus, device, and storage medium based on a feature engineering platform.

[0004] According to the first aspect, a feature processing method based on a feature engineering platform is provided. The feature engineering platform's visual interface displays N feature engineering nodes, where N is an integer greater than 1. The method includes: displaying a first object list, wherein the first object list contains multiple objects that correspond to multiple data sequences; displaying an (i+1)th object list through the following operations: in response to the operation of selecting at least one object from the i-th object list, using at least one object as the input object of the i-th feature engineering node, and determining the output object of the i-th feature engineering node; determining an (i+1)th object list based on the output object of the i-th feature engineering node; displaying the (i+1)th object list, where i = 1, ..., N-1; and determining the data structure for feature engineering of the multiple data sequences based on the N-th object list.

[0005] According to the second aspect, a feature generation method based on a feature engineering platform is provided. The visual interface of the feature engineering platform displays N feature engineering nodes, an input object for each feature engineering node, and an object list, where N is an integer greater than 1. The input objects are selected according to the aforementioned feature processing method based on the feature engineering platform, and the object list is generated according to the aforementioned feature processing method based on the feature engineering platform. The method includes: acquiring multiple data sequences, wherein each data sequence corresponds to a multiple object in the object list of the first feature engineering node, and the data structure of each data sequence is consistent with the corresponding object; responding to an instruction for instructing feature generation, for each feature engineering node, generating a feature sequence of the multiple data sequences at that feature engineering node based on the input object of that feature engineering node, wherein the data structure of the feature sequence of the multiple data sequences at each feature engineering node is consistent with the object in the object list of that feature engineering node; and determining the feature sequence of the multiple data sequences at the Nth feature engineering node as the feature engineering of the multiple data sequences.

[0006] According to a third aspect, a feature processing apparatus based on a feature engineering platform is provided. The visual interface of the feature engineering platform displays N feature engineering nodes, where N is an integer greater than 1. The apparatus includes: a first display module for displaying a first object list, wherein the first object list contains multiple objects that correspond to multiple data sequences; a second display module for displaying an (i+1)th object list, comprising: a first determining submodule for, in response to an operation of selecting at least one object from the i-th object list, using at least one object as an input object of the i-th feature engineering node and determining the output object of the i-th feature engineering node; a second determining submodule for determining the (i+1)th object list based on the output object of the i-th feature engineering node; a display submodule for displaying the (i+1)th object list, where i = 1, ..., N-1; and a first determining module for determining the data structure of the feature engineering for the multiple data sequences based on the N-th object list.

[0007] According to the fourth aspect, a feature generation apparatus based on a feature engineering platform is provided. The visual interface of the feature engineering platform displays N feature engineering nodes, an input object for each feature engineering node, and an object list, where N is an integer greater than 1. The input object is selected according to the feature processing device based on the feature engineering platform, and the object list is generated according to the feature processing device based on the feature engineering platform. The apparatus includes: an acquisition module for acquiring multiple data sequences, wherein each data sequence corresponds to a multiple object in the object list of the first feature engineering node, and the data structure of each data sequence is consistent with the corresponding object; a generation module for generating a feature sequence of the multiple data sequences at each feature engineering node, based on the input object of the feature engineering node, in response to an instruction to generate features; and a second determination module for determining the feature sequence of the multiple data sequences at the Nth feature engineering node as a feature engineering feature of the multiple data sequences.

[0008] According to a fifth aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to the present disclosure.

[0009] According to a sixth aspect, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods provided in this disclosure.

[0010] According to a seventh aspect, a computer program product is provided, comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program implementing the method provided in this disclosure when executed by a processor.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 This is an exemplary system architecture diagram illustrating a feature processing method and a feature generation method based on a feature engineering platform that can be applied according to an embodiment of the present disclosure;

[0014] Figure 2 This is a flowchart of a feature processing method based on a feature engineering platform according to an embodiment of the present disclosure;

[0015] Figure 3A This is a schematic diagram showing multiple feature engineering nodes on a visual interface of a feature engineering platform according to an embodiment of the present disclosure;

[0016] Figure 3B This is a schematic diagram of a list of feature engineering nodes according to an embodiment of the present disclosure;

[0017] Figure 4 This is a flowchart of a feature generation method based on a feature engineering platform according to an embodiment of the present disclosure;

[0018] Figure 5 This is a block diagram of a feature processing apparatus based on a feature engineering platform according to an embodiment of the present disclosure;

[0019] Figure 6 This is a block diagram of a feature generation apparatus based on a feature engineering platform according to an embodiment of the present disclosure;

[0020] Figure 7 This is a block diagram of an electronic device based on a feature processing method and / or a feature generation method based on a feature engineering platform according to an embodiment of the present disclosure. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] Many AI development platforms integrate feature engineering functionality, enabling users to create feature engineering steps in a visual and interactive manner. These steps cover data processing (e.g., filtering, column selection, column deletion), feature processing (e.g., scaling, normalization, OneHot encoding), feature derivation (e.g., feature crossing, feature hashing, multinomial expansion, timestamp feature derivation), and feature automation. For the original dataset, the resulting feature engineering project can be a set of features from the original dataset.

[0023] Each feature engineering step can be considered a feature engineering node. All feature engineering nodes (steps, operators) are chained together to form a feature engineering platform (e.g., a pipeline). Each node can select the feature columns to be processed in the current step. The output data of the previous node serves as the input data for the next node. The input data of the first node is the original dataset, and the output data of the last node is the feature engineering result.

[0024] Feature engineering nodes include column selection, column deletion, oversampling, undersampling, filtering, feature crossing, multinomial expansion, outlier handling, numerical replacement, data type conversion, rounding, max-min normalization, standardization, dummy coding, OneHot coding, sequential coding, scaling, timestamp derivation, feature anomaly smoothing, binning, feature hashing, etc.

[0025] In the feature engineering platform, before the feature engineering node is actually executed, subsequent nodes cannot know the output of the preceding node (including the output column names and data types). In other words, the columns and data types that subsequent nodes can select can only be determined after the preceding node has actually run.

[0026] Therefore, feature engineering nodes can only be executed step by step. Based on the results of the previous nodes, the complete input for subsequent nodes is determined, and then the feature input columns are selected. It is impossible to build a complete feature project containing the processing results of multiple nodes at once. Single-step execution and single-step debugging are very inefficient.

[0027] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0029] Figure 1 This is a schematic diagram of an exemplary system architecture according to an embodiment of the present disclosure, illustrating a feature processing method and a feature generation method based on a feature engineering platform. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0030] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0031] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices, including but not limited to smartphones, tablets, laptops, etc.

[0032] The feature processing method based on the feature engineering platform provided in this disclosure can generally be executed by terminal devices 101, 102, and 103. Accordingly, the feature processing device based on the feature engineering platform provided in this disclosure can generally be disposed in terminal devices 101, 102, and 103.

[0033] The feature generation method based on the feature engineering platform provided in this disclosure can generally be executed by the server 105. Accordingly, the feature generation device based on the feature engineering platform provided in this disclosure can generally be located in the server 105.

[0034] Figure 2 This is a flowchart of a feature processing method based on a feature engineering platform according to an embodiment of the present disclosure.

[0035] The feature engineering platform's visual interface displays N feature engineering nodes, where N is an integer greater than 1. These N feature engineering nodes may include, for example, filtering nodes, sampling nodes, normalization nodes, column deletion nodes, etc. Users can add these N feature engineering nodes through the visual interface, and can also adjust their order.

[0036] This feature engineering platform can construct feature engineering for the original dataset, which may include multiple data sequences (multiple columns of data). The data structure (schema) of each column of data includes column identifiers (e.g., column names) and data types (e.g., bool, int, double, string, etc.).

[0037] like Figure 2 As shown, the feature processing method 200 based on the feature engineering platform may include operations S210 to S230. Operation S220 includes operations S221 to S223.

[0038] In operation S210, the first object list is displayed, which contains multiple objects and multiple data sequences, each corresponding to one another.

[0039] In operation S220, the list of the (i+1)th object is displayed through operations S221 to S223.

[0040] In operation S221, in response to the operation of selecting at least one object from the list of objects of the i-th type, at least one object is used as the input object of the i-th feature engineering node, and the output object of the i-th feature engineering node is determined.

[0041] In operation S222, the list of objects of the (i+1)th object is determined based on the output object of the i-th feature engineering node.

[0042] In operation S223, the list of the (i+1)th object is displayed.

[0043] In operation S230, the data structure of the feature engineering of multiple data sequences is determined based on the Nth object list.

[0044] For example, the first object list contains multiple objects, each corresponding to a different data sequence in the original dataset. The original dataset can be the dataset for which feature engineering is to be performed, and the resulting feature engineering of the original dataset can be a collection of features from the multiple data sequences in the original dataset. The data structure schema for each data sequence in the original dataset includes an identifier and a data type. This data structure schema can be abstracted as an object, which also includes an identifier and a data type. Objects in the object list can be object identifiers, and each object in the object list has a corresponding data type.

[0045] For example, when i = 1, ..., N-1, and i = 1, the first object list can serve as the selectable list for the first feature engineering node. The user can select at least one object from this first object list as the input object for the first feature engineering node. This first feature engineering node can be configured with information that allows the system to infer how many new objects will be generated by the selected objects, the types of the new objects, and the generation of new identifiers for the new objects. In other words, by selecting the input object for the first feature engineering node before it actually runs, the system can automatically infer the output object of the first feature engineering node.

[0046] For example, the first feature engineering node is a OneHot encoding node, and the first object list is {a, b, c, d}. The user selects a and b as the input objects for the first feature engineering node, which can produce output objects {a_onehot, b_onehot}. The data type of the output objects {a_onehot, b_onehot} is int. a_onehot and b_onehot are the identifiers of the output objects.

[0047] Therefore, before the first feature engineering node is actually run, the input and output objects of the first feature engineering node can be displayed intuitively and quickly on the feature engineering platform. Furthermore, users can obtain the desired output object by adjusting the input object of the first feature engineering node.

[0048] According to an embodiment of this disclosure, operation S222 specifically includes taking the output object of the i-th feature engineering node and the object list composed of objects in the i-th object list other than the input object of the i-th feature engineering node as the (i+1)-th object list; or adding the output object of the i-th feature engineering node to the i-th object list to obtain a new object list as the (i+1)-th object list.

[0049] For example, after obtaining the output objects of the first feature engineering node (e.g., {a_onehot, b_onehot}), a second object list can be generated. For instance, the second object list can be a combination of the output objects {a_onehot, b_onehot} of the first feature engineering node and the objects {c, d} in the object list excluding the input objects, forming the list {a_onehot, b_onehot, c, d}. Alternatively, the output objects {a_onehot, b_onehot} of the first feature engineering node can be added to the first object list {a, b, c, d}, resulting in a new object list {a, b, c, d, a_onehot, b_onehot}, which can then be used as the second object list.

[0050] For example, the second object list can be displayed on the visual interface of the feature engineering platform. The second object list can serve as a selectable list for the second feature engineering node. Similar to the above operation, the user can select at least one object from the second object list as the input object for the second feature engineering node, obtain the output object of the second feature engineering node, determine the third object list, and display the third object list on the visual interface of the feature engineering platform.

[0051] Similarly, N object lists can be displayed sequentially. Users can select at least one object from each displayed object list as the input object for the corresponding feature engineering node, which can then generate an output object.

[0052] Therefore, the output schema (including output column names and data types) of each node can be automatically inferred before the entire feature engineering node is actually executed. Thus, the data structures of multiple objects in the Nth object list form the data structure of the feature engineering of the original dataset, and the inference result is consistent with the actual running result.

[0053] In this embodiment of the disclosure, the input and output objects of each feature engineering node can be displayed intuitively and quickly on the feature engineering platform before the actual operation of the entire feature engineering process. By adjusting the input object of any feature engineering node, the inference of all feature engineering nodes is automatically completed, thereby enabling the construction of the feature engineering data structure of the original dataset in one go.

[0054] The inference process of feature engineering nodes in this embodiment can be implemented entirely by the front end, with minimal computational load, no reliance on backend servers, no server latency, and a superior user experience. Even without server computing resources, users can complete the initial construction of feature engineering.

[0055] According to an embodiment of this disclosure, operation S221 specifically includes determining the number of output objects of the i-th feature engineering node based on the inference type of the i-th feature engineering node; determining the identifier of the output objects of the i-th feature engineering node based on the suffix attribute of the i-th feature engineering node; and determining the data type of the output objects of the i-th feature engineering node based on the output type attribute of the i-th feature engineering node.

[0056] For example, to enable each feature engineering node to automatically infer the output object, you can configure the inference type, suffix attribute, and output type attribute for each feature engineering node.

[0057] For example, based on the inference type of a feature-engineered node, the number of output objects can be determined from the number of input objects to the feature-engineered node. Adding the feature-engineered node's suffix to the identifier of the input objects yields the identifier of the output objects. The output type of the feature-engineered node can be assigned to the output objects as their data type.

[0058] The inference type (SchemaInferType) can include the first type (Filter), the second type (Equal), the third type (Complement), the fourth type (Map), the fifth type (Reduce), and the sixth type (FlatMap).

[0059] The ColSuffix attribute allows you to customize the suffix for each feature engineering node. For example, you can define the suffix for a OneHot encoded node as "onehot" and the suffix for a feature cross node as "cross", etc.

[0060] The OutputType attribute allows you to customize the output type for each feature engineering node, specifying the data type of the output object. The OutputType attribute can include standard data types such as bool, int, double, and string, as well as array. <int>、array <string>Composite data types, and origin representing primitive types.

[0061] To facilitate the explanation of the inference type, suffix attribute, and output type attribute provided in this disclosure, the following concepts are explained first.

[0062] Initial columns (all_cols): All columns of the original dataset;

[0063] Feature input columns (input_cols): The input objects selected by the current feature engineering node;

[0064] Feature output columns (output_cols): The output objects generated by the current feature engineering node after processing the input objects;

[0065] The result column (result_cols) is the complete output column of the current feature engineering node, which is the object list of the next feature engineering node. It contains the output objects of the current feature engineering node and the object list of the current feature engineering node (or the objects in the object list that are not selected).

[0066] Table 1 shows the inference type, suffix attribute, and output type attribute information according to embodiments of this disclosure.

[0067] Table 1

[0068]

[0069]

[0070] This embodiment provides configuration information for various feature engineering nodes, including inference type, suffix attribute, and output type attribute. This enables each feature engineering node to automatically infer the output object based on the input object before actual operation, thus achieving the initial construction of the feature engineering data structure.

[0071] For example, nodes such as filtering and sampling, which belong to the first type of inference (Filter), generally need to process all columns of the original dataset. Therefore, the result columns are equal to the initial columns, and the schema of the feature output columns remains unchanged. For example, the initial columns are {a, b, c, d}, the feature input columns are {a, b, c, d}, the feature output columns are {a, b, c, d}, and the result columns are {a, b, c, d}.

[0072] The column selection node, belonging to the second type (Equal), selects the columns to be processed for features. The selected columns (feature input columns) participate in the processing of subsequent feature engineering nodes, so the result column is equal to the feature input column. For example, the initial columns are {a, b, c, d}, the feature input columns are {a, b, c}, the feature output columns are {a, b, c}, and the result column is {a, b, c}.

[0073] The deletion node, belonging to the third type (Complement), deletes the feature input column. Therefore, the result column is the difference between the initial column and the feature input column, and the schema of the feature output column remains unchanged. For example, the initial column is {a, b, c, d}, the feature input column is {d}, the feature output column is {a, b, c}, and the result column is {a, b, c}.

[0074] The schema of the feature output columns of the feature engineering nodes of the first type (Filter), the second type (Equal), and the third type (Complement) remains unchanged, and the result columns are not determined based on the feature output columns, but on the meaning of the operation. Therefore, the feature engineering nodes of the first type (Filter), the second type (Equal), and the third type (Complement) can be regarded as special cases.

[0075] The inference type belongs to the fourth type (Map) of nodes such as outlier handling, normalization, standardization, and encoding, and further belongs to the one-to-one mapping type. According to the embodiments of this disclosure, for the i-th feature engineering node belonging to the one-to-one mapping type, the identifier of the output object of the i-th feature engineering node is determined according to the identifier of the input object of the i-th feature engineering node and the suffix attribute of the i-th feature engineering node.

[0076] For example, for a OneHot encoded node belonging to the fourth type (Map), a custom suffix "onehot" is used, and a custom output type "int" is used. If the selectable columns (object list) are {a, b, c, d}, and the feature input columns are {a, b}, then the feature output columns are {a_onehot, b_onehot}, the data type of the feature output columns is {int, int}, and the result columns are {a, b, c, d, a_onehot, b_onehot}, where a, b, c, and d in the result columns retain their existing column names and data types.

[0077] This embodiment targets feature engineering nodes of one-to-one mapping type, and can automatically generate the identifier and data type of the output object. The identifier information of the output object can include feature engineering information (for example, object a_onehot contains onehot information), which can make the processing of the output object clearer.

[0078] The inference type belongs to the fifth type (Reduce), including feature cross, multinomial expansion, feature hash, and other nodes, which further belong to the many-to-one mapping type. According to an embodiment of this disclosure, for the i-th feature engineering node belonging to the many-to-one mapping type, the identifier of one output object of the i-th feature engineering node is determined based on the identifiers of the multiple input objects of the i-th feature engineering node and the suffix attribute of the i-th feature engineering node.

[0079] For example, for feature cross nodes belonging to the fifth type (Reduce), a custom suffix "cross" is used, and a custom output type "string" is used. If the selectable columns are {a, b, c, d}, and the feature input columns are {a, b, c}, then the feature output column is {a_b_c_cross}, the data type of the feature output column is {string}, and the result column is {a, b, c, d, a_b_c_cross}, where a, b, c, and d in the result column retain the existing column names and data types.

[0080] This embodiment targets feature engineering nodes of many-to-one mapping type, which can automatically determine the number, identifier and data type of output objects, and the identifier information of output objects can include feature engineering information (for example, object a_b_c_cross contains cross information), which can make the processing of the output object clearer.

[0081] The inferred type belongs to the sixth type (FlatMap) of timestamp-derived nodes, which further belongs to the one-to-many mapping type. According to an embodiment of this disclosure, for the i-th feature-engineered node belonging to the many-to-one mapping type, the identifiers of the multiple output objects of the i-th feature-engineered node are determined based on the identifier of one input object of the i-th feature-engineered node and the suffix attribute of the i-th feature-engineered node.

[0082] For example, for timestamp-derived nodes belonging to the sixth type (FlatMap), custom suffixes include ["year", "month", "day", "hour", "minute", "second", "season"], and custom output types include ["int", "int", "int", "int", "int", "int", "string"]. If the selectable columns are {a, b, c, d}, and the feature input column is {a}, where 'a' contains time information, then the feature output columns are {a_year, a_month, a_day, a_hour, a_minute, a_second, a_season}, the data type of the feature output columns is {int, int, int, int, int, int, string}, and the result columns are {a, b, c, d, a_year, a_month, a_day, a_hour, a_minute, a_second, a_season}, where a, b, c, and d in the result columns retain their existing column names and data types.

[0083] This embodiment targets feature engineering nodes of one-to-many mapping type, and can automatically determine the number, identifier, and data type of output objects. Furthermore, the identifier information of the output objects can include feature engineering information (e.g., object a_year contains year information), which can make the processing of the output objects clearer.

[0084] Table 2 shows the inference type, suffix attribute, and output type attribute information of each feature engineering node according to embodiments of the present disclosure.

[0085] Table 2

[0086]

[0087]

[0088] The following is combined with Figures 3A-3B This document describes the display content of the visual interface of the feature engineering platform provided in this disclosure.

[0089] Figure 3A This is a schematic diagram showing multiple feature engineering nodes on a visual interface of a feature engineering platform according to an embodiment of the present disclosure.

[0090] like Figure 3A As shown, multiple feature engineering nodes include a maximum-minimum normalization node, a data type conversion node, a OneHot encoding node, and a feature cross node. Users can also add new nodes according to their actual needs.

[0091] like Figure 3A As shown, each feature engineering node can select feature input columns (input objects). For example, the max-min normalization node selects {slen, swid} as feature input columns for max-min normalization, where "slen" and "swid" are column names (input object identifiers). Similarly, the data type conversion node selects {plen, pwid} as feature input columns for data type conversion, where "plen" and "pwid" are column names (input object identifiers). The OneHot encoding node selects {class} as a feature input column for OneHot encoding, where "class" is a column name (input object identifier). The feature cross node selects {slen, swid} as feature input columns for feature cross encoding, where "slen" and "swid" are column names (input object identifiers).

[0092] Figure 3B This is a schematic diagram of a list of feature engineering nodes according to an embodiment of the present disclosure.

[0093] like Figure 3B As shown, after expanding the data type conversion node, the user can see a list of selectable objects: {slen, swid, plen, pwid, class, slen_norm}, where "slen_norm" is the output object of the previous feature engineering node's max-min normalization node. For example, the user can select at least one object from this list as an input object (feature input column).

[0094] This disclosure provides a visual representation of the object list for each feature engineering node, facilitating user selection of input objects and intuitive observation of output objects. Furthermore, the input objects of any feature engineering node can be adjusted according to actual needs to obtain the desired output object.

[0095] Figure 4 This is a flowchart of a feature generation method based on a feature engineering platform according to an embodiment of the present disclosure.

[0096] like Figure 4 As shown, the feature generation method 400 based on the feature engineering platform includes operations S410 to S420. The visual interface of the feature engineering platform displays N feature engineering nodes and an object list for each feature engineering node, where N is an integer greater than 1. The object list is generated according to the feature processing method based on the feature engineering platform described above.

[0097] During operation of S410, multiple data sequences are acquired.

[0098] In operation S420, in response to the instruction for instructing the generation of features, for each feature engineering node, a feature sequence of multiple data sequences is generated at that feature engineering node based on the input object of that feature engineering node.

[0099] In operation S430, the feature sequence of multiple data sequences at the Nth feature engineering node is determined as the feature engineering of multiple data sequences.

[0100] For example, N feature engineering nodes include filtering nodes, sampling nodes, normalization nodes, column deletion nodes, etc. Multiple data sequences can be multiple columns of data from the original dataset, which is the dataset to be feature-engineered. The feature engineering of the resulting original dataset can be a set of features from multiple data sequences in the original dataset.

[0101] For example, multiple data sequences correspond to multiple objects in the object list of the first feature engineering node. The data structure schema of each data sequence includes an identifier and a data type, and the data structure of each data sequence is consistent with the corresponding object. For example, in the first object list {a, b, c, d}, object a represents the identifier of the first data sequence, object b represents the identifier of the second data sequence, object c represents the identifier of the third data sequence, and object d represents the identifier of the fourth data sequence.

[0102] For example, each feature engineering node's object list contains the objects that the feature engineering node can select. The selected objects serve as the input objects for the feature engineering node, which can then generate output objects based on these input objects. Furthermore, the next feature engineering node's object list can be generated based on the output objects. In other words, before the feature engineering nodes are actually executed, the output objects (including output column names and data types) of each node can be automatically inferred. Thus, the data structures of multiple objects in the Nth object list form the feature engineering data structure of the original dataset, and the inference result is consistent with the actual execution result.

[0103] For example, users can infer results based on each feature engineering node and manually adjust the input objects to achieve their desired effect and output. Furthermore, users can complete the entire feature engineering process in one go and submit it to the backend server for actual feature engineering execution.

[0104] For example, in response to a user's instruction to generate features (e.g., the user selects to run an operation on the visual interface of the feature engineering platform), the server can sequentially execute the actual operation of feature engineering for multiple data sequences of the original dataset according to the order of multiple feature engineering nodes and the selected input object, thereby generating feature sequences of multiple data sequences of the original dataset at each feature engineering node.

[0105] For example, at the i-th feature engineering node, based on the input object of the i-th feature engineering node, multiple data sequences can be generated. The feature sequence at the i-th feature engineering node can be called the i-th feature sequence. The data structure of the i-th feature sequence is consistent with the objects in the object list of the i-th feature engineering node. For example, the object list of the i-th feature engineering node is {a_onehot, b_onehot, c, d}, where object a_onehot is the identifier of the first feature in the i-th feature sequence, object b_onehot is the identifier of the second feature in the i-th feature sequence, object a is the identifier of the third feature in the i-th feature sequence, and object b is the identifier of the fourth feature in the i-th feature sequence. i = 1, ..., N.

[0106] For example, the feature sequence of multiple data sequences in the original dataset at the Nth feature engineering node is determined as the feature engineering of the original dataset, and the data structure of the feature engineering of the original dataset is consistent with the objects in the inferred object list of the Nth feature engineering node.

[0107] In this embodiment, the entire feature engineering is constructed in one go and submitted to the backend server for actual execution. The debugging results are consistent with the actual running results, which can greatly save debugging time.

[0108] Figure 5 This is a block diagram of a feature processing apparatus based on a feature engineering platform according to an embodiment of the present disclosure.

[0109] like Figure 5 As shown, the feature processing device 500 based on the feature engineering platform includes a first display module 510, a second display module 520, and a first determination module 530. The second display module 520 includes a first determination submodule 521, a second determination submodule 522, and a display submodule 523.

[0110] The first display module 510 is used to display the first object list, wherein the first object list contains multiple objects that correspond to multiple data sequences.

[0111] The second display module 520 is used to display the (i+1)th object list.

[0112] The first determining submodule 521 is configured to, in response to an operation of selecting at least one object from the i-th object list, use at least one object as the input object of the i-th feature engineering node and determine the output object of the i-th feature engineering node.

[0113] The second determining submodule 522 is used to determine the (i+1)th object list based on the output object of the i-th feature engineering node.

[0114] The display submodule 523 is used to display the list of the (i+1)th object, where i = 1, ..., N-1.

[0115] The first determining module 530 is used to determine the data structure of the feature engineering of multiple data sequences based on the Nth object list.

[0116] The second determining submodule 522 is used to take the output object of the i-th feature engineering node and the object list composed of the i-th object list excluding the input object of the i-th feature engineering node as the (i+1)-th object list; or add the output object of the i-th feature engineering node to the i-th object list to obtain a new object list, which is then used as the (i+1)-th object list.

[0117] According to embodiments of this disclosure, the i-th feature engineering node has preset inference type, suffix attribute, and output type attribute.

[0118] The first determining submodule 521 includes a first determining unit, a second determining unit, and a third determining unit.

[0119] The first determining unit is used to determine the number of output objects of the i-th feature engineering node based on the inference type of the i-th feature engineering node.

[0120] The second determining unit is used to determine the identifier of the output object of the i-th feature engineering node based on the suffix attribute of the i-th feature engineering node.

[0121] The third determining unit is used to determine the data type of the output object of the i-th feature engineering node based on the output type attribute of the i-th feature engineering node.

[0122] The inference type includes a one-to-one mapping type. The second determining unit is used to determine the identifier of the output object of the i-th feature engineering node, which belongs to the one-to-one mapping type, based on the identifier of the input object of the i-th feature engineering node and the suffix attribute of the i-th feature engineering node.

[0123] The inference type includes many-to-one mapping type. The second determining unit is used to determine the identifier of one output object of the i-th feature engineering node, which belongs to the many-to-one mapping type, based on the identifiers of multiple input objects of the i-th feature engineering node and the suffix attribute of the i-th feature engineering node.

[0124] The inference type includes one-to-many mapping type. The second determining unit is used to determine the identifiers of multiple output objects of the i-th feature engineering node based on the identifier of one input object of the i-th feature engineering node and the suffix attribute of the i-th feature engineering node.

[0125] Figure 6 This is a block diagram of a feature generation apparatus based on a feature engineering platform according to an embodiment of the present disclosure.

[0126] The visual interface of the feature engineering platform displays N feature engineering nodes, as well as the input object and object list for each feature engineering node. N is an integer greater than 1. The input object is selected according to the feature processing device 500 based on the feature engineering platform, and the object list is generated according to the feature processing device 500 based on the feature engineering platform.

[0127] like Figure 6 As shown, the feature generation device 600 based on the feature engineering platform may include an acquisition module 610, a generation module 620, and a second determination module 630.

[0128] The acquisition module 610 is used to acquire multiple data sequences.

[0129] The generation module 620 is configured to, in response to an instruction indicating the generation of features, generate, for each feature engineering node, a feature sequence of multiple data sequences at that feature engineering node based on the input object of that feature engineering node, wherein the data structure of the feature sequence of the multiple data sequences at each feature engineering node is consistent with the objects in the object list of that feature engineering node.

[0130] The second determining module 630 is used to determine the feature sequence of multiple data sequences at the Nth feature engineering node as the feature engineering of multiple data sequences.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0132] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0134] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as feature processing methods and / or feature generation methods based on a feature engineering platform. For example, in some embodiments, the feature processing methods and / or feature generation methods based on a feature engineering platform can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the feature processing methods and / or feature generation methods based on a feature engineering platform described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured, by any other suitable means (e.g., by means of firmware), to perform feature processing methods and / or feature generation methods based on a feature engineering platform.

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0141] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0142] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.< / string> < / int>

Claims

1. A feature processing method based on a feature engineering platform, wherein a visual interface of the feature engineering platform displays N feature engineering nodes, N being an integer greater than 1; the method comprising: displaying a first object list, wherein the first object list contains objects corresponding to data sequences respectively; displaying an i+1th object list, obtaining N object lists by: in response to an operation of selecting at least one object from an ith object list, determining a number of output objects of the ith feature engineering node according to an inference type of the ith feature engineering node; determining an identifier of the output objects of the ith feature engineering node according to a suffix name attribute of the ith feature engineering node; determining a data type of the output objects of the ith feature engineering node according to an output type attribute of the ith feature engineering node; forming an object list comprising the output objects of the ith feature engineering node and objects in the ith object list except for input objects of the ith feature engineering node, and taking the object list as the i+1th object list; or adding the output objects of the ith feature engineering node to the ith object list to obtain a new object list, and taking the new object list as the i+1th object list; displaying the i+1th object list, wherein i=1,……,N-1; and determining a data structure of feature engineering of the data sequences according to an Nth object list.

2. The method of claim 1, wherein, The inference type comprises a one-to-one mapping type; the determining of the identifier of the output objects of the ith feature engineering node according to the suffix name attribute of the ith feature engineering node comprises: for the ith feature engineering node belonging to the one-to-one mapping type, determining the identifier of the output objects of the ith feature engineering node according to an identifier of an input object of the ith feature engineering node and the suffix name attribute of the ith feature engineering node. The inference type comprises a many-to-one mapping type; the determining of the identifier of the output objects of the ith feature engineering node according to the suffix name attribute of the ith feature engineering node comprises: for the ith feature engineering node belonging to the many-to-one mapping type, determining the identifier of one output object of the ith feature engineering node according to identifiers of multiple input objects of the ith feature engineering node and the suffix name attribute of the ith feature engineering node.

3. The method of claim 1, wherein, The inference type comprises a one-to-many mapping type; the determining of the identifier of the output objects of the ith feature engineering node according to the suffix name attribute of the ith feature engineering node comprises: for the ith feature engineering node belonging to the one-to-many mapping type, determining identifiers of multiple output objects of the ith feature engineering node according to an identifier of one input object of the ith feature engineering node and the suffix name attribute of the ith feature engineering node. The feature generation method comprises:

4. The method of claim 1, wherein, ​ ​ 5. A feature generation method based on a feature engineering platform, wherein a visual interface of the feature engineering platform displays N feature engineering nodes, input objects and object lists of each feature engineering node, N is an integer greater than 1, the input objects are selected according to any one of the methods in claims 1-4, and the object lists are generated according to any one of the methods in claims 1-4. ​ obtaining a plurality of data sequences, wherein the plurality of data sequences respectively correspond to a plurality of objects in an object list of a first feature engineering node, and a data structure of each data sequence is consistent with the corresponding object; in response to an instruction for indicating generation of a feature, for each feature engineering node, generating a feature sequence of the plurality of data sequences at the feature engineering node according to an input object of the feature engineering node, wherein a data structure of the feature sequence of the plurality of data sequences at each feature engineering node is consistent with the object in the object list of the feature engineering node; and determining the feature sequence of the plurality of data sequences at an Nth feature engineering node as a feature engineering of the plurality of data sequences.

6. A feature processing apparatus based on a feature engineering platform, a visual interface of the feature engineering platform displaying N feature engineering nodes, N being an integer greater than 1; the apparatus comprising: a first display module configured to display a first object list, wherein the first object list contains a plurality of objects respectively corresponding to a plurality of data sequences; a second display module configured to display an (i+1)th object list, comprising: a first determination submodule configured to, in response to an operation of selecting at least one object from an ith object list, determine the at least one object as an input object of the ith feature engineering node and determine an output object of the ith feature engineering node; a second determination submodule configured to determine an (i+1)th object list according to the output object of the ith feature engineering node; a display submodule configured to display the (i+1)th object list, wherein i = 1, …, N-1; and a first determination module configured to determine a data structure of a feature engineering of the plurality of data sequences according to an Nth object list; wherein the first determination submodule comprises: a first determination unit configured to determine a number of the output object of the ith feature engineering node according to an inference type of the ith feature engineering node; a second determination unit configured to determine an identifier of the output object of the ith feature engineering node according to a suffix name attribute of the ith feature engineering node; a third determination unit configured to determine a data type of the output object of the ith feature engineering node according to an output type attribute of the ith feature engineering node; the second determination submodule is further configured to group the output object of the ith feature engineering node and objects in the ith object list except the input object of the ith feature engineering node to form an object list, and take the object list as the (i+1)th object list; or add the output object of the ith feature engineering node to the ith object list to obtain a new object list, and take the new object list as the (i+1)th object list.

7. The apparatus of claim 6, wherein, The inference type comprises a one-to-one mapping type; and the second determination unit is configured to, for the ith feature engineering node belonging to the one-to-one mapping type, determine the identifier of the output object of the ith feature engineering node according to an identifier of the input object of the ith feature engineering node and the suffix name attribute of the ith feature engineering node.

8. The apparatus of claim 6, wherein, The inference types include a many-to-one mapping type; the second determining unit is configured to determine, for an i-th feature engineering node belonging to the many-to-one mapping type, an identifier of one output object of the i-th feature engineering node according to identifiers of a plurality of input objects of the i-th feature engineering node and a suffix name attribute of the i-th feature engineering node.

9. The apparatus of claim 6, wherein, The inference types include a one-to-many mapping type; the second determining unit is configured to determine, according to an identifier of one input object of the i-th feature engineering node and the suffix name attribute of the i-th feature engineering node, identifiers of a plurality of output objects of the i-th feature engineering node.

10. A feature generation device based on a feature engineering platform, a visual interface of the feature engineering platform displaying N feature engineering nodes and an input object and an object list of each feature engineering node, N being an integer greater than 1, the input object being selected by the device according to any one of claims 6 to 9, the object list being generated by the device according to any one of claims 6 to 9. The feature generation apparatus comprises: an acquisition module configured to acquire a plurality of data sequences, wherein the plurality of data sequences respectively correspond to a plurality of objects in an object list of a first feature engineering node, and a data structure of each data sequence is consistent with the corresponding object; a generation module configured to, in response to an instruction for instructing generation of a feature, generate, for each feature engineering node, a feature sequence of the plurality of data sequences at the feature engineering node according to input objects of the feature engineering node, wherein a data structure of the feature sequence of the plurality of data sequences at each feature engineering node is consistent with an object in an object list of the feature engineering node; and a second determining module configured to determine a feature sequence of the plurality of data sequences at an N-th feature engineering node as a feature engineering of the plurality of data sequences. 11.An electronic device comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5. The computer instructions are used to make the computer execute the method according to any one of claims 1 to 5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, 13.A computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program, when executed by a processor, implements the method according to any one of claims 1 to 5. ​

Images