Feature processing method, device, storage medium and electronic device

Abstract

The present invention discloses a feature processing method, device, storage medium and electronic device. The method comprises: reading a feature group and a feature group logic of a feature group in a database to obtain the feature logic of each feature in the feature group; when the feature group includes multiple features with the same feature logic, loading the feature logic of a first feature among the multiple features, and reading the feature data of the dependent features of the loaded first feature to obtain the loading result; using the loading result as the loading result of each feature among the multiple features; and outputting each feature among the multiple features and the loading result of each feature to a task structure. The present invention solves the technical problem of low efficiency in processing features.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a feature processing method, device, storage medium and electronic device. Background Art

[0002] Model training requires the support of feature data. As the use of models becomes more and more widespread, the use of feature data is also expanding and the frequency of use is increasing.

[0003] In the prior art, when processing features in a feature group, a feature and its dependent features are usually output into a feature structure. If the dependent features of the features involved in this method are the same, repeated dependent features will be loaded multiple times during the feature processing, resulting in low feature processing efficiency. Summary of the invention

[0004] The embodiments of the present invention provide a feature processing method, device, storage medium and electronic device to at least solve the technical problem of low feature processing efficiency.

[0005] According to one aspect of an embodiment of the present invention, a feature processing method is provided, comprising: reading a feature group in a database and the feature group logic of the feature group to obtain the feature logic of each feature in the feature group; in the case where the feature group includes multiple features with the same feature logic, loading the feature logic of a first feature of the multiple features, and reading feature data of dependent features of the loaded first feature to obtain a loading result; using the loading result as the loading result of each feature in the multiple features; and outputting each feature in the multiple features and the loading result of each feature to a task structure.

[0006] According to another aspect of an embodiment of the present invention, there is provided a feature processing device, comprising: a reading module, used to read a feature group in a database and the feature group logic of the feature group, and obtain the feature logic of each feature in the feature group; a loading module, used to load the feature logic of a first feature of the multiple features when the feature group includes multiple features with the same feature logic, and read feature data of dependent features of the loaded first feature to obtain a loading result; a first determining module, used to use the loading result as the loading result of each feature in the multiple features; and an output module, used to output each feature in the multiple features and the loading result of each feature to a task structure.

[0007] As an optional example, the apparatus further includes: a second determination module, configured to determine that the feature logics of the multiple features are the same when the dependent features on which the multiple features depend are the same.

[0008] As an optional example, the above-mentioned device also includes: a connection module, which is used to take each feature in the above-mentioned feature group as the current feature, and connect the value of the above-mentioned current feature and the value of the dependent feature of the above-mentioned current feature in a string according to the reading order to generate a hash value; a first generation module, which is used to take the above-mentioned hash value as the key of a key-value pair, and the list where the above-mentioned current feature is located as the value of the key-value pair to generate a key-value pair; a third determination module, which is used to take the multiple features contained in the above-mentioned list as the above-mentioned multiple features with the same dependent features when the list corresponding to the value of the above-mentioned key-value pair contains multiple features.

[0009] As an optional example, the above-mentioned output module includes: an output unit, used to take each of the above-mentioned multiple features as the current feature, and output the above-mentioned current feature and the loading result of the above-mentioned current feature into a task structure; or output the above-mentioned multiple features and the above-mentioned loading result into the same task structure.

[0010] As an optional example, the above-mentioned output unit includes: an output sub-unit, used to output the feature logic of the first feature among the above-mentioned multiple features to the target task structure; and an appending unit, used to append the output columns and output tables of the remaining features among the above-mentioned multiple features to the above-mentioned target task structure.

[0011] As an optional example, the above-mentioned device also includes: a second generating module, which is used to generate the target task structure or each task structure into an executable statement.

[0012] As an optional example, the above-mentioned device also includes: a traversal module, used to traverse the hash values ​​of the dependent features in the above-mentioned task structure when the number of dependent features in the above-mentioned task structure is greater than or equal to 2; a configuration module, used to add an output table configuration in the task structure of the features corresponding to the repeated hash values, and merge the repeated feature columns and output them into a temporary table.

[0013] According to another aspect of an embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program executes the above-mentioned feature processing method when executed by a processor.

[0014] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the feature processing method through the computer program.

[0015] In an embodiment of the present invention, a method is adopted in which a feature group in a database and the feature group logic of the feature group are read to obtain the feature logic of each feature in the feature group; in the case where the feature group includes multiple features with the same feature logic, the feature logic of a first feature of the multiple features is loaded, and feature data of dependent features of the loaded first feature is read to obtain a loading result; the loading result is used as the loading result of each of the multiple features; and each of the multiple features and the loading result of each of the features are output to a task structure. In the method, in the process of processing features, for multiple features with the same feature logic, it is only necessary to load the feature logic of the feature once to obtain the loading result, and the loading result can be used as the loading result of each feature, thereby achieving the effect of improving feature processing efficiency, thereby solving the technical problem of low feature processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0017] Figure 1 is a flow chart of an optional feature processing method according to an embodiment of the present invention;

[0018] Figure 2 is a schematic diagram of feature merging according to an optional feature processing method according to an embodiment of the present invention;

[0019] Figure 3 is a feature processing schematic diagram of an optional feature processing method according to an embodiment of the present invention;

[0020] Figure 4 is a schematic structural diagram of an optional feature processing device according to an embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] According to a first aspect of an embodiment of the present invention, a feature processing method is provided. Optionally, as follows: Figure 1 As shown, the above method includes:

[0025] S102, reading feature groups and feature group logics of feature groups in a database, and obtaining feature logics of each feature in the feature group;

[0026] S104, when the feature group includes multiple features with the same feature logic, loading the feature logic of a first feature among the multiple features, and reading feature data of dependent features of the loaded first feature to obtain a loading result;

[0027] S106, using the loading result as the loading result of each of the multiple features;

[0028] S108, outputting each of the multiple features and the loading result of each feature into the task structure.

[0029] Optionally, the above feature processing method can be applied in the process of generating feature data used in the model training process. The database is a data source, and the source data, i.e., features, are stored in the database. The type of database is not limited. The features in the database can be output as a feature structure, and each feature structure can generate an executable statement. Through the executable statement, the features can be calculated into feature data, and the feature data can be used for model training.

[0030] In this embodiment, the configuration data of the feature logic of the feature can be obtained from each table of the database. The configuration involved includes:

[0031] a) Dependence on data sources (reference external data) and features (reference feature data generated by feature platforms);

[0032] b) Current feature output data column

[0033] c) Operator logic and operator parameter examples

[0034] d) Feature name, from table, where condition, having condition

[0035] e)join relationship

[0036] f) Group by relationship

[0037] Based on the above data relationships, the features will be assembled into executable SQL statements in the following format:

[0038] INSERT overwrite table [feature name] partition (dt = [date partition])

[0039] SELECT [output data column] FROM [dependency 1]

[0040] If there is a join relationship

[0041] JOIN other dependent tables JOIN [dependency 2] ON [dependent join columns...]

[0042] JOIN other dependent tables JOIN [dependency n] ON [dependent join columns...] 】

[0044] where[where condition]

[0045] group by[group condition]

[0046] having [having condition]

[0047] In the above process, if there are multiple features with the same feature logic, the feature logic of one of the multiple features can be recorded, and then the feature logic of the one feature can be used as the feature logic of each of the multiple features. With this method, there is no need to repeatedly load feature logic for features with the same feature logic, thereby improving the efficiency of feature processing.

[0048] As an optional example, the above method further includes:

[0049] In the case that the dependent features on which multiple features depend are the same, it is determined that the feature logics of the multiple features are the same.

[0050] Optionally, in this embodiment, for multiple features, each feature can be traversed to check the dependent features of each feature. If a feature has dependent features, after the dependent features are calculated, the calculation result is used as the input of the feature, and the feature is calculated to obtain the calculation result of the feature. If the dependent features of multiple features are the same, then the multiple features can be regarded as having the same feature logic.

[0051] As an optional example, the above method further includes:

[0052] Take each feature in the feature group as the current feature, concatenate the value of the current feature with the value of the dependent feature of the current feature in the order of reading in the form of a string, and generate a hash value;

[0053] Use the hash value as the key of the key-value pair and the list where the current feature is located as the value of the key-value pair to generate a key-value pair;

[0054] When the list corresponding to the value of the key-value pair contains multiple features, the multiple features contained in the list are regarded as multiple features with the same dependent features.

[0055] Optionally, in this embodiment, when checking whether the dependent features of a feature are the same, in order to improve the efficiency of checking and comparing, the comparison can be performed by hashing. When traversing the features, the values ​​of all features are sequentially connected in string form to generate a hash value. For example: hash = md5 (feature.where + "|" + feature.group + "|" + feature.having + ...).

[0056] The generated hash result is saved in a key-valuemap, where the key is hash and the value is a list containing the current feature.id. If multiple features have the same hash value, they are homologous. Under the key of the hash in the map, there is a list containing the ids of multiple features. Traverse all the hashkeys in the map. If the list in the value contains only one element, it indicates that there is no homologous feature for the feature, and the feature logic is directly output to the task structure (the task structure is a structure that can output executable SQL, including the necessary elements in SQL, such as output table name, output column, where condition, group, etc.); if the list in the value contains multiple elements, this is a homologous feature.

[0057] Optionally, in this embodiment, for multiple features with the same logic, one feature can be selected from the multiple features, and the selection method is not limited. For example, the first feature can be selected. The feature data of the dependent feature of the selected feature is read, and the feature data of the dependent feature is used as the loading result. Each of the multiple features uses the loading result.

[0058] As an optional example, the above-mentioned outputting each feature of the multiple features and the loading result of each feature to the task structure includes:

[0059] Take each of the multiple features as the current feature, and output the current feature and the loading result of the current feature to a task structure; or

[0060] Output multiple features and loading results to the same task structure.

[0061] Optionally, after reading any one of the features to obtain the loading result, the multiple features in this embodiment can output a feature and a loading result to a task structure. The task structure can generate executable statements. Using this method, when generating a task structure, multiple features only need to load the loading result once, and a task structure can be generated for each feature.

[0062] Optionally, in this embodiment, for the above-mentioned multiple features, since the dependent features are the same, the multiple features and the loading results can be output to one task structure.

[0063] As an optional example, the above output of multiple features and loading results to the same task structure includes:

[0064] Output the feature logic of the first feature among multiple features to the target task structure;

[0065] Append the output columns and output tables of the remaining features in the multiple features to the target task structure.

[0066] Optionally, in this embodiment, when multiple features and loading results are output to a task structure, the feature logic of the first feature among the multiple features can be output to a target task structure. Then, the output columns and output tables of the remaining features among the multiple features are appended to the target task structure, thereby outputting the multiple features to a task structure.

[0067] As an optional example, the above method further includes:

[0068] Generate the target task structure or each task structure into executable statements.

[0069] Each task structure in this embodiment can generate an executable statement. After the executable statement is executed, feature data is obtained, and the feature data can be used for model training.

[0070] As an optional example, the above method further includes:

[0071] When the number of dependent features in the task structure is greater than or equal to 2, traverse the hash values ​​of the dependent features in the task structure;

[0072] Add output table configuration to the task structure of the features corresponding to the repeated hash values, and merge the repeated feature columns and output them to a temporary table.

[0073] Optional, explain with examples. For reading from the same data source. For example, feature 1 and feature 2 have the same dependent features, and the dependent data is table s. Then, before each feature calculation, since each feature SQL is an independent task, hadoop needs to load the contents of table s in hive, which is a repeated loading process.

[0074] Similar output operations. The output table t of each feature contains columns for the feature's own data columns and other columns that must be passed upstream, such as primary keys, groupby columns, and partition columns. Then there may be multiple features, and except for their own data columns, other columns are the same data. Then when multiple similar features are output, there is also a repeated data output process.

[0075] For example:

[0076] The logic of feature 1 is as follows:

[0077]

[0078]

[0079] The logic of feature 2 is as follows:

[0080] In order to solve the problem of repeated loading and repeated output, in this embodiment, the feature homology is defined: multiple features have the same dependent source table, the same where condition, the same group, and the same having condition. If multiple tables are joined, the join method is exactly the same.

[0081] When multiple features come from the same source, multiple features can be output at once by calculating the dependent source, avoiding reading the dependent source multiple times. Figure 2 As shown, feature 1 and feature 2 with the same dependent features are merged and output.

[0082] For example: Features 1 and 2 in the above example can be combined into the following feature combination calculation output

[0083]

[0084] During output, one or more feature structures will be merged into a task structure, and the task structure will eventually be parsed and constructed into an executable SQL statement.

[0085] Through the relationship between feature groups, obtain all the features that the feature group directly depends on (that is, the features that need to be output)

[0086] First, traverse all features, traverse the features and all dependent features of the features by depth or breadth, obtain all related features and form a dependency tree structure

[0087] Get all the logic of the feature when traversing the feature (from, where, group, having, etc.)

[0088] In order to quickly determine whether multiple logical points of a feature are the same (i.e., homology conditions), when traversing the features, the values ​​of all features are sequentially connected in string form to generate a hash value. For example: hash = md5 (feature.where + "|" + feature.group + "|" + feature.having + ...).

[0089] The generated hash result is saved in a key-valuemap, where the key is the hash and the value is a list containing the current feature.id. If there are multiple features with the same hash value, their table names are the same, and under the hash key of the map, there is a list containing the ids of multiple features.

[0090] Traverse all hash keys in the map. If the list in the value contains only one element, it indicates that there is no homologous feature for the feature, and the feature logic is directly output to the task structure (the task structure is a structure that can output executable SQL, including the necessary elements in SQL, such as output table name, output column, where condition, group, etc.); if the list in the value contains multiple elements, the table name is a homologous feature.

[0091] Take the first feature in the homologous feature list as the prototype, obtain all logic (from, where, group, having and other information), traverse other features in the homologous feature list, and append the output columns and output tables of all features to the corresponding task structure.

[0092] After generating a task structure list for all homologous feature lists (the number of task structures = the number of homologous features ≤ the number of features), an executable statement SQL is uniformly generated according to the task structure.

[0093] As another example, if there are multiple dependent features;

[0094] The dependent features meet the same source conditions, that is, two or more dependent features have the same from, where, group, having, and join conditions.

[0095] For example:

[0096] Feature 3 depends on upstream features 1 and 2, where features 1 and 2 are homologous features.

[0097]

[0098] The optimized logic is as follows:

[0099]

[0100]

[0101] It can be seen that feature 3 requires a JOIN operation on feature 1 and feature 2, which takes up a lot of computing time. In this embodiment, feature 1 and feature 2 can directly output the merged temporary data table during the upstream merge optimization process, so the JOIN operation can be omitted. Figure 3 As shown, feature 1 and feature 2 are optimally merged.

[0102] That is, the optimization logic of features 1 and 2 is optimized again as follows:

[0103]

[0104]

[0105] Feature 3 can be optimized to:

[0106]

[0107] That is, the task structures of one or more feature outputs will be merged into a new task structure, and finally the new task structure will be parsed and constructed into an executable SQL statement.

[0108] Traverse the generated task structure list to determine whether the number of dependent features in the structure is greater than or equal to 2. If the number of dependent features in the structure is less than 2, ignore the following steps and directly output the current feature structure; if the number of dependent features in the structure is greater than or equal to 2, traverse the hash values ​​of the dependent features in the structure. If there are repeated hash values, it indicates that there are homologous features that meet strategy 1; if there are no repeated hash values, ignore the following steps and directly output the current feature structure;

[0109] By repeating the hash value, find the task structure of the corresponding feature and make corresponding modifications: add output table configuration and output all repeated feature columns to the merged temporary table;

[0110] Modify the current structure: delete all join tables of the same source and merge them into a temporary merged output table; if all join tables are of the same source, cancel the join table, modify the from table to be a temporary merged output table, and output the modified task structure.

[0111] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0112] According to another aspect of the embodiment of the present application, a feature processing device is also provided. Figure 4 As shown, including:

[0113] A reading module 402 is used to read the feature group and the feature group logic of the feature group in the database to obtain the feature logic of each feature in the feature group;

[0114] A loading module 404 is used to load the feature logic of a first feature among the multiple features when the feature group includes multiple features with the same feature logic, and read feature data of dependent features of the loaded first feature to obtain a loading result;

[0115] A first determining module 406, configured to use the loading result as a loading result of each of the plurality of features;

[0116] The output module 408 is used to output each of the multiple features and the loading result of each feature to the task structure.

[0117] Optionally, the feature processing device can be applied in the process of generating feature data used in the model training process. The database is a data source, and the source data, i.e., features, are stored in the database. The type of database is not limited. The features in the database can be output as a feature structure, and each feature structure can generate an executable statement. Through the executable statement, the features can be calculated into feature data, and the feature data can be used for model training.

[0118] In this embodiment, the configuration data of the feature logic of the feature can be obtained from each table of the database. The configuration involved includes:

[0119] a) Dependence on data sources (reference external data) and features (reference feature data generated by feature platforms);

[0120] b) Current feature output data column

[0121] c) Operator logic and operator parameter examples

[0122] d) Feature name, from table, where condition, having condition

[0123] e)join relationship

[0124] f) Group by relationship

[0125] Based on the above data relationships, the features will be assembled into executable SQL statements in the following format:

[0126] INSERT overwrite table [feature name] partition (dt = [date partition])

[0127] SELECT [output data column] FROM [dependency 1]

[0128] If there is a join relationship

[0129] JOIN other dependent tables JOIN [dependency 2] ON [dependent join columns...]

[0130] JOIN other dependent tables JOIN [dependency n] ON [dependent join columns...] 】

[0132] where[where condition]

[0133] group by[group condition]

[0134] having [having condition]

[0135] In the above process, if there are multiple features with the same feature logic, the feature logic of one of the multiple features can be recorded, and then the feature logic of the one feature can be used as the feature logic of each of the multiple features. With this method, there is no need to repeatedly load feature logic for features with the same feature logic, thereby improving the efficiency of feature processing.

[0136] As an optional example, the above device further includes:

[0137] The second determination module is used to determine that the feature logics of the above-mentioned multiple features are the same when the dependent features on which the above-mentioned multiple features depend are the same.

[0138] Optionally, in this embodiment, for multiple features, each feature can be traversed to check the dependent features of each feature. If a feature has dependent features, after the dependent features are calculated, the calculation result is used as the input of the feature, and the feature is calculated to obtain the calculation result of the feature. If the dependent features of multiple features are the same, then the multiple features can be regarded as having the same feature logic.

[0139] As an optional example, the above device further includes:

[0140] A connection module, used to take each feature in the feature group as a current feature, connect the value of the current feature with the value of the dependent feature of the current feature in a string in a reading order, and generate a hash value;

[0141] A first generating module, configured to use the hash value as the key of a key-value pair and the list where the current feature is located as the value of the key-value pair to generate a key-value pair;

[0142] The third determining module is used for, when the list corresponding to the value of the key-value pair contains multiple features, taking the multiple features contained in the list as the multiple features with the same dependent features.

[0143] Optionally, in this embodiment, when checking whether the dependent features of a feature are the same, in order to improve the efficiency of checking and comparing, the comparison can be performed by hashing. When traversing the features, the values ​​of all features are sequentially connected in string form to generate a hash value. For example: hash = md5 (feature.where + "|" + feature.group + "|" + feature.having + ...).

[0144] The generated hash result is saved in a key-valuemap, where the key is hash and the value is a list containing the current feature.id. If multiple features have the same hash value, they are homologous. Under the key of the hash in the map, there is a list containing the ids of multiple features. Traverse all the hashkeys in the map. If the list in the value contains only one element, it indicates that there is no homologous feature for the feature, and the feature logic is directly output to the task structure (the task structure is a structure that can output executable SQL, including the necessary elements in SQL, such as output table name, output column, where condition, group, etc.); if the list in the value contains multiple elements, this is a homologous feature.

[0145] Optionally, in this embodiment, for multiple features with the same logic, one feature can be selected from the multiple features, and the selection method is not limited. For example, the first feature can be selected. The feature data of the dependent feature of the selected feature is read, and the feature data of the dependent feature is used as the loading result. Each of the multiple features uses the loading result.

[0146] As an optional example, the output module includes:

[0147] The output unit is used to take each of the multiple features as the current feature, and output the current feature and the loading result of the current feature into a task structure; or output multiple features and the loading results into the same task structure.

[0148] Optionally, after reading any one of the features to obtain the loading result, the multiple features in this embodiment can output a feature and a loading result to a task structure. The task structure can generate executable statements. Using this method, when generating a task structure, multiple features only need to load the loading result once, and a task structure can be generated for each feature.

[0149] Optionally, in this embodiment, for the above-mentioned multiple features, since the dependent features are the same, the multiple features and the loading results can be output to one task structure.

[0150] As an optional example, the output unit includes:

[0151] An output subunit, used for outputting the feature logic of the first feature among the above-mentioned multiple features to the target task structure;

[0152] The appending unit is used to append the output columns and output tables of the remaining features in the above-mentioned multiple features to the above-mentioned target task structure.

[0153] Optionally, in this embodiment, when multiple features and loading results are output to a task structure, the feature logic of the first feature among the multiple features can be output to a target task structure. Then, the output columns and output tables of the remaining features among the multiple features are appended to the target task structure, thereby outputting the multiple features to a task structure.

[0154] As an optional example, the above device further includes:

[0155] The second generation module is used to generate the target task structure or each task structure into an executable statement.

[0156] As an optional example, the above device further includes:

[0157] A traversal module, used for traversing the hash values ​​of the dependent features in the task structure when the number of dependent features in the task structure is greater than or equal to 2;

[0158] The configuration module is used to add output table configuration to the task structure of the features corresponding to the repeated hash values, and merge the repeated feature columns and output them into a temporary table.

[0159] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0160] Figure 5 is a structural block diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5As shown, it includes a processor 502, a communication interface 504, a memory 506 and a communication bus 508, wherein the processor 502, the communication interface 504 and the memory 506 communicate with each other through the communication bus 508, wherein,

[0161] Memory 506, used for storing computer programs;

[0162] The processor 502 is used to execute the computer program stored in the memory 506 to implement the following steps:

[0163] Read the feature groups and feature group logic of the feature groups in the database to obtain the feature logic of each feature in the feature group;

[0164] In the case where the feature group includes multiple features with the same feature logic, the feature logic of a first feature among the multiple features is loaded, and feature data of dependent features of the loaded first feature is read to obtain a loading result;

[0165] Using the loading result as the loading result of each of the multiple features;

[0166] Output each of the multiple features and the loading results of each feature to the task structure.

[0167] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The communication interface is used for communication between the above electronic device and other devices.

[0168] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0169] As an example, the memory 506 may include but is not limited to the reading module 402, loading module 404, first determination module 406 and output module 408 in the feature processing device. In addition, it may also include but is not limited to other module units in the request processing device, which will not be repeated in this example.

[0170] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0171] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0172] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only. The device for implementing the above-mentioned feature processing method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.

[0173] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0174] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned feature processing method when executed by a processor.

[0175] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0176] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0177] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0178] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0179] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0182] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A feature processing method, characterized in that: include: Reading a feature group and a feature group logic of the feature group in a database, and obtaining a feature logic of each feature in the feature group; In the case where the feature group includes multiple features with the same feature logic, the feature logic of a first feature among the multiple features is loaded, and feature data of a dependent feature of the loaded first feature is read to obtain a loading result, wherein, in the case where the dependent features on which the multiple features depend are the same, it is determined that the feature logics of the multiple features are the same; Using the loading result as the loading result of each of the multiple features; Outputting each of the plurality of features and the loading result of each of the features into a task structure; The method also includes: taking each feature in the feature group as a current feature, connecting the value of the current feature and the value of the dependent feature of the current feature in a string in a reading order to generate a hash value; taking the hash value as the key of a key-value pair, and taking the list where the current feature is located as the value of the key-value pair to generate a key-value pair; when the list corresponding to the value of the key-value pair contains multiple features, taking the multiple features contained in the list as the multiple features with the same dependent features.

2. The method according to claim 1, characterized in that The step of outputting each of the plurality of features and the loading result of each of the features to the task structure includes: Taking each of the multiple features as a current feature, and outputting the current feature and the loading result of the current feature into a task structure; or The multiple features and the loading result are output to the same task structure.

3. The method according to claim 2, characterized in that The step of outputting the plurality of features and the loading result to the same task structure includes: Outputting the feature logic of the first feature among the multiple features to the target task structure; The output columns and output tables of the remaining features among the plurality of features are added to the target task structure.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: Generate the target task structure or each task structure into executable statements.

5. The method according to claim 3, characterized in that: The method further comprises: When the number of dependent features in the task structure is greater than or equal to 2, traversing the hash values ​​of the dependent features in the task structure; Add output table configuration to the task structure of the features corresponding to the repeated hash values, and merge the repeated feature columns and output them to a temporary table.

6. A feature processing device, characterized in that: include: A reading module, used for reading a feature group and a feature group logic of the feature group in a database, and obtaining a feature logic of each feature in the feature group; a loading module, configured to load the feature logic of a first feature among the multiple features when the feature group includes multiple features with the same feature logic, and read feature data of a dependent feature of the loaded first feature to obtain a loading result, wherein when the dependent features on which the multiple features depend are the same, it is determined that the feature logics of the multiple features are the same; A first determining module, configured to use the loading result as a loading result of each of the multiple features; An output module, used for outputting each of the multiple features and the loading result of each feature into a task structure; The device is also used to: take each feature in the feature group as the current feature, connect the value of the current feature and the value of the dependent feature of the current feature in a string according to the reading order, and generate a hash value; take the hash value as the key of a key-value pair, and take the list where the current feature is located as the value of the key-value pair to generate a key-value pair; when the list corresponding to the value of the key-value pair contains multiple features, take the multiple features contained in the list as the multiple features with the same dependent features.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is executed.

8. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.

Images