Feature generation method and device, electronic equipment and storage medium

Abstract

The invention provides a feature generation method and device, electronic equipment and a storage medium, and relates to the technical field of feature, and the method comprises the steps: carrying out the grouping of a plurality of to-be-calculated features based on the data range information of the to-be-calculated features, and obtaining a to-be-calculated feature set; obtaining a streaming computation operator based on the measurement information corresponding to the to-be-computed feature, and obtaining a to-be-computed data entry based on the data range information; traversing the to-be-calculated feature set through the to-be-calculated data item, and inputting the to-be-calculated data item into a streaming computation operator corresponding to the to-be-calculated feature contained in the to-be-calculated feature set under the condition that the to-be-calculated data item is matched with the data range information corresponding to the to-be-calculated feature set to obtain a feature value. According to the method, the corresponding streaming computation operator can be generated for a large number of to-be-computed features, batch feature computation can be completed only by performing one-time reading and one-time traversing on the to-be-computed data items, repeated data access is avoided, and the batch feature generation speed is increased.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of feature technology, and in particular, to a feature generation method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of computer technology, more and more businesses need to generate corresponding features for events, users, products and other entities, and use these features to achieve downstream power supply such as data analysis and experience optimization.

[0003] In the related technology, it is usually necessary to set a large number of features for a large number of subjects. For each feature that needs to be calculated, the required calculation data and algorithm will be read and loaded first, and then the feature value of the feature will be calculated using these data. After completing the calculation of one feature, the calculation data required for the next feature will be read and loaded for calculation until the calculation of all features is completed.

[0004] However, there may be some common data that need to be used between different features. The above feature generation method requires repeated loading of these data, resulting in low efficiency in the batch feature generation process. Summary of the invention

[0005] The embodiments of the present application provide a feature generation method, device, electronic device and storage medium to solve the problem of low efficiency of batch feature generation in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a feature generation method, the method comprising:

[0007] Obtaining measurement information and data range information corresponding to a plurality of features to be calculated, respectively; wherein the data range information is used to indicate a selection range of data required for the features to be calculated;

[0008] The plurality of features to be calculated are grouped based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated included in each set of features to be calculated have the same data range information;

[0009] Based on the metric information corresponding to the feature to be calculated, a stream computing operator corresponding to the feature to be calculated is obtained, and based on the data range information corresponding to the multiple features to be calculated, a data entry to be calculated is obtained; wherein the stream computing operator is used to calculate the feature value of the feature to be calculated based on the target data in the data entry to be calculated that matches the data type of the feature to be calculated;

[0010] The feature set to be calculated is traversed through the data entry to be calculated. When the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, the data entry to be calculated is input into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, and the feature value of the corresponding feature to be calculated output by the streaming computing operator is obtained.

[0011] Optionally, after grouping the multiple features to be calculated based on the data range information to obtain a set of features to be calculated, the method further includes:

[0012] The features to be calculated in the feature set to be calculated are grouped based on the algorithm type contained in the metric information to obtain a feature subset to be calculated; wherein the features to be calculated contained in each feature subset to be calculated have the same algorithm type;

[0013] The step of inputting the data item to be calculated into the stream computing operator corresponding to the feature to be calculated included in the feature set to be calculated, and obtaining the feature value of the feature to be calculated output by the stream computing operator, includes:

[0014] Traversing each feature subset to be calculated in the feature set to be calculated through the data entry to be calculated, and determining a target data set corresponding to the feature subset to be calculated from the data entry to be calculated based on the algorithm type corresponding to the feature subset to be calculated;

[0015] The target data set is input into the stream computing operator corresponding to the feature to be calculated contained in the feature subset to be calculated, and the feature value of the corresponding feature to be calculated output by the stream computing operator is obtained.

[0016] Optionally, the acquiring, based on the metric information corresponding to the feature to be calculated, a stream computing operator corresponding to the feature to be calculated includes:

[0017] Determining a target algorithm and an initialization value of a first variable based on the algorithm type of the feature to be calculated; wherein the target algorithm is used to update the first variable according to the data item to be calculated and the initialization value of the first variable;

[0018] Based on the target algorithm and the first variable, a streaming computing operator corresponding to the feature to be calculated is constructed.

[0019] Optionally, the stream computing operator includes a second variable, and constructing the stream computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable includes:

[0020] Initialize the second variable to 0; wherein the second variable is used to count the number of operations input into the stream computing operator;

[0021] Based on the target algorithm, the first variable and the second variable, a streaming computing operator corresponding to the feature to be calculated is constructed; wherein the streaming computing operator is also used to output first information when the traversal of the data entries to be calculated is completed and the second variable is 0, and to determine that the feature value is the variable value of the first variable when the traversal of the data entries to be calculated is completed and the second variable is non-0, and to output the feature confidence corresponding to the feature value based on the variable value of the second variable.

[0022] Optionally, the data range information includes time period range information, and the streaming computing operator is also used to match the variable value of the second variable with the unit time quantity indicated by the time period range information. When the variable value of the second variable matches the unit time quantity, the variable value of the first variable is output and the streaming computing operator is released.

[0023] Optionally, the stream computing operator further includes a third variable, and constructing the stream computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable includes:

[0024] Based on the measurement information corresponding to the feature to be calculated, determine the target data name required for the feature to be calculated, and assign the target data name to the third variable;

[0025] Based on the target algorithm, the first variable and the third variable, a stream computing operator corresponding to the feature to be calculated is constructed, so that the stream computing operator extracts the target data from the data entry to be calculated based on the third variable.

[0026] Optionally, the stream computing operator includes a common stream computing operator, the data range information includes time period range information and non-time period range information; and obtaining the stream computing operator corresponding to the feature to be calculated based on the metric information corresponding to the feature to be calculated includes:

[0027] Determine, from the plurality of features to be calculated, a group of features to be calculated that have the same measurement information, the same non-period range information, and one end of the period range information is aligned;

[0028] Based on the metric information corresponding to the features to be calculated in the feature group to be calculated, construct a common stream computing operator shared by all the features to be calculated in the feature group to be calculated;

[0029] The step of inputting the data item to be calculated into the stream computing operator corresponding to the feature to be calculated included in the feature set to be calculated, and obtaining the feature value of the feature to be calculated output by the stream computing operator, includes:

[0030] In the case where the data item to be calculated matches the feature group to be calculated corresponding to the feature to be calculated included in the feature set to be calculated, the data item to be calculated is sequentially input into the common stream computing operator corresponding to the feature group to be calculated where the feature to be calculated is located according to the data collection time of the data item to be calculated;

[0031] After all the data items to be calculated that match the time period range information of the feature to be calculated are input into the common stream computing operator corresponding to the feature to be calculated, the variable value of the first variable in the common stream computing operator is obtained to obtain the feature value of the feature to be calculated.

[0032] In a second aspect, an embodiment of the present application provides a feature generation device, the device comprising:

[0033] An acquisition module, used to acquire measurement information and data range information corresponding to a plurality of features to be calculated, wherein the data range information is used to indicate a selection range of data required for the features to be calculated;

[0034] A collection module, used for grouping the plurality of features to be calculated based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated contained in each set of features to be calculated have the same data range information;

[0035] An operator module, used to obtain a stream computing operator corresponding to the feature to be calculated based on the metric information corresponding to the feature to be calculated, and to obtain a data entry to be calculated based on the data range information corresponding to the multiple features to be calculated; wherein the stream computing operator is used to calculate a feature value of the feature to be calculated based on target data in the data entry to be calculated that matches the data type of the feature to be calculated;

[0036] A traversal module is used to traverse the feature set to be calculated through the data entry to be calculated. When the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, the data entry to be calculated is input into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, so as to obtain the feature value of the corresponding feature to be calculated output by the streaming computing operator.

[0037] Optionally, the device further comprises:

[0038] A subset module, used for grouping the features to be calculated in the feature set to be calculated based on the algorithm type contained in the metric information to obtain a subset of features to be calculated; wherein the features to be calculated contained in each subset of features to be calculated have the same algorithm type;

[0039] The traversal module includes:

[0040] A target data set submodule, configured to traverse each feature subset to be calculated in the feature set to be calculated through the data entry to be calculated, and determine a target data set corresponding to the feature subset to be calculated from the data entry to be calculated based on the algorithm type corresponding to the feature subset to be calculated;

[0041] The feature value submodule is used to input the target data set into the stream computing operator corresponding to the feature to be calculated contained in the feature subset to be calculated, and obtain the feature value of the corresponding feature to be calculated output by the stream computing operator.

[0042] Optionally, the operator module includes:

[0043] An initialization submodule, used to determine a target algorithm and an initialization value of a first variable based on the algorithm type of the feature to be calculated; wherein the target algorithm is used to update the first variable according to the data item to be calculated and the initialization value of the first variable;

[0044] An operator construction submodule is used to construct a streaming computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable.

[0045] Optionally, the stream computing operator includes a second variable, and the operator construction submodule includes:

[0046] A first initialization unit, configured to initialize the second variable to 0; wherein the second variable is used to count the number of operations input into the stream computing operator;

[0047] A first construction unit is used to construct a streaming computing operator corresponding to the feature to be calculated based on the target algorithm, the first variable and the second variable; wherein the streaming computing operator is also used to output first information when the traversal of the data entries to be calculated is completed and the second variable is 0, and to determine that the feature value is the variable value of the first variable when the traversal of the data entries to be calculated is completed and the second variable is non-0, and to output the feature confidence corresponding to the feature value based on the variable value of the second variable.

[0048] Optionally, the data range information includes time period range information, and the streaming computing operator is also used to match the variable value of the second variable with the unit time quantity indicated by the time period range information. When the variable value of the second variable matches the unit time quantity, the variable value of the first variable is output and the streaming computing operator is released.

[0049] Optionally, the stream computing operator further includes a third variable, and the operator construction submodule includes:

[0050] A second initialization unit, configured to determine a target data name required for the feature to be calculated based on the measurement information corresponding to the feature to be calculated, and assign the target data name to the third variable;

[0051] The second construction unit is used to construct a streaming computing operator corresponding to the feature to be calculated based on the target algorithm, the first variable and the third variable, so that the streaming computing operator extracts the target data from the data entry to be calculated based on the third variable.

[0052] Optionally, the stream computing operator includes a common stream computing operator, the data range information includes time period range information and non-time period range information; the operator module includes:

[0053] The feature group to be calculated submodule is used to determine, from the multiple features to be calculated, a feature group to be calculated that has the same metric information, the same non-time period range information, and one end of the time period range information is aligned;

[0054] A construction submodule, configured to construct a common stream computing operator shared by all the features to be calculated in the feature group to be calculated based on the metric information corresponding to the features to be calculated in the feature group to be calculated;

[0055] The traversal module includes:

[0056] A public stream computing operator input submodule, for, when the data item to be calculated matches the feature group to be calculated corresponding to the feature to be calculated contained in the feature set to be calculated, inputting the data item to be calculated into the public stream computing operator corresponding to the feature group to be calculated where the feature to be calculated is located in sequence according to the data collection time of the data item to be calculated;

[0057] The common streaming computing operator output submodule is used to obtain the variable value of the first variable in the common streaming computing operator after all the data entries to be calculated that match the time period range information of the feature to be calculated are input into the common streaming computing operator corresponding to the feature to be calculated, so as to obtain the feature value of the feature to be calculated.

[0058] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a feature generation method as in the first aspect.

[0059] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the feature generation method as in the first aspect.

[0060] In an embodiment of the present application, the method includes: obtaining metric information and data range information corresponding to a plurality of features to be calculated, wherein the data range information is used to indicate the selection range of data required for the features to be calculated; grouping a plurality of features to be calculated based on the data range information to obtain a set of features to be calculated; obtaining a stream computing operator corresponding to the features to be calculated based on the metric information corresponding to the features to be calculated, and obtaining a data entry to be calculated based on the data range information corresponding to the plurality of features to be calculated; traversing the set of features to be calculated through the data entry to be calculated, and when the data entry to be calculated matches the data range information corresponding to the set of features to be calculated, inputting the data entry to be calculated into the stream computing operator corresponding to the features to be calculated contained in the set of features to be calculated, and obtaining the feature value of the corresponding feature to be calculated output by the stream computing operator. The method can generate corresponding stream computing operators for a large number of features to be calculated, and batch feature calculation can be completed by only reading and traversing the data entry to be calculated once, which can avoid repeated access to data, and the stream computing operator is used in the feature calculation process, and the content of the data entry to be calculated does not need to be stored in the calculation process, which further reduces the storage resource overhead in the batch feature calculation process and effectively improves the efficiency of batch feature calculation. It is also possible to traverse the set of features to be calculated that is generated by grouping the features to be calculated, thereby effectively improving the efficiency of the streaming computing operator that needs to input the data items to be calculated, and speeding up the generation of batch features.

[0061] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0063] Figure 1 is a flow chart of the steps of a feature generation method provided in an embodiment of the present application;

[0064] Figure 2 is a flowchart of another feature generation method provided in an embodiment of the present application;

[0065] Figure 3 is a structural diagram of a feature generation device provided in an embodiment of the present application;

[0066] Figure 4It is a logic block diagram of an electronic device according to an embodiment of the present application;

[0067] Figure 5 It is a logic block diagram of an electronic device of another embodiment provided by an embodiment of the present application. DETAILED DESCRIPTION

[0068] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0069] Figure 1 is a flow chart of the steps of a feature generation method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0070] Step 101, obtaining measurement information and data range information corresponding to a plurality of features to be calculated respectively; wherein the data range information is used to indicate a selection range of data required for the features to be calculated.

[0071] In an embodiment of the present application, a plurality of features to be calculated specified by a user may be received, and batch calculations may be performed on the plurality of features to be calculated to obtain feature values ​​corresponding to each feature to be calculated. The feature to be calculated may also be referred to as an indicator to be calculated or a feature indicator to be calculated, and correspondingly, the feature value may also be referred to as an indicator value, which is not specifically limited in the embodiment of the present application. The above-mentioned features to be calculated may be feature indicators of any type, such as risk feature indicators, transaction volume feature indicators, login times feature indicators, browsing times feature indicators, trading volume indicators, transaction amount indicators, etc., which are not specifically limited in the embodiment of the present application.

[0072] The metric information may indicate the information related to the algorithm and data type used when calculating the feature. The above algorithm may include but is not limited to finding the average, variance, maximum, minimum, etc. The above data type may include but is not limited to risk score, sales volume, sales amount, transaction volume, etc. The data range information may indicate the source of the required calculation data when calculating the feature. The data range information is used to indicate the selection range of the data required for the feature to be calculated, and may include business range information, event range information, time period range information, etc., which is not specifically limited in the embodiments of the present application.

[0073] In the embodiment of the present application, the corresponding database can be queried by the feature name of the feature to be calculated, and the measurement information and data range information of the feature to be calculated can be read according to the correspondence between the feature name and the feature information stored in the database. In addition, the measurement information and data range information of the feature to be calculated can be directly carried in the feature name of the feature to be calculated, and the measurement information and data range information of the feature to be calculated can be obtained by parsing the feature name of the feature to be calculated.

[0074] For example, Table 1 shows an exemplary list of features to be calculated provided in an embodiment of the present application:

[0075]

[0076] Table 1

[0077] Taking m_max_riskscore_Login_Bank_60day as an example, after parsing the feature name of the feature to be calculated, its measurement information can be obtained as max_riskscore, that is, the maximum value in the statistical risk value, and its data range information includes Login event, Bank business, and 60-day data period, that is, the selection range of the data required for the feature to be calculated is: login events generated in banking business in the past 60 days, so the feature value corresponding to the feature to be calculated is: the maximum value in the risk value corresponding to the login events generated in banking business in the past 60 days.

[0078] It should be noted that, according to different needs, technicians can flexibly set the measurement information and data range information of the features to be calculated. For example, the measurement information can also be average, sum, etc., and the data range information can also include sales data, purchase events, e-commerce business, 24-month data period, etc., which is not specifically limited in the embodiments of the present application.

[0079] Step 102: grouping the plurality of features to be calculated based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated included in each set of features to be calculated have the same data range information.

[0080] In an embodiment of the present application, multiple features to be calculated can be grouped based on data range information, and features to be calculated having the same data range information can be grouped to obtain a set of features to be calculated, so that the features to be calculated contained in each set of features to be calculated have the same data range information. It should be noted that the above grouping process can also be expressed as a bucketing operation on multiple features to be calculated based on data range information to obtain data range dimension buckets (feature sets to be calculated) corresponding to each data range information.

[0081] Exemplarily, after grouping the multiple features to be calculated shown in Table 1 above, 6 feature sets to be calculated can be obtained, and the data range information corresponding to each feature set to be calculated is shown in Table 2 below:

[0082] The set of features to be calculated Data range information Feature set to be calculated 1 Login_Bank_60day Feature set to be calculated 2 Login_O2O_30day Feature set to be calculated 3 Register_Bank_30day Feature set to be calculated 4 Register_Bank_60day Feature set to be calculated 5 Register_O2O_30day Feature set to be calculated 6 Register_O2O_60day

[0083] Table 2

[0084] Step 103: obtaining a streaming computing operator corresponding to the feature to be calculated based on the measurement information corresponding to the feature to be calculated, and obtaining a data entry to be calculated based on the data range information corresponding to the multiple features to be calculated; wherein the streaming computing operator is used to calculate the characteristic value of the feature to be calculated based on the target data in the data entry to be calculated that matches the data type of the feature to be calculated.

[0085] In an embodiment of the present application, a stream computing operator corresponding to the feature to be calculated can be obtained based on the metric information corresponding to the feature to be calculated, and a data entry to be calculated can be obtained based on the data range information corresponding to multiple features to be calculated. Among them, the stream computing operator can represent an algorithm model that can use the data entry to be calculated corresponding to the feature to be calculated and the algorithm indicated by the metric information to calculate the feature value corresponding to the feature to be calculated, and its existence in the computer can be a process or an instance. The stream computing operator can be a computing model that processes continuous and real-time data streams. The data enters the operator in the form of a stream. The stream computing operator will calculate each input data once and update the calculation result. When the next data is input, the above calculation result and the new data are used to calculate again and update the calculation result again until all the data to be calculated are input. The calculation result updated for the last time is the final output calculation result. There is no need to accumulate data and then process it at one time during the calculation process, and there is no need to save each calculation result, which can effectively reduce the memory overhead required for the calculation process.

[0086] Specifically, the streaming computing algorithm corresponding to each metric information can be pre-set, and for each feature to be calculated, the streaming computing algorithm corresponding to its metric information is loaded. Taking the metric information as the summation algorithm for sales as an example, the corresponding streaming computing algorithm can receive the first data entry to be calculated, read the first sales from the first data entry to be calculated, assign the first sales to variable A, then receive the second data entry to be calculated, read the second sales from the second data entry to be calculated, add the second sales to variable A, update variable A with the addition result, then receive the third sales, update variable A again, and so on, until all data input is completed, and variable A is the final calculation result.

[0087] It should be noted that different streaming computing algorithms can be used to implement different feature calculation algorithms, such as finding maximum values, average values, etc. Technicians can preset multiple streaming computing algorithms according to actual needs, and the embodiments of this application do not make specific limitations.

[0088] Since the data range information can indicate the data location and time period required for feature calculation, the data entry to be calculated can be obtained based on the data range information of the feature to be calculated. For example, if the data range information corresponding to a feature to be calculated is "sales events generated in the e-commerce business in the past 30 days", the daily data generated by the sales events in the e-commerce business in the past 30 days can be obtained as the data entry to be calculated. Each data entry to be calculated can have multiple data, such as sales volume, sales volume, etc. According to the different data ranges corresponding to the data entry to be calculated, the data type contained in the data entry to be calculated may also be different, and the embodiments of the present application do not make specific limitations.

[0089] It should be noted that the above-mentioned multiple features to be calculated may also be for a certain target object.

[0090] For example, target goods, target businesses, target users, etc. In this case, the data items to be calculated can be filtered from the data of the target object based on the data range information.

[0091] Step 104, traverse the feature set to be calculated through the data entry to be calculated, and when the data range information corresponding to the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, input the data entry to be calculated into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, and obtain the feature value of the corresponding feature to be calculated output by the streaming computing operator.

[0092] In an embodiment of the present application, after determining the data entries to be calculated required for multiple features to be calculated, the data entries to be calculated can be matched with the data range information corresponding to each feature set to be calculated in turn. If the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, that is, the data entry to be calculated meets the requirements of the data range information, then the data entry to be calculated is input into the streaming computing operator corresponding to each feature to be calculated in the feature set to be calculated. The streaming computing operator can obtain the target data of the data type required for the corresponding feature to be calculated from the input data entry to be calculated, and perform a calculation once until all data entries to be calculated are traversed, all features to be calculated are also calculated synchronously, and the streaming computing operators corresponding to all features to be calculated output the feature values ​​corresponding to the features to be calculated.

[0093] For example, if {eventtype (event)}_{industry (business)} = Login_Bank of a data entry record1 to be calculated matches the event range information and business range information of the data range information corresponding to the feature set 1 to be calculated in Table 2, then further calculate whether the date value of record1 matches the time range information (30 days) of the data range information corresponding to the feature set 1 to be calculated. If the current date is 2027-07-30, and the date value of record1 is 2024-07-25, then the current date - the date value = 5, which is less than the time range information 30, it can be determined that the date value of record1 also matches the time range information corresponding to the feature set 1 to be calculated, so that it can be determined that record1 matches the feature set 1 to be calculated. If the feature set 1 to be calculated contains 3 features to be calculated as shown in Table 3 below:

[0094] Features to be calculated Data Types m_max_riskscore_Login_Bank_60day max_riskscore m_max_payamount_Login_Bank_60day max_payamount m_sum_payamount_Login_Bank_60day sum_payamount

[0095] Table 3

[0096] The data entry record may include data such as the maximum risk value of the day (max_riskscore), the maximum payment amount of the day (max_payamount), the cumulative payment amount of the day (sum_payamount), etc. These data may be saved in the form of key-value pairs. For example, if max_riskscore=1, the data entry to be calculated may be input into the streaming computing operator corresponding to the feature to be calculated (m_max_riskscore_Login_Bank_60day). The streaming computing operator matches the data value of the required data type from the data entry to be calculated, i.e., max_riskscore=1, and performs a calculation based on the data value 1.

[0097] It should be noted that data type matching may not be performed in the streaming computing operator, but is performed outside the streaming computing operator. When the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, the data value of the data type required for each feature to be calculated contained in the feature set to be calculated is matched from the data entry to be calculated. If a match is found, the data value is directly obtained and input into the corresponding loss calculation operator, thereby simplifying the structure of the streaming computing operator, improving the loading and construction speed of the streaming computing operator, and helping to improve the computing efficiency when performing batch calculations on a large number of features.

[0098] A feature generation method provided by an embodiment of the present application includes obtaining metric information and data range information corresponding to a plurality of features to be calculated, wherein the data range information is used to indicate the selection range of data required for the features to be calculated; grouping the plurality of features to be calculated based on the data range information to obtain a set of features to be calculated; obtaining a stream computing operator corresponding to the features to be calculated based on the metric information corresponding to the features to be calculated, and obtaining a data entry to be calculated based on the data range information corresponding to the plurality of features to be calculated; traversing the set of features to be calculated through the data entry to be calculated, and when the data entry to be calculated matches the data range information corresponding to the set of features to be calculated, inputting the data entry to be calculated into the stream computing operator corresponding to the features to be calculated contained in the set of features to be calculated, and obtaining the feature value of the corresponding features to be calculated output by the stream computing operator. The corresponding stream computing operator can be generated for a large number of features to be calculated, and batch feature calculation can be completed by only reading and traversing the data entry to be calculated once, which can avoid repeated access to data, and the stream computing operator is used in the feature calculation process, and the content of the data entry to be calculated does not need to be stored in the calculation process, which further reduces the storage resource overhead in the batch feature calculation process and effectively improves the efficiency of batch feature calculation. It is also possible to traverse the set of features to be calculated that is generated by grouping the features to be calculated, thereby effectively improving the efficiency of the streaming computing operator that needs to input the data items to be calculated, and speeding up the generation of batch features.

[0099] Reference Figure 2 , Figure 2 is a flowchart of another feature generation method provided in an embodiment of the present application. Figure 2 As shown, the method includes:

[0100] Step 201, obtaining measurement information and data range information corresponding to a plurality of features to be calculated respectively; wherein the data range information is used to indicate a selection range of data required for the features to be calculated.

[0101] This step can refer to step 101 and will not be described in detail in this embodiment of the present application.

[0102] Step 202: group the multiple features to be calculated based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated included in each set of features to be calculated have the same data range information.

[0103] This step can be referred to as step 102, and will not be described in detail in this embodiment of the present application.

[0104] Step 203: grouping the features to be calculated in the feature set to be calculated based on the algorithm type contained in the metric information to obtain feature subsets to be calculated; wherein the features to be calculated contained in each feature subset to be calculated have the same algorithm type.

[0105] In the embodiment of the present application, after determining the set of features to be calculated, the features to be calculated contained in the set of features to be calculated may be further grouped. Specifically, the features to be calculated in the set of features to be calculated may be grouped based on the algorithm type contained in the metric information to obtain a subset of features to be calculated in the set of features to be calculated, so that the features to be calculated contained in each subset of features to be calculated have the same algorithm type. Among them, the algorithm type represents the type of calculation method of the feature to be calculated, which may include but is not limited to summing, averaging, maximum value, etc., and is not specifically limited in the embodiment of the present application.

[0106] Exemplarily, the following is an example of a feature set to be calculated and a feature subset to be calculated:

[0107] Feature set to be calculated 1 (Login_Bank_60day)

[0108] Feature subset to be calculated 1(max)

[0109] m_max_riskscore_Login_Bank_60day

[0110] m_max_payamount_Login_Bank_60day

[0111] Feature subset to be calculated 2 (sum)

[0112] m_sum_payamount_Login_Bank_60day

[0113] Feature set to be calculated 2 (Login_O2O_60day)

[0114] Feature subset to be calculated 3 (max)

[0115] m_max_riskscore_Login_O2O_60day

[0116] Feature subset to be calculated 4 (sum)

[0117] m_sum_payamount_Login_O2O_60day

[0118] Among them, the feature set 1 to be calculated includes feature subset 1 to be calculated and feature subset 2 to be calculated, the feature set 2 to be calculated includes feature subset 3 to be calculated and feature subset 4 to be calculated, the feature set 1 to be calculated corresponds to the data range information (Login_Bank_60day), the feature set 2 to be calculated corresponds to the data range information (Login_O2O_60day), and all the features to be calculated in the feature subset 1 to be calculated have the same algorithm type (max).

[0119] Step 204: obtaining data items to be calculated based on the data range information corresponding to the multiple features to be calculated.

[0120] This step can be referred to as step 103, and will not be described in detail in this embodiment of the present application.

[0121] Step 205, determining a target algorithm and an initialization value of a first variable based on the algorithm type of the feature to be calculated; wherein the target algorithm is used to update the first variable according to the data entry to be calculated and the initialization value of the first variable.

[0122] In an embodiment of the present application, in order to realize streaming computing, a streaming computing operator including a first variable can be set so that the streaming computing operator continuously updates the first variable according to the input to realize streaming computing. Specifically, the target algorithm and the initialization value of the first variable can be determined based on the measurement information of the feature to be calculated, wherein the target algorithm is used to update the first variable according to the data entry to be calculated and the initialization value of the first variable, and the target algorithm may include but is not limited to an accumulation algorithm, a maximum value algorithm, a minimum value algorithm, an average value algorithm, etc., which are not specifically limited in the embodiment of the present application.

[0123] For different algorithm types, different target algorithms and initialization values ​​of the first variable can be set. For example, for the algorithm type max, the target algorithm can be set to the "maximum value algorithm", and the initialization value of the first variable can be a sufficiently small value (such as -999999999); for the algorithm type sum, the target algorithm can be set to the "addition algorithm", and the initialization value of the first variable can be 0.

[0124] Step 206: construct a stream computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable.

[0125] In an embodiment of the present application, a streaming computing operator corresponding to the feature to be calculated can be constructed based on the target algorithm and the first variable corresponding to the feature to be calculated. The target algorithm and the initialization value of the first variable are determined based on the algorithm type of the feature to be calculated; wherein the target algorithm is used to update the first variable according to the data entry to be calculated and the initialization value of the first variable; based on the target algorithm and the first variable, a streaming computing operator corresponding to the feature to be calculated is constructed. When it is necessary to calculate a feature, the corresponding streaming computing operator can be directly constructed according to the characteristics of the feature to be calculated, so that it can adapt to a large number of different feature calculation tasks, which helps to improve the scope of application of the feature calculation using this solution.

[0126] Optionally, step 206 may include:

[0127] Sub-step 2061, initializing the second variable to 0; wherein the second variable is used to count the number of operations input into the stream computing operator.

[0128] In an embodiment of the present application, a second variable can also be set in the streaming computing operator. The second variable can be used to count the number of operations of the streaming computing operator. When constructing the streaming computing operator, the second variable can be initialized to 0, indicating that the streaming computing operator has not yet performed calculations.

[0129] Sub-step 2062, based on the target algorithm, the first variable and the second variable, construct a streaming computing operator corresponding to the feature to be calculated; wherein the streaming computing operator is also used to output first information when the traversal of the data entries to be calculated is completed and the second variable is 0, and to determine that the feature value is the variable value of the first variable when the traversal of the data entries to be calculated is completed and the second variable is non-0, and output the feature confidence corresponding to the feature value based on the variable value of the second variable.

[0130] In the embodiment of the present application, a stream computing operator corresponding to the feature to be calculated can be constructed based on the target algorithm, the first variable and the second variable. With each calculation of the stream computing operator, the second variable can be incremented by 1 to update the variable value of the second variable. For example, after the first calculation, the second variable is updated to 1, after the second calculation, the second variable is updated to 2, and so on.

[0131] After all the data entries to be calculated have been traversed, the streaming operator can output the final feature calculation result according to the value of the second variable. Specifically, if the second variable is 0, it means that the data required by the streaming operator does not exist in the data entry to be calculated. The streaming operator can output the first information, and the first information can indicate that there is no computable data or a feature calculation error. For example, a possible first information may be "feature calculation failed, no data available". When the second variable is non-zero, the variable value of the first variable in the streaming operator can be obtained as the feature value of the corresponding feature to be calculated and output.

[0132] In addition, if the value of the second variable is larger, it means that the amount of data used in the calculation of the eigenvalue is larger, and the eigenvalue is more accurate. Therefore, when the second variable is non-zero, the feature confidence of the eigenvalue can also be calculated based on the second variable, so that the larger the second variable, the larger the feature confidence. Specifically, the ratio of the variable value of the second variable to the preset value can be calculated to obtain the feature confidence, for example, feature confidence = variable value of the second variable / preset value. Since the confidence measurement standards of different features may be different, different preset values ​​can be set for feature confidence calculation for streaming computing operators corresponding to different types of features to be calculated, and the embodiments of the present application are not specifically limited.

[0133] By initializing the second variable to 0, wherein the second variable is used to count the number of operations of the input stream computing operator, and constructing the stream computing operator corresponding to the feature to be calculated based on the target algorithm, the first variable, and the second variable, the stream computing operator can output the first information when the traversal of the data entry to be calculated is completed and the second variable is 0, and when the traversal of the data entry to be calculated is completed and the second variable is non-0, determine that the feature value is the variable value of the first variable, and output the feature confidence corresponding to the feature value based on the variable value of the second variable.

[0134] Optionally, the data range information includes time period range information, and the streaming computing operator is also used to match the variable value of the second variable with the unit time quantity indicated by the time period range information. When the variable value of the second variable matches the unit time quantity, the variable value of the first variable is output and the streaming computing operator is released.

[0135] In an embodiment of the present application, the data range information may include time period range information. Since the second variable can reflect the amount of data involved in the calculation, and the time period range information contains the number of unit times that need to be calculated, for example, if the time period range information of the feature to be calculated is 30 days, then the calculation of the feature to be calculated requires 30 units of time (days) of data.

[0136] The variable value of the second variable in the streaming operator can be matched with the time period range information of the feature to be calculated corresponding to the streaming operator, and the number of unit times indicated. If the variable value of the second variable matches the number of unit times, it means that the number of calculations of the streaming operator has reached the number of unit times, that is, the data involved in the calculation has covered the complete time period indicated by the time period range information. At this time, the feature to be calculated has been calculated. In order to avoid the subsequent continuous traversal of the data entries to be calculated to affect the feature value of the feature to be calculated, the streaming operator can terminate the calculation, output the variable value of the first variable as the feature value of the corresponding feature to be calculated, and release the streaming operator to save system resources. Among them, releasing the streaming operator can mean releasing the memory space occupied by the streaming operator.

[0137] In an embodiment of the present application, the streaming computing operator can match the variable value of the second variable with the unit time quantity indicated by the time period range information. When the variable value of the second variable matches the unit time quantity, the variable value of the first variable is output, and the streaming computing operator is released. This helps the streaming computing operator to terminate the calculation in time when the data entries to be calculated are continuously traversed, ensuring that the calculation results meet the requirements of the corresponding features to be calculated, and can release the occupied resources in time after the calculation is completed, which helps to improve the system operation efficiency.

[0138] Optionally, the streaming computing operator may further include a third variable for determining target data required for the feature to be calculated, so step 206 may further include:

[0139] Sub-step 2063, based on the measurement information corresponding to the feature to be calculated, determine the target data name required for the feature to be calculated, and assign the target data name to the third variable.

[0140] In an embodiment of the present application, since the data names of the data required to calculate different feature values ​​are not the same, the measurement information of the feature to be calculated can indicate the data type required to calculate the feature value, and the data name in the data entry to be calculated may change with system updates and business changes. Therefore, the correspondence between the measurement information and the data name can be maintained, and the corresponding relationship can be queried through the measurement information corresponding to the feature to be calculated to obtain the target data name corresponding to the feature to be calculated, and the target data name is assigned to the third variable.

[0141] It should be noted that the measurement information can also be directly assigned to the third variable as the target data name, and the various fields in the measurement information can also be combined according to the preset rules to obtain the target data name and assign it to the third variable. For example, if there is a feature name to be calculated as "m_sum_payamount_Login_Bank_60day", where the algorithm type of the measurement information is "sum" and the data type is "payamount", the preset rule is: use "_" to connect the algorithm type and the data type, and the target data name "sum_payamount" can be obtained.

[0142] Sub-step 2064, based on the target algorithm, the first variable and the third variable, construct a streaming computing operator corresponding to the feature to be calculated, so that the streaming computing operator extracts the target data from the data entry to be calculated based on the third variable.

[0143] In an embodiment of the present application, a stream computing operator corresponding to the feature to be calculated can be constructed based on the target algorithm, the first variable and the third variable, so that the stream computing operator extracts the target data from the data entry to be calculated based on the third variable. The stream computing operator can match the acquired data entry to be calculated with the variable value of the third variable, determine the target data required for the corresponding feature to be calculated from the data entry to be calculated, and enable each stream computing operator to autonomously query the required data when batch computing features, which helps to improve the efficiency and operational stability of batch computing features.

[0144] For example, if the target algorithm corresponding to a feature to be calculated is the "maximum value algorithm", the initialization value of the first variable is -999999, the initialization value of the second variable is 0, and the initialization value of the third variable is "sum_payamount", the code of the constructed streaming computing operator can be as follows:

[0145] class Max(column:String,indexName:String)extends Calculator{

[0146] var maxValue=Double.MinValue

[0147] var validRecordCount = 0

[0148] override def extractData(record:Map[String,Any]):Unit={

[0149] var vl = record(column)

[0150] maxValue=if(vl>maxValue)vl else maxValue

[0151] validRecordCount += 1}}

[0152] override def calculate():String={

[0153] if(validRecordCount==0){return NOT_COUNT_DEFAULT_VALUE}

[0154] maxValue.toString}

[0155] override def clean():Unit={

[0156] maxValue = Double.MinValue

[0157] validRecordCount=0}}

[0158] Among them, maxValue represents the first variable, Double.MinValue represents the initialization value "-999999" of the first variable, validRecordCount represents the second variable, column represents the third variable, and the initialization value "sum_payamount" of the third variable can be passed in during construction. The extractData method can be the target algorithm, which is used to extract the data value indicated by the third variable from the input data entry to be calculated and update the first variable maxValue. The calculate method is used to return the calculation result. The clean method is used to reset the initial state of the streaming computing operator in the case of continuous feature calculation, so as to reduce the resource consumption of frequent release and construction of streaming computing operators.

[0159] Optionally, a stream computing operator can also be constructed based on the following steps A1 to A2:

[0160] Step A1, determining a group of features to be calculated having the same measurement information, the same non-time period range information, and the time period range information being aligned at one end from the multiple features to be calculated.

[0161] In the embodiment of the present application, the data range information may include time period range information and non-time period range information. A feature to be calculated having the same metric information and the same non-time period range information may be determined from a plurality of features to be calculated, and then a feature to be calculated with one end of the time period range information aligned may be determined from these features to be calculated to form a feature group to be calculated.

[0162] Exemplarily, the feature to be calculated m_max_riskscore_Login_Bank_30day and the feature to be calculated m_max_riskscore_Login_Bank_60day have the same measurement information (max_riskscore), and their time period range information is one for the past 30 days and the other for the past 60 days, both starting from the current time, so their time period range information is aligned at one end, and these two features to be calculated can be formed into a feature group to be calculated.

[0163] Step A2: constructing a common stream computing operator shared by all the features to be calculated in the feature group to be calculated based on the metric information corresponding to the features to be calculated in the feature group to be calculated.

[0164] In an embodiment of the present application, the calculation data required for each feature to be calculated in the same feature group to be calculated is the same, and the target algorithm is the same. The difference lies in the amount of calculation data. Therefore, based on the measurement information corresponding to the feature to be calculated in the feature group to be calculated, a common streaming calculation operator shared by all the features to be calculated in the feature group to be calculated can be constructed, so that multiple features to be calculated share the same common streaming calculation operator, thereby reducing the number of streaming calculation operators required for batch calculation of features and improving calculation efficiency.

[0165] Step 207, traverse the feature set to be calculated through the data entry to be calculated, and when the data range information corresponding to the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, input the data entry to be calculated into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, and obtain the feature value of the corresponding feature to be calculated output by the streaming computing operator.

[0166] This step can be referred to as step 104, and will not be described in detail in this embodiment of the present application.

[0167] Optionally, step 207 may include:

[0168] Sub-step 2071, traversing each feature subset to be calculated in the feature set to be calculated through the data entry to be calculated, and determining the target data set corresponding to the feature subset to be calculated from the data entry to be calculated based on the algorithm type corresponding to the feature subset to be calculated.

[0169] In the embodiment of the present application, each feature subset to be calculated in the feature set to be calculated can be traversed through the data entry to be calculated, and the algorithm type corresponding to the traversed feature subset to be calculated can be matched with each data in the data entry to be calculated. The above matching operation can be to match the algorithm type corresponding to the feature subset to be calculated with the name, key, attribute information, etc. of each data in the data entry to be calculated.

[0170] For example, if the algorithm type corresponding to a feature subset to be calculated is sum, the acquired data entry to be calculated for the sales business of a certain commodity on a certain day includes 3 data as shown in the following Table 4:

[0171] Data Name Data Value sum_revenue 10000 max_revenue 1000 min_revenue 100

[0172] Table 4

[0173] Among them, sum_revenue represents the cumulative value of a single sales, max_revenue represents the maximum value of a single sales, and min_revenue represents the minimum value of a single sales. The algorithm type sum corresponding to the feature subset to be calculated can be matched with the data contained in the above data entry to be calculated to determine that the target data set corresponding to the feature subset to be calculated contains the data sum_revenue.

[0174] It should be noted that the above-mentioned target data set may include multiple data or only one data, and the embodiment of the present application does not make any specific limitation thereto.

[0175] Sub-step 2072: input the target data set into the stream computing operator corresponding to the feature to be calculated contained in the feature subset to be calculated, and obtain the feature value of the corresponding feature to be calculated output by the stream computing operator.

[0176] After determining the target data set corresponding to each feature subset to be calculated, the target data set corresponding to the feature subset to be calculated can be input into the streaming computing operator corresponding to all the features to be calculated contained in the feature subset to be calculated, so that the streaming computing operator can filter the required target data from the target data set for feature calculation, and output the feature value corresponding to the feature to be calculated after all the required data are input.

[0177] In the embodiment of the present application, since each feature set to be calculated may contain a large number of features to be calculated, and the data entry to be calculated may also contain a large amount of data, the stream computing operator corresponding to each feature to be calculated needs to consume more computing resources to analyze and extract the target data from the complete data entry to be calculated. Therefore, by traversing each subset of features to be calculated in the feature set to be calculated through the data entry to be calculated, based on the algorithm type corresponding to the feature subset to be calculated, the target data set corresponding to the feature subset to be calculated is determined from the data entry to be calculated; the target data set is input into the stream computing operator corresponding to the feature to be calculated contained in the feature subset to be calculated, and the feature value of the corresponding feature to be calculated output by the stream computing operator is obtained, so that each stream computing operator corresponding to the feature to be calculated can only search and obtain the target data from the target data set matching its algorithm type in the data entry to be calculated, which helps to improve the operation efficiency of the stream computing operator, and then improve the efficiency of batch feature calculation.

[0178] Optionally, in the case where there is a shared common streaming computing operator, the characteristic value may be determined based on the following steps B1 to B2:

[0179] Step B1, when the data item to be calculated matches the feature group to be calculated corresponding to the feature to be calculated contained in the feature set to be calculated, the data item to be calculated is sequentially input into the public stream computing operator corresponding to the feature group to be calculated where the feature to be calculated is located.

[0180] In the embodiment of the present application, if a common streaming computing operator is generated based on the above steps A1 to A2, it can be first determined whether the data entry to be calculated matches the feature group to be calculated corresponding to the feature to be calculated contained in the feature set to be calculated. If it matches, starting from the alignment end between the time period range information of the feature to be calculated in the data group to be calculated, the data entry to be calculated is sequentially input into the common streaming computing operator corresponding to the feature group to be calculated where the feature to be calculated is located in the order of data collection time of the data entry to be calculated. Among them, the matching of the data entry to be calculated and the feature group to be calculated indicates that the data source of the data entry to be calculated is the same as the non-time period range information of the feature to be calculated contained in the feature group to be calculated.

[0181] Step B2, after all the data items to be calculated that match the time period range information of the feature to be calculated are input into the common stream computing operator corresponding to the feature to be calculated, the variable value of the first variable in the common stream computing operator is obtained to obtain the feature value of the feature to be calculated.

[0182] In an embodiment of the present application, the time period range information of each feature to be calculated in the feature group to be calculated results in different numbers of data entries to be calculated required for each feature to be calculated in the feature group to be calculated. For example, if there are two features to be calculated in a feature group to be calculated, and the time period range information of one feature a to be calculated is 30 days, and the time period range information of the other feature b to be calculated is 60 days, then the feature a to be calculated needs the data entries to be calculated in the previous 30 days (30) to calculate the feature value, and the feature b to be calculated needs the data entries to be calculated in the previous 60 days (60) to calculate the feature value. Therefore, the data entries to be calculated in the previous 60 days can be input into the common stream computing operator corresponding to the feature group to be calculated, starting from the most recent data entry to be calculated, in sequence. When 30 data entries to be calculated are input, the variable value of the first variable in the common stream computing operator is read to obtain the feature value of the feature a to be calculated. When 60 data entries to be calculated are input, the variable value of the first variable in the common stream computing operator is read again to obtain the feature value of the feature b to be calculated.

[0183] Furthermore, a fourth variable sequence can also be set in the common streaming computing operator according to the number of unit times contained in the time period range information of the feature to be calculated corresponding to the feature group to be calculated, so that when the number of calculations of the common streaming computing operator (the second variable) matches the fourth variable in the fourth variable sequence, the first variable is automatically output as the characteristic value of the feature to be calculated corresponding to the fourth variable in the feature group to be calculated.

[0184] In the embodiment of the present application, a group of features to be calculated that have the same metric information, the same non-period range information, and one end of the period range information is aligned can be determined from the multiple features to be calculated; based on the metric information corresponding to the features to be calculated in the group of features to be calculated, a common stream computing operator shared by all the features to be calculated in the group of features to be calculated is constructed. In this way, multiple features to be calculated can share the same common stream computing operator, avoiding repeated calculations, which helps to improve the efficiency of batch feature calculations.

[0185] Another feature generation method provided by the embodiment of the present application includes: obtaining metric information and data range information corresponding to multiple features to be calculated respectively; wherein the data range information is used to indicate the selection range of data required for the features to be calculated; grouping multiple features to be calculated based on the data range information to obtain a set of features to be calculated; obtaining a stream computing operator corresponding to the features to be calculated based on the metric information corresponding to the features to be calculated, and obtaining a data entry to be calculated based on the data range information corresponding to the multiple features to be calculated; traversing the set of features to be calculated through the data entry to be calculated, and when the data entry to be calculated matches the data range information corresponding to the set of features to be calculated, inputting the data entry to be calculated into the stream computing operator corresponding to the features to be calculated contained in the set of features to be calculated, and obtaining the feature value of the corresponding feature to be calculated output by the stream computing operator. The corresponding stream computing operator can be generated for a large number of features to be calculated, and batch feature calculation can be completed by only reading and traversing the data entry to be calculated once, which can avoid repeated access to data, and the stream computing operator is used in the feature calculation process, and the content of the data entry to be calculated does not need to be stored in the calculation process, which further reduces the storage resource overhead in the batch feature calculation process and effectively improves the efficiency of batch feature calculation. It is also possible to traverse the set of features to be calculated that is generated by grouping the features to be calculated, thereby effectively improving the efficiency of the streaming computing operator that needs to input the data items to be calculated, and speeding up the generation of batch features.

[0186] Corresponding to the method provided in the above-mentioned feature generation method embodiment of the present application, see Figure 3 , the present application also provides a structural diagram of a feature generation device, such as Figure 3 As shown, in this embodiment, the feature generating device 30 includes:

[0187] The acquisition module 31 is used to acquire the measurement information and data range information corresponding to the plurality of features to be calculated respectively; wherein the data range information is used to indicate the selection range of the data required for the features to be calculated;

[0188] A collection module 32, configured to group the plurality of features to be calculated based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated included in each set of features to be calculated have the same data range information;

[0189] An operator module 33 is used to obtain a stream computing operator corresponding to the feature to be calculated based on the metric information corresponding to the feature to be calculated, and to obtain a data entry to be calculated based on the data range information corresponding to the multiple features to be calculated; wherein the stream computing operator is used to calculate a feature value of the feature to be calculated based on target data in the data entry to be calculated that matches the data type of the feature to be calculated;

[0190] The traversal module 34 is used to traverse the feature set to be calculated through the data entry to be calculated. When the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, the data entry to be calculated is input into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, so as to obtain the feature value of the corresponding feature to be calculated output by the streaming computing operator.

[0191] Optionally, the device further comprises:

[0192] A subset module, used for grouping the features to be calculated in the feature set to be calculated based on the algorithm type contained in the metric information to obtain a subset of features to be calculated; wherein the features to be calculated contained in each subset of features to be calculated have the same algorithm type;

[0193] The traversal module includes:

[0194] A target data set submodule, configured to traverse each feature subset to be calculated in the feature set to be calculated through the data entry to be calculated, and determine a target data set corresponding to the feature subset to be calculated from the data entry to be calculated based on the algorithm type corresponding to the feature subset to be calculated;

[0195] The feature value submodule is used to input the target data set into the stream computing operator corresponding to the feature to be calculated contained in the feature subset to be calculated, and obtain the feature value of the corresponding feature to be calculated output by the stream computing operator.

[0196] Optionally, the operator module includes:

[0197] An initialization submodule, used to determine a target algorithm and an initialization value of a first variable based on the algorithm type of the feature to be calculated; wherein the target algorithm is used to update the first variable according to the data item to be calculated and the initialization value of the first variable;

[0198] An operator construction submodule is used to construct a streaming computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable.

[0199] Optionally, the stream computing operator includes a second variable, and the operator construction submodule includes:

[0200] A first initialization unit, configured to initialize the second variable to 0; wherein the second variable is used to count the number of operations input into the stream computing operator;

[0201] A first construction unit is used to construct a streaming computing operator corresponding to the feature to be calculated based on the target algorithm, the first variable and the second variable; wherein the streaming computing operator is also used to output first information when the traversal of the data entries to be calculated is completed and the second variable is 0, and to determine that the feature value is the variable value of the first variable when the traversal of the data entries to be calculated is completed and the second variable is non-0, and to output the feature confidence corresponding to the feature value based on the variable value of the second variable.

[0202] Optionally, the data range information includes time period range information, and the streaming computing operator is also used to match the variable value of the second variable with the unit time quantity indicated by the time period range information. When the variable value of the second variable matches the unit time quantity, the variable value of the first variable is output and the streaming computing operator is released.

[0203] Optionally, the stream computing operator further includes a third variable, and the operator construction submodule includes:

[0204] A second initialization unit, configured to determine a target data name required for the feature to be calculated based on the measurement information corresponding to the feature to be calculated, and assign the target data name to the third variable;

[0205] The second construction unit is used to construct a streaming computing operator corresponding to the feature to be calculated based on the target algorithm, the first variable and the third variable, so that the streaming computing operator extracts the target data from the data entry to be calculated based on the third variable.

[0206] Optionally, the stream computing operator includes a common stream computing operator, the data range information includes time period range information and non-time period range information; the operator module includes:

[0207] The feature group to be calculated submodule is used to determine, from the multiple features to be calculated, a feature group to be calculated that has the same metric information, the same non-time period range information, and one end of the time period range information is aligned;

[0208] A construction submodule, configured to construct a common stream computing operator shared by all the features to be calculated in the feature group to be calculated based on the metric information corresponding to the features to be calculated in the feature group to be calculated;

[0209] The traversal module includes:

[0210] A public stream computing operator input submodule, for, when the data item to be calculated matches the feature group to be calculated corresponding to the feature to be calculated contained in the feature set to be calculated, inputting the data item to be calculated into the public stream computing operator corresponding to the feature group to be calculated where the feature to be calculated is located in sequence according to the data collection time of the data item to be calculated;

[0211] The common streaming computing operator output submodule is used to obtain the variable value of the first variable in the common streaming computing operator after all the data entries to be calculated that match the time period range information of the feature to be calculated are input into the common streaming computing operator corresponding to the feature to be calculated, so as to obtain the feature value of the feature to be calculated.

[0212] The feature generation device provided by the embodiment of the present application can generate corresponding streaming computing operators for a large number of features to be calculated, and batch feature calculation can be completed by reading and traversing the data items to be calculated once, which can avoid repeated access to data, and the feature calculation process is performed through streaming computing operators. The content of the data items to be calculated does not need to be stored during the calculation process, which further reduces the storage resource overhead during the batch feature calculation process and effectively improves the efficiency of batch feature calculation. It is also possible to traverse the set of features to be calculated generated by grouping the features to be calculated, effectively improving the efficiency of the streaming computing operators that need to be input to determine the data items to be calculated, and speeding up the generation of batch features.

[0213] Figure 4 6 is a logic block diagram of an electronic device 600 according to an embodiment of the present application. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0214] Reference Figure 4 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0215] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0216] The memory 604 is used to store various types of data to support the operation of the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0217] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0218] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0219] The audio component 610 is used to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0220] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0221] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of contact between the user and the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0222] The communication component 616 is used to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0223] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to implement a feature generation method provided in an embodiment of the present application.

[0224] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to perform the above method. For example, the non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0225] Figure 5700 is a logic block diagram of an electronic device 700 according to an embodiment of the present application. For example, the electronic device 700 may be provided as a server. Figure 5 The electronic device 700 includes a processing component 722, which further includes one or more processors, and a memory resource represented by a memory 732 for storing instructions executable by the processing component 722, such as an application. The application stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform a feature generation method provided in an embodiment of the present application.

[0226] The electronic device 700 may also include a power supply component 726 configured to perform power management of the electronic device 700, a wired or wireless network interface 750 configured to connect the electronic device 700 to a network, and an input / output (I / O) interface 758. The electronic device 700 may operate based on an operating system stored in the memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0227] An embodiment of the present application also provides a computer program product, including a computer program, which implements the feature generation method when executed by a processor.

[0228] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0229] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A feature generation method, characterized in that: The feature generation method comprises: Obtaining measurement information and data range information corresponding to a plurality of features to be calculated, respectively; wherein the data range information is used to indicate a selection range of data required for the features to be calculated; The plurality of features to be calculated are grouped based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated included in each set of features to be calculated have the same data range information; Based on the metric information corresponding to the feature to be calculated, a stream computing operator corresponding to the feature to be calculated is obtained, and based on the data range information corresponding to the multiple features to be calculated, a data entry to be calculated is obtained; wherein the stream computing operator is used to calculate the feature value of the feature to be calculated based on the target data in the data entry to be calculated that matches the data type of the feature to be calculated; The feature set to be calculated is traversed through the data entry to be calculated. When the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, the data entry to be calculated is input into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, and the feature value of the corresponding feature to be calculated output by the streaming computing operator is obtained.

2. The method according to claim 1, characterized in that After the plurality of features to be calculated are grouped based on the data range information to obtain a set of features to be calculated, the method further includes: The features to be calculated in the feature set to be calculated are grouped based on the algorithm type contained in the metric information to obtain a feature subset to be calculated; wherein the features to be calculated contained in each feature subset to be calculated have the same algorithm type; The step of inputting the data item to be calculated into the stream computing operator corresponding to the feature to be calculated included in the feature set to be calculated, and obtaining the feature value of the feature to be calculated output by the stream computing operator, includes: Traversing each feature subset to be calculated in the feature set to be calculated through the data entry to be calculated, and determining a target data set corresponding to the feature subset to be calculated from the data entry to be calculated based on the algorithm type corresponding to the feature subset to be calculated; The target data set is input into the stream computing operator corresponding to the feature to be calculated contained in the feature subset to be calculated, and the feature value of the corresponding feature to be calculated output by the stream computing operator is obtained.

3. The method according to claim 1, characterized in that The acquiring, based on the metric information corresponding to the feature to be calculated, the streaming computing operator corresponding to the feature to be calculated includes: Determining a target algorithm and an initialization value of a first variable based on the algorithm type of the feature to be calculated; wherein the target algorithm is used to update the first variable according to the data item to be calculated and the initialization value of the first variable; Based on the target algorithm and the first variable, a streaming computing operator corresponding to the feature to be calculated is constructed.

4. The method according to claim 3, characterized in that The stream computing operator includes a second variable, and constructing the stream computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable includes: Initialize the second variable to 0; wherein the second variable is used to count the number of operations input into the stream computing operator; Based on the target algorithm, the first variable and the second variable, a streaming computing operator corresponding to the feature to be calculated is constructed; wherein the streaming computing operator is also used to output first information when the traversal of the data entries to be calculated is completed and the second variable is 0, and to determine that the feature value is the variable value of the first variable when the traversal of the data entries to be calculated is completed and the second variable is non-0, and to output the feature confidence corresponding to the feature value based on the variable value of the second variable.

5. The method according to claim 4, characterized in that The data range information includes time period range information, and the streaming computing operator is also used to match the variable value of the second variable with the unit time quantity indicated by the time period range information. When the variable value of the second variable matches the unit time quantity, the variable value of the first variable is output and the streaming computing operator is released.

6. The method according to claim 3, characterized in that The stream computing operator further includes a third variable, and constructing the stream computing operator corresponding to the feature to be calculated based on the target algorithm and the first variable includes: Based on the measurement information corresponding to the feature to be calculated, determine the target data name required for the feature to be calculated, and assign the target data name to the third variable; Based on the target algorithm, the first variable and the third variable, a stream computing operator corresponding to the feature to be calculated is constructed, so that the stream computing operator extracts the target data from the data entry to be calculated based on the third variable.

7. The method according to claim 1, characterized in that The stream computing operator includes a common stream computing operator, the data range information includes time period range information and non-time period range information; the acquiring the stream computing operator corresponding to the feature to be calculated based on the metric information corresponding to the feature to be calculated includes: Determine, from the plurality of features to be calculated, a group of features to be calculated that have the same measurement information, the same non-period range information, and one end of the period range information is aligned; Based on the metric information corresponding to the features to be calculated in the feature group to be calculated, construct a common stream computing operator shared by all the features to be calculated in the feature group to be calculated; The step of inputting the data item to be calculated into the stream computing operator corresponding to the feature to be calculated included in the feature set to be calculated, and obtaining the feature value of the feature to be calculated output by the stream computing operator, includes: In the case where the data item to be calculated matches the feature group to be calculated corresponding to the feature to be calculated included in the feature set to be calculated, the data item to be calculated is sequentially input into the common stream computing operator corresponding to the feature group to be calculated where the feature to be calculated is located according to the data collection time of the data item to be calculated; After all the data items to be calculated that match the time period range information of the feature to be calculated are input into the common stream computing operator corresponding to the feature to be calculated, the variable value of the first variable in the common stream computing operator is obtained to obtain the feature value of the feature to be calculated.

8. A feature generation device, characterized in that: The feature generating device comprises: An acquisition module, used to acquire measurement information and data range information corresponding to a plurality of features to be calculated, wherein the data range information is used to indicate a selection range of data required for the features to be calculated; A collection module, used for grouping the plurality of features to be calculated based on the data range information to obtain a set of features to be calculated; wherein the features to be calculated contained in each set of features to be calculated have the same data range information; An operator module, used to obtain a stream computing operator corresponding to the feature to be calculated based on the metric information corresponding to the feature to be calculated, and to obtain a data entry to be calculated based on the data range information corresponding to the multiple features to be calculated; wherein the stream computing operator is used to calculate a feature value of the feature to be calculated based on target data in the data entry to be calculated that matches the data type of the feature to be calculated; A traversal module is used to traverse the feature set to be calculated through the data entry to be calculated. When the data entry to be calculated matches the data range information corresponding to the feature set to be calculated, the data entry to be calculated is input into the streaming computing operator corresponding to the feature to be calculated contained in the feature set to be calculated, so as to obtain the feature value of the corresponding feature to be calculated output by the streaming computing operator.

9. An electronic device, characterized in that: include: The electronic device comprises a processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the feature generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the feature generation method as claimed in any one of claims 1 to 7.

Images