Incremental calculation updating method based on expert statistical features, electronic device, medium

Abstract

The application discloses a kind of based on expert statistical feature incremental calculation updating method, electronic equipment, medium, the method of the present application is divided into offline calculation and online calculation, wherein offline calculation first reads the configuration information of expert statistical feature;Then extract the user statistical association information of the statistical feature of historical data;Finally, the statistical association information of historical data is persisted.Online calculation first reads the configuration information of expert statistical feature;Then extract the user statistical association information of the statistical feature of new data;While reading the user statistical association information persisted by historical data;Next, the statistical association information generated based on historical and new data extraction is updated offline data statistical association information and executes persistence operation;Finally, the corresponding statistical feature is generated based on the statistical association information of updated offline data.The method of the present application can still efficiently output statistical features for low server resources.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence feature engineering, statistical variable calculation and data storage technology, and in particular to an incremental calculation and update method based on expert statistical features, an electronic device, and a medium. Background Art

[0002] For artificial intelligence (AI), data and features determine the upper limit of machine learning, while models and algorithms only approximate this upper limit. Therefore, feature engineering plays a crucial role in machine learning. In practical applications, the data and features generated during the feature engineering phase are crucial to the success of machine learning. Existing automatic feature generation techniques generate a large number of new features by performing simple transformations and aggregations on low-level features. These techniques then retain the most important features and use them to retrain the model. This is a common approach, but all of this relies on extremely powerful computing resources. Most users lack sufficient computing resources to handle such a high computational load. Therefore, constructing appropriate sample features within traditional models, using limited computing resources, poses new challenges for feature engineering. Constructing such features often requires expert expertise. Existing expert experience features can generally be categorized as static features and dynamic features: features derived by combining and calculating multiple simple low-level features, and features derived by performing statistical operations on these features in the spatiotemporal dimension.

[0003] In conventional feature statistical calculations, static features are typically obtained by simply executing calculation logic on multiple low-level features. However, dynamic features require statistical calculations on some or all of the historical data. Obtaining these features requires powerful computing resources or a long period of time, which inadvertently increases the computational complexity and cost of generating expert features.

[0004] Therefore, it is urgent to propose an incremental calculation and update method based on expert statistical features, which can still efficiently output statistical features in the case of low server resources. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention proposes an incremental calculation and updating method based on expert statistical features.

[0006] To achieve the above technical objectives, the technical solution of the present invention is as follows: A first aspect of an embodiment of the present invention provides an incremental calculation and update method based on expert statistical features, the method specifically comprising the following steps:

[0007] (1) Constructing an expert statistical feature configuration file, wherein the expert statistical feature configuration file includes several fields, wherein the column name of the first field is the name of the statistical expert feature, and the column names of the remaining fields include the statistical calculation category corresponding to the statistical expert feature, the grouping attribute name, the filtering condition associated attribute name, the filtering condition, the attribute name of the statistical operation performed, and the statistical operation type;

[0008] (2) reading the expert statistical feature configuration file constructed in step (1), using the first field name as the primary key, verifying the remaining fields according to the statistical calculation category corresponding to the statistical expert statistics, and then splicing the remaining fields that pass the verification into a linked list as the value to form a hash map;

[0009] (3) Offline generation of associated statistical information of historical data: extracting the expert statistical features to be generated according to the hash map obtained in step (2), obtaining the calculation category of the corresponding expert statistical features according to the value obtained in step (2), and saving the associated statistical information of the historical data corresponding to the statistical expert features according to the calculation category of the statistical expert features;

[0010] (4) persisting the associated statistical information of the historical data saved in step (3);

[0011] (5) Generate the associated statistical information of the newly added data online, extract the expert statistical features to be generated according to the hash map obtained in step (2), obtain the calculation category of the corresponding expert statistical features according to the value obtained in step (2), and save the statistical associated information of the newly added data corresponding to the statistical expert features according to the statistical calculation category corresponding to the statistical expert features;

[0012] (6) updating the historical correlation statistical information, updating the corresponding historical correlation statistical information according to the correlation statistical information of the historical data saved in step (4) and the statistical correlation information of the newly added data generated in step (5), taking the current newly added data time as the new historical data time node, obtaining the updated historical correlation statistical information, and persisting it as the new historical data statistical correlation information;

[0013] (7) Using the historical data statistical correlation information updated in step (6), the numerical value of the statistical expert feature is inferred.

[0014] A second aspect of an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned incremental calculation update method based on expert statistical features.

[0015] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned incremental calculation and updating method based on expert statistical features when executed by a processor.

[0016] Based on the above technical background and business scenarios, the present invention provides a general, efficient, highly compatible and practical method for calculating the statistical characteristics of financial risk control experts.

[0017] The beneficial effects of the present invention are as follows: the method of the present invention provides an incremental calculation and update method based on expert statistical features, which obtains the associated statistical information of historical data and the associated statistical information of new data through offline calculation and online calculation, and persists the statistical association information of historical data. The updated statistical association information of offline data is generated based on the statistical association information extracted from historical and new data, and a persistence operation is performed; finally, the corresponding statistical features are generated based on the statistical association information of the updated offline data. Compared with the existing full data statistical feature update method, the incremental calculation and update method based on expert statistical features provided by the present invention can achieve a speed increase of more than 10 times on the test data set; in addition, for the case of low server resources, the method of the present invention can still efficiently output statistical features, and can be applied to real-time data inference of AI models in financial risk control business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a method for updating statistical features of financial risk control experts based on incremental calculation according to the present invention;

[0019] Figure 2 The figure is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0021] The present invention will be described in detail below with reference to the accompanying drawings. Unless there is any conflict, the features of the following embodiments and implementations may be combined with each other.

[0022] The present invention provides an incremental calculation update method based on expert statistical features, such as Figure 1 As shown, the specific steps include:

[0023] (1) Construct expert statistical feature profiles, including:

[0024] Configure the column name of the first field. The first field participates in statistical calculations in the original data form and uses the statistical expert feature as the column name of the first field.

[0025] Configure the column name of the second field, which is composed of the categories corresponding to the statistical expert features in the first field; in an embodiment of the present invention, the categories of the expert statistical features include: historical data statistical values ​​based on a certain feature, historical data statistical values ​​based on a certain feature within a specific valid time, statistical values ​​of the number of occurrences within a specific valid time, and numerical statistical values ​​based on recent periods of a certain feature.

[0026] Configure the column name of the third field. The third field is composed of the column names corresponding to the grouped statistical features formed by performing grouped statistical operations on the original data table that generates the expert statistical features of the first field, and the third field must be the column names in the original table where the statistical operations are performed.

[0027] Configure the column name of the fourth field. The fourth field consists of the column name of the original data table that generates the expert statistical features of the first field and the filter condition when performing the conditional filter operation. This field must be the column name in the original table where the statistical operation is performed.

[0028] Configure the column name of the fifth field. The fifth field consists of the conditional range values ​​set by the filter conditions. It can be subdivided into: pure numeric type, and the combined representation type of the numeric value and its corresponding unit of measurement. When there are multiple parameters, the parameters are separated by "#".

[0029] Configure the column name of the sixth field. The sixth field consists of the column name to which the statistical result value of the expert statistical feature output of the generated first field belongs, and this field must be the column name in the original table where the statistical operation is performed.

[0030] The column name of the seventh field is configured. The seventh field consists of the statistical operation type required to be performed to generate the expert statistical feature identified by the first field based on the original table value associated with the column name of the sixth field.

[0031] As a preferred solution, in the configured field list, the fields are separated by commas.

[0032] (2) Read the expert statistical feature configuration information, use the corresponding first field name as the primary key (Key), and splice the subsequent field names into a linked list as the value (Value) to form a hash map (HashMap).

[0033] As a preferred solution, in step (2), arrays are used to concatenate HashMaps, as follows:

[0034] Read each record in the expert statistical feature configuration file constructed in step (1), use the value of the first field as the key, verify the values ​​of the second to seventh fields, and construct a linked list of the values ​​of the second to seventh fields that have passed the verification, and use the linked list as the value;

[0035] The configuration validity of the second to seventh fields is verified and judged, specifically judging the following conditions:

[0036] Condition 1: Whether the category configured in the second field of the expert statistical feature configuration file is within the configured category range (the configured category range is based on the historical data statistics of a feature, the historical data statistics of a feature within a specific valid time, the statistical value of the number of occurrences within a specific valid time, and the numerical statistics of a feature in recent periods);

[0037] Condition 2: Determines whether the values ​​of the third to seventh fields meet the non-null value requirement based on the configured category;

[0038] Exemplarily, according to the category configured in the second field, which is to perform numerical statistics based on historical data of a certain feature, it is determined whether the third field, the fourth field, the sixth field, and the seventh field are empty.

[0039] For example, if the category configured in the second field is execution numerical statistics based on historical data of a certain feature within a specific valid time, it is determined whether the third field, the fourth field, the fifth field, the sixth field, and the seventh field are empty.

[0040] If any of the conditions is not met, there is a problem with the expert statistical feature configuration file, and the fields that do not meet the category requirements are empty values.

[0041] (3) Generate the associated statistical information of historical data offline: Extract the expert statistical features to be generated according to the HashMap obtained in step (2), obtain the calculation category of the corresponding expert statistical features according to the Value, and save the statistical association information of the expert statistical features according to the calculation category of the expert statistical features to ensure that the generated historical data statistical association information can be used to derive the corresponding expert statistical features.

[0042] Exemplarily, the value of each expert feature in step (3) is generated according to the extraction of the statistical expert feature in step (2) by using the HashMap, the statistical calculation category of the corresponding expert feature is obtained according to the Value (i.e. determined by the second field in step (1)), and the saving of different statistical association information is respectively performed according to the historical data statistical value of a certain feature (STA_TYPE_HISTORY_VALUE), the historical data statistical value of a certain feature within a certain effective time (STA_TYPE_RECENT_DATE_VALUE), the statistical value of the number of occurrences within a certain effective time (STA_TYPE_RECENT_DATE), and the numerical statistical value of a certain feature in recent periods (STA_TYPE_RECENT_PERIODS_VALUE).

[0043] And in the process of saving different statistical association information, the saved association information is determined by the following contents:

[0044] Content 1: The third value of the Value value chain table obtained in step (2), i.e. the fourth field (i.e. the statistical filtering condition association attribute name) described in step (1), determines the executed filtering condition attribute.

[0045] Content 2: The fifth value of the Value value chain table obtained in step (2), i.e. the sixth field (i.e. the statistical result dependent attribute name) described in step (1), determines the executed statistical operation associated attribute.

[0046] Among them, the association information needs to be stored in independent chain table structures according to content 1 and content 2 respectively.

[0047] Exemplarily, in the process of saving statistical association information according to the statistical calculation category of the historical data statistical value of a certain feature (STA_TYPE_HISTORY_VALUE), the filtering condition is determined by the above content 1, the filtering is performed, and the stored association information [Value1, Value2, Value3...] is obtained.

[0048] Exemplarily, in the process of saving statistical association information according to the statistical calculation category of the attribute column of the number of executions of a certain feature within a certain effective time (STA_TYPE_RECENT_DATE_VALUE), the saved association information is determined by the above content 1 and content 2, and the stored association information [Date1, Date2, Date3...] and [Value1, Value2, Value3...] is obtained.

[0049] For example, in the process of saving statistical association information according to the statistical calculation category of the statistical value based on the number of occurrences within the valid time (STA_TYPE_RECENT_DATE), the association information to be saved is determined by the above content 2, and the stored association information [Date1, Date2, Date3...] is obtained.

[0050] (4) Store the historical data association information generated in offline mode, persist the statistical information associated with the historical data, and ensure the persistence of the associated statistical information.

[0051] For example, in step (4), you can choose to use a database such as MySQL, ORACLE, or use a file such as csv, xls, pkl, etc. to store data.

[0052] (5) Generate the associated statistical information of the newly added data online, extract the expert statistical features to be generated according to the hash map (HashMap) obtained in step (2), obtain the calculation category of the corresponding expert statistical features according to the Value, and save the statistical associated information of the newly added data corresponding to the expert statistical features according to the calculation category of the expert statistical features, so as to ensure that the generated statistical associated information of the newly added data can be used to derive the corresponding expert statistical features.

[0053] Exemplarily, in step (5):

[0054] (5.1) Generate the numerical value of each expert feature by extracting the expert statistical feature to be generated according to the Hash Map in step (2), and obtain the statistical calculation category of the corresponding expert feature according to the Value, that is, the second field in step (1) determines the execution of different statistical association information storage respectively;

[0055] Specifically, the stored association information is determined by the following:

[0056] Content 1: Determine the filter condition attribute to be executed based on the third value of the Value list obtained in step (2), i.e., the fourth field in step (1);

[0057] Content 2: According to the fifth value of the Value list obtained in step (2), that is, the sixth field in step (1), the associated attribute of the statistical operation to be performed is determined;

[0058] Among them, it is necessary to save the associated information of the newly added data that meets the statistical conditions, and the associated information is stored in an independent linked list structure according to content 1 and content 2 respectively.

[0059] (6) Update the historical correlation statistical information. According to the historical statistical correlation information saved in step (4) and the correlation statistical information of the newly added data generated online in step (5), update the corresponding historical correlation statistical information, use the current newly added data time as the new historical data time node, obtain the updated historical correlation statistical information, and persist it as the new historical correlation statistical information.

[0060] Exemplarily, in step (6):

[0061] (6.1) Incorporate the associated statistical information of the historical data into the statistical associated information of the newly added data, and update the associated statistical information of the historical data with the statistical associated information of the newly added data to form an updated version of the associated statistical information of the historical data. Then perform a persistence operation, wait for the next newly added data, and perform the next round of iteration of the associated statistical information of the historical data.

[0062] (6.1) Incorporating the statistical information associated with historical data into the statistical information associated with newly added data can be done in the following two ways:

[0063] Method 1: Based on the statistical correlation information of the newly added data, find the historical statistical correlation information of the newly added data that needs to be merged from the statistical correlation information of the historical data.

[0064] Method 2: Based on the filtering conditions of the statistical values ​​(i.e., the attribute name associated with the fourth field filtering condition and the fifth field statistical filtering condition), find the data that needs to be updated in the statistical association information of the historical data, perform an update operation on this part of the data, and merge the updated information into the statistical association information of the newly added data.

[0065] (6.2) There are two ways to update historical data with new data:

[0066] Method 1: For the part that overlaps with historical data information, the statistical correlation information of the historical data is replaced by the statistical correlation information of the newly added data.

[0067] Method 2: For the parts that do not overlap with historical data information, the newly added data statistical related information is directly incorporated into the data.

[0068] (7) Output expert statistical features. Based on the new historical data association statistical information generated in step (6), output expert statistical features for the data of the next inference cycle.

[0069] For example, in step (7): using the historical data statistical association information updated in step (6), statistical operations such as summing (sum), finding the maximum value (max), finding the minimum value (min), finding the average value (avg), counting (count), and extracting unique values ​​(unique) are performed on the linked list information of the stored association statistical information, so as to quickly calculate the numerical value of the expert statistical feature of the inferred data.

[0070] Example 1

[0071] (1) Construct expert statistical feature profile:

[0072] Table 1 is the table structure of the expert statistical feature configuration file of the embodiment. The business scenario of this example is to provide relevant statistical information of expert statistical features in the field of financial risk control.

[0073] Table 1: Table structure of expert statistical feature configuration file

[0074]

[0075] Configure the column name of the first field and use the statistical expert feature name (new_generated_col_name) generated in the original data form as the column name of the first field.

[0076] The column name of the second field is configured as the statistical calculation category (statistic_type) abstracted from the statistical features of statistical experts. It is based on the statistical calculation category abstracted from the expert statistical features of the financial (covering the banking and insurance fields) risk control system. There are four statistical calculation categories, namely: historical statistical values ​​based on a certain feature (STA_TYPE_HISTORY_VALUE), attribute columns based on the execution numerical statistics of a certain feature within the valid time (STA_TYPE_RECENT_DATE_VALUE), statistical values ​​based on the number of occurrences within the valid time (STA_TYPE_RECENT_DATE), and numerical statistical values ​​based on the recent periods of a certain feature (STA_TYPE_RECENT_PERIODS_VALUE).

[0077] Configure the column name of the third field as the name of the statistical grouping attribute (col_groupby_name), configure the column name of the fourth field as the name of the statistical filtering condition associated attribute (col_filter), configure the column name of the fifth field as the statistical filtering condition (filter_condition), configure the column name of the sixth field as the name of the attribute that the statistical result depends on (col_value), and configure the column name of the seventh field as the statistical operation type (calc_func).

[0078] (2) Read each piece of information in the expert statistical feature configuration file constructed in step (1):

[0079] The first field name is used as the key, and the subsequent field names are concatenated into a List as the value to form a hash map (HashMap). At the same time, experts are required to statistically analyze the legality of the feature configuration and determine if the feature configuration is illegal. The feature generation will be canceled. The steps for legality judgment are as follows:

[0080] (2.1) Whether the statistical calculation category belongs to a legal configuration item;

[0081] (2.2) Whether the parameters that cannot be defaulted in the statistical calculation category are configured reasonably and correctly;

[0082] (2.3) If the expert statistical feature is not configured correctly, the feature will not be placed in the HashMap.

[0083] Figure 2 The non-empty parameter configuration items are the parameter items that cannot be defaulted for the corresponding statistical calculation category. After selecting the statistical calculation category, the corresponding parameter items must be filled in, otherwise the corresponding expert features cannot be generated.

[0084] (3) Generate relevant statistical information of historical data offline:

[0085] According to the hash map (HashMap) generated in step (2), the statistical calculation category parameters in the Value list are obtained, and the associated information statistics of different offline historical data are performed according to the calculation category parameters. The purpose of storing the associated information is to effectively execute the subsequent iteration of the newly added data and ensure that the expert statistical features are generated correctly and quickly. Therefore, for different statistical calculation category data, the associated information that needs to be stored is shown in Table 2 below.

[0086] Table 2: Statistics calculation category and storage association information table

[0087]

[0088] In order to optimize the performance of updating the associated statistical information of historical data using the newly added data, when the stored associated information includes a Date linked list, the Date value with the smallest value in the linked list will be recorded at the same time.

[0089] (4) Persistent association statistics:

[0090] It is necessary to save the associated statistical information of the historical data generated offline. In this example, a CSV file is used for storage. The associated statistical information table of the historical data generated offline in the embodiment is shown in Table 3 below.

[0091] Table 3: Correlation statistics of offline generated historical data

[0092]

[0093]

[0094] (5) Generate online statistical information about newly added data:

[0095] This step is similar to the generation process in step 3). Because the amount of data updated online is much smaller than the amount of offline data, generating the correlation statistics for the newly added data online requires less computation and results can be obtained quickly. Table 4 shows the correlation statistics for the newly added data generated online in this embodiment (for easier comparison, this example stores the data; actual online calculations do not require persistence).

[0096] Table 4: Correlation statistics of newly added data generated online

[0097]

[0098] (6) Update historical association statistics:

[0099] For the statistical association information of updated data and historical data, a two-step approach is adopted. First, the statistical association information of historical data involving update operations (statistical calculation categories with time windows, i.e., STA_TYPE_RECENT_DATE_VALUE and STA_TYPE_RECENT_DATE) is filtered. The filtering is based on the value of the minimum valid time attribute column of the effective time list of the statistical association information (see Table 4, i.e., the offline_Date_count_within_12_month_date_min attribute column). The statistical association information of historical data whose minimum valid time in the effective time list of the association information does not meet the time window requirements needs to be filtered based on the time window and this part of the historical data information is incorporated into the statistical association information of the updated data. Next, for the statistical association information of historical data and the statistical association information of updated data, in the case where there are the same grouping values ​​(see Table 4, i.e., the ID attribute column), the statistical association information of the updated data shall prevail; for the grouping values ​​that only exist in the statistical association information of historical data (see Table 4, i.e., the ID attribute column), the statistical association information of the historical data shall prevail; for the grouping values ​​that only exist in the statistical association information of updated data (see Table 4, i.e., the ID attribute column), the statistical association information of the updated data shall prevail. An updated version of the historical data association statistical information is formed, and a persistence operation is performed, waiting for the next new data to be added, and the next round of iteration of the association statistical information of the historical data is performed. The information table after the operation of updating the historical association statistical information in the embodiment is shown in Table 5.

[0100] Table 5: Updated historical data association statistics table

[0101]

[0102] (7) Output expert statistical features:

[0103] The history association statistical information is updated according to step (6), and the sum, max, min, avg, count, unique, and other statistical operations are performed on the stored association statistical information according to the category of the statistical operation in the Value chain table of the HashMap generated in step (2), so that the corresponding expert statistical feature values can be quickly obtained.

[0104] The application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the incremental calculation and update method based on expert statistical features as described above. As Figure 2 shown, a hardware structure diagram of any data processing capable device where the device based on the expert statistical feature incremental calculation and update method provided by the embodiment of the application is located, in addition to Figure 2 the processor, memory, DMA controller, disk, and non-volatile memory shown, any data processing capable device where the device based on the embodiment is located can also include other hardware according to the actual functions of the data processing capable device, and no further description is given.

[0105] Accordingly, the present application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the incremental calculation and update method based on expert statistical features as described above. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of a wind turbine, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output. The above examples are illustrative and should not be understood as limiting the present invention. The scope of patent protection of the present invention includes but is not limited to the above specific example embodiments. Any changes, replacements and modifications of expert statistical feature configuration files, database tables and persistent storage methods that are consistent with the claims of the present invention and that any technician in this field can make within the scope of the present invention according to different business scenarios and example data shall fall within the scope of the patent protection of the present invention.

Claims

1. An incremental calculation and update method based on expert statistical features, characterized in that: The method specifically comprises the following steps: (1) Constructing an expert statistical feature configuration file, wherein the expert statistical feature configuration file includes several fields, wherein the column name of the first field is the name of the statistical expert feature, and the column names of the remaining fields include the statistical calculation category corresponding to the statistical expert feature, the grouping attribute name, the filtering condition associated attribute name, the filtering condition, the attribute name of the statistical operation performed, and the statistical operation type; (2) Read the expert statistical feature configuration file constructed in step (1), use the first field name as the primary key, verify the remaining fields according to the statistical calculation category corresponding to the statistical expert feature, and then splice the remaining fields that pass the verification into a linked list as the value to form a hash map; (3) Offline generation of associated statistical information of historical data: extract the expert statistical features to be generated according to the hash map obtained in step (2), obtain the calculation category of the corresponding expert statistical features according to the value obtained in step (2), and save the associated statistical information of the historical data corresponding to the statistical expert features according to the calculation category of the statistical expert features; (4) Persistence of the associated statistical information of the historical data saved in step (3); (5) Generate the associated statistical information of the newly added data online, extract the expert statistical features to be generated according to the hash map obtained in step (2), obtain the calculation category of the corresponding expert statistical features according to the value obtained in step (2), and save the statistical associated information of the newly added data corresponding to the statistical expert features according to the statistical calculation category corresponding to the statistical expert features; (6) Update the historical correlation statistical information. According to the correlation statistical information of the historical data saved in step (4) and the statistical correlation information of the newly added data generated in step (5), update the corresponding historical correlation statistical information, take the current newly added data time as the new historical data time node, obtain the updated historical correlation statistical information, and persist it as the new historical data statistical correlation information; (7) Using the historical data statistical correlation information updated in step (6), the numerical value of the statistical expert feature is inferred.

2. The incremental calculation and updating method based on expert statistical features according to claim 1 is characterized in that: The statistical calculation categories corresponding to the statistical expert features in step (1) include historical data statistics based on a certain feature, historical data statistics based on a certain feature within a specific valid time, statistics of the number of occurrences within a specific valid time, and numerical statistics based on recent periods of a certain feature.

3. The incremental calculation and updating method based on expert statistical features according to claim 1 is characterized in that: Several fields are separated by commas.

4. The incremental calculation and updating method based on expert statistical features according to claim 2 is characterized in that: The specific process of verifying the remaining fields according to the statistical calculation category corresponding to the statistical expert features in step (2) is as follows: The configuration validity of the remaining fields is verified and judged based on the following conditions: Condition 1: Whether the category configured in the second field of the expert statistical feature configuration file is within the configured category range, which is the configured category range, i.e., the historical data statistics of a feature, the historical data statistics of a feature within a specific valid time, the statistical value of the number of occurrences within a specific valid time, and the numerical statistics of a feature in recent periods; Condition 2: Determines whether the non-null value requirements of the remaining fields are met based on the configured categories; If any of the above conditions is not met, there is a problem with the expert statistical feature configuration file, and the fields that do not meet the category requirements are empty values.

5. The incremental calculation and updating method based on expert statistical features according to claim 2 is characterized in that: The step (3) of storing the associated statistical information of the historical data corresponding to the statistical expert features according to the calculation category of the statistical expert features, and the step (5) of storing the statistical associated information of the newly added data corresponding to the statistical expert features according to the statistical calculation category corresponding to the statistical expert features, are also determined by the following contents: Content 1: Determine the filter condition attribute to be executed based on the statistical filter condition associated attribute name in the value list obtained in step (2); Content 2: According to the attribute name that the statistical result in the value list obtained in step (2) depends on, determine the associated attribute of the statistical operation to be performed; The associated information needs to be stored in independent linked list structures according to the needs of content 1 and content 2.

6. The incremental calculation and updating method based on expert statistical features according to claim 1 is characterized in that: The step (4) specifically includes: using MySQL, ORACLE database, and / or using csv, xls, pkl file format to persistently store the associated statistical information of the historical data saved in step (3).

7. The incremental calculation and updating method based on expert statistical features according to claim 2 is characterized in that: The specific steps in step (6) are: (6.1) Incorporate statistical information related to historical data into the statistical information related to newly added data, including: Method 1: Based on the statistical correlation information of the new data, find the historical statistical correlation information of the new data that needs to be merged from the statistical correlation information of the historical data; Method 2: Based on the attribute name associated with the filter condition, find the data that needs to be updated in the statistical association information of the historical data, perform an update operation on the data that needs to be updated, and merge the updated information into the statistical association information of the newly added data; (6.2) Taking the current newly added data time as the new historical data time node, the updated historical correlation statistical information includes: Method 1: For the part that overlaps with historical data, the statistical correlation information of the historical data is replaced by the statistical correlation information of the newly added data; Method 2: For the parts that do not overlap with historical data information, the newly added data statistical related information is directly incorporated into the data.

8. The incremental calculation and updating method based on expert statistical features according to claim 1 is characterized in that: The step (7) is specifically as follows: using the historical data statistical association information updated in step (6), statistical operations including summing, finding the maximum value, finding the minimum value, finding the average value, counting, and extracting unique values ​​are performed on the linked list information of the updated historical data statistical association information, thereby obtaining the numerical value of the statistical expert feature.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the incremental calculation and update method based on expert statistical features as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the incremental calculation and updating method based on expert statistical features as described in any one of claims 1 to 8 is implemented.

Images