Insurance industry data scientific automation feature engineering method and device and electronic equipment

Abstract

The invention provides an insurance industry data scientific automation feature engineering method and device and electronic equipment, and relates to the field of feature engineering, and the method comprises the steps: obtaining an insurance sample table and a plurality of data tables comprising historical insurance data; determining an undirected graph taking the insurance sample table as a starting table and at least one data table as a branch table according to the primary key of the insurance sample table and the foreign key of each data table; determining a plurality of generation paths based on the undirected graph and the configured maximum path depth; and for any generation path, according to configured feature operators corresponding to different data types, calculating historical insurance data of different data types related to a data table corresponding to the generation path to obtain a plurality of target features. According to the method, a user does not need to know any insurance industry knowledge, and after the insurance sample table is organized and completed, the remaining processes are automatically completed by the method provided by the invention, that is, the feature matrix capable of directly entering a machine learning model is output.

Description

Technical Field

[0001] The present application relates to the field of feature engineering, and more specifically, to a method, device and electronic equipment for automated feature engineering in data science for the insurance industry. Background Art

[0002] When insurance companies construct insurance features, they need to have an in-depth understanding of the background of the modeling field, business paths, insurance data generation and storage processes before they can design features that represent the characteristics of insurance data. In this case, modelers need to master all of the above information at the same time or communicate frequently with industry experts, core system developers or operation and maintenance personnel, and data developers every time they construct insurance features, which leads to low efficiency and accuracy in determining insurance data features; insurance data often contains a large amount of categorical data (such as customer ID, product ID, behavior type, etc.) and text data, which generally cannot be directly used in traditional machine learning models; if directly used in traditional machine learning models, only using one-hot encoding to process categorical data will result in the generation of high-dimensional sparse vectors, which increases the computational complexity and cannot effectively express the similarity between categorical data; customer behavior data is usually time series data, such as consulting, insurance, and cancellation. There are complex temporal dependencies between behaviors. Traditional machine learning models often ignore these dependencies or rely on feature engineering in specific computing methods, and traditional sequence models (such as RNN and LSTM) are prone to losing important information when processing long sequence data, and have low computational efficiency. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a data science automated feature engineering method, device and electronic equipment for the insurance industry, which is used to solve the above-mentioned problems existing in the prior art, and can improve the efficiency of determining insurance data features while improving the accuracy of insurance data features.

[0004] In a first aspect, a data science automated feature engineering method for the insurance industry is provided, which may include: Obtaining an insurance sample table and multiple data tables including historical insurance data; According to the primary key of the insurance sample table and the foreign keys of each data table, determining an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table; Based on the undirected graph and the configured maximum path depth, a plurality of generated paths are determined; each generated path includes the insurance sample table and at least one data table; For any generation path, according to the feature operators corresponding to the configured different data types, the historical insurance data of different data types involved in the data table corresponding to the generation path are calculated to obtain multiple target features, and the multiple target features are connected to the insurance sample table.

[0005] In a possible implementation, before determining the undirected graph, the method further includes: Standardize the time series historical insurance data in each data table to obtain processed time series historical insurance data; Furthermore, data tables with the same primary key and the same foreign key are deduplicated to obtain multiple deduplicated data tables.

[0006] In a possible implementation, before determining an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table according to the primary key of the insurance sample table and the foreign keys of each data table, the method further includes: For any data table after deduplication, when the primary key of the data table is a foreign key of other data tables except the data table in multiple data tables, and the foreign key of the data table is the same as the target foreign key of other data tables, the target foreign keys in the corresponding other data tables are deleted to obtain a data table with the target foreign keys deleted, so as to determine an undirected graph with the insurance sample table as the starting table and at least one data table as a branch table based on the primary key of the insurance sample table and the foreign keys of each data table.

[0007] In a possible implementation, the feature operator includes: an aggregation operator and a conversion operator.

[0008] In a possible implementation, after determining a plurality of directed generation paths, the method further includes: For any generation path, according to the configured feature operators and time windows corresponding to different data types, the historical insurance data in the different data types and time windows involved in the data table corresponding to the generation path are calculated to obtain multiple target features.

[0009] In a possible implementation, the insurance sample table is composed of different types of ID-type data and time series-type calculation deadlines.

[0010] In a possible implementation, when an undirected graph with the insurance sample table as the starting table and at least one data table as the branch table is determined based on the primary key of the insurance sample table and the foreign keys of each data table, if the insurance sample table is not associated with at least one data table, a prompt message of ID-type data error in the insurance sample table is generated.

[0011] In a second aspect, a data science automated feature engineering device for the insurance industry is provided, which may include: An acquisition unit, used for acquiring an insurance sample table and a plurality of data tables including historical insurance data; A determination unit, configured to determine an undirected graph having the insurance sample table as a starting table and at least one data table as a branch table according to a primary key of the insurance sample table and foreign keys of each data table; and, based on the undirected graph and the configured maximum path depth, determining a plurality of directed generation paths; each generation path comprising the insurance sample table and at least one data table; A calculation unit, for calculating, for any generation path, the historical insurance data of different data types involved in the data table corresponding to the generation path according to the configured feature operators corresponding to different data types, to obtain multiple target features; An associating unit is used to associate multiple target features to the insurance sample table in sequence along the reverse direction of the generation path.

[0012] In a third aspect, an electronic device is provided, the electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0013] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0014] The present application provides an insurance industry data science automated feature engineering method, the method comprising: obtaining an insurance sample table and multiple data tables including historical insurance data; determining an undirected graph with the insurance sample table as the starting table and at least one data table as the branch table according to the primary key of the insurance sample table and the foreign keys of each data table; determining multiple generation paths based on the undirected graph and the maximum path depth of the configuration; for any generation path, according to the feature operators corresponding to the configured different data types, the historical insurance data of different data types involved in the data table corresponding to the generation path are calculated to obtain multiple target features, and the multiple target features are connected to the insurance sample table. The user of the present application does not need to understand any insurance industry knowledge, including but not limited to insurance products, business processes, customer behavior and sales tendencies, because after the organization completes the insurance sample table, the remaining process will be automatically completed by the method provided by the present application, that is, the output can directly enter the feature matrix of the machine learning model; the user does not need professional data knowledge, including but not limited to data source identification and data quality assessment, and only needs to construct the insurance sample table in a specific scenario; the user does not need to cooperate with personnel from multiple business systems or departments in the task of organizing the feature matrix; the constructed target feature effect is stable and does not change due to changes in the user or business scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A system architecture diagram for an insurance industry data science automated feature engineering method provided in an embodiment of the present application; Figure 2 A flowchart of an insurance industry data science automated feature engineering method provided in an embodiment of the present application; Figure 3 A schematic diagram of the log data generation stage provided in an embodiment of the present application; Figure 4 A schematic diagram of the data types provided in the embodiments of the present application; Figure 5 A schematic diagram of deleting redundant foreign keys provided in an embodiment of the present application; Figure 6 A schematic diagram of target features provided by an embodiment of the present application; Figure 7 A schematic diagram of a process for determining target features provided in an embodiment of the present application; Figure 8 A schematic diagram showing the relationship between the insurance sample table, customer index table, customer history track table, policy index table, and policy history track table provided in the embodiment of the present application; Fig. 9 A schematic diagram of a first generation path provided in an embodiment of the present application; Fig.10 A schematic diagram of a second generation path provided in an embodiment of the present application; Fig.11 A schematic diagram of the structure of an insurance industry data science automated feature engineering device provided in an embodiment of the present application; Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] The insurance industry data science automated feature engineering method provided in the embodiment of the present application can be applied to Figure 1 In the system architecture shown in Figure 1 As shown, the system may include: a server in the background of the insurance company and a terminal of the insurance company employee. The server may be a physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal may be a user equipment (UE) such as a mobile phone, smart phone, laptop, digital broadcast receiver, personal digital assistant (PDA), tablet computer (PAD), handheld device, vehicle-mounted device, wearable device, computing device or other processing device connected to a wireless modem, mobile station (MS), mobile terminal, etc. The terminal and the server may be directly or indirectly connected by wired or wireless communication, which is not limited in this application.

[0019] The terminal is used to obtain the insurance sample form and send the insurance sample form to the server.

[0020] The server is used to obtain the insurance sample table and multiple data tables in the database to execute an insurance industry data science automated feature engineering method provided by the present application.

[0021] When insurance companies construct insurance features, they need to have a deep understanding of the background of the modeling field, business paths, insurance data generation and storage processes before they can design features that represent the characteristics of insurance data. In this case, modelers need to master all of the above information at the same time or communicate frequently with industry experts, core system developers or operation and maintenance personnel, and data developers every time they construct insurance features.

[0022] Insurance data often contains a large amount of categorical data (such as customer ID, product ID, behavior type, etc.) and text data, which generally cannot be directly used in traditional machine learning models. If used directly in traditional machine learning models, only one-hot encoding is used to process categorical data, which will result in the generation of high-dimensional sparse vectors, which increases the computational complexity and cannot effectively express the similarities between categorical data. Customer behavior data is usually time series data. For example, there are complex temporal dependencies between consultation, insurance, and policy cancellation. Traditional machine learning models often ignore these dependencies or rely on feature engineering based on specific computing methods. Traditional sequence models (such as RNN and LSTM) are prone to losing important information when processing long sequence data and have low computational efficiency.

[0023] For example, when predicting whether a customer will cancel their insurance in the next year, it is possible to construct records of interactions between the customer and the insurance company, such as consultation, preservation, claims, complaints, etc.; policy usage, such as the use of health checks in the policy benefits; and even the activity of the policy agent marketing personnel, such as whether they are selling policies steadily and developing old customers. To construct these features in this case, machine learning modelers need to have the following expertise: 1. Insurance industry knowledge: 1. Product understanding: Understand different types of insurance products and their characteristics, such as life insurance, health insurance, car insurance, etc.

[0024] 2. Business process: Be familiar with all aspects of insurance business, including underwriting, preservation, claims, renewal and cancellation.

[0025] 3. Customer behavior: Understand the typical behavior patterns and preferences of customers at different stages of their life cycle.

[0026] 4. Sales tendency: the distribution of direct insurance companies’ sales channels (bank insurance, individual agents, telemarketing or online sales) and terminal marketing behaviors.

[0027] 2. Data knowledge: 1. Data source identification: Understand the logical form (log data or modifiable latest status data) and technical form of the required business data (whether it is stored as structured data, in a traditional database or in a big data warehouse).

[0028] 2. Data quality assessment: Evaluate the completeness, accuracy, and consistency of data. For example, for data that is claimed to be log data, how to confirm that it is complete log data.

[0029] 3. The following external conditions are also required: 1. Data acquisition: (1) Data source: The system can obtain data from various systems within the insurance company and link the data from each system. For example, the core business system records business information at the policy level (such as preservation, claims, etc.), and the app system records certain customer behavior data (such as monthly health information, physical examination appointments, etc.). For the same customer, different systems have different primary keys. How to link these information at the customer level?

[0030] (2) Data permissions: For data from different business systems, how to ensure that modelers have legal access rights and a complete authorization chain.

[0031] 2. Business cooperation: Cross-departmental collaboration: Work closely with business, IT and data departments to ensure accurate data acquisition and understanding.

[0032] In summary, the feature extraction of insurance data in the existing technology relies heavily on experienced modelers in various aspects; the decision on feature batch deployment is too subjective, rather than based on the model effect of a specific scenario; the features deployed in batches are more oriented towards traditional business analysis rather than machine learning modeling. Therefore, in the short engineering time of model construction, the time cost and personnel cost of the insurance feature determination process are increased.

[0033] In other words, to achieve the aforementioned insurance feature construction, the insurance industry currently uses two solutions: one is to rely on experienced modelers in various fields to construct features based on scenarios, which may have a certain degree of reusability; the other is to organize the data middle platform architecture and directly batch deploy insurance features with strong reusability. The actual process of constructing insurance features is more likely to be carried out in combination with the above two solutions.

[0034] Therefore, this application proposes an insurance industry data science automated feature engineering method to solve the above-mentioned problems existing in the prior art, which can improve the efficiency of determining insurance data features while improving the accuracy of insurance data features.

[0035] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0036] Figure 2 A flowchart of an insurance industry data science automated feature engineering method provided in an embodiment of the present application. Figure 2 As shown, the method may include: Step S210: Obtain an insurance sample table and multiple data tables including historical insurance data.

[0037] The insurance sample table may be composed of ID-type data and time series-type end-of-calculation time. Each data table is composed of different types of ID-type data and historical insurance data.

[0038] Before executing step S210, the method further includes: a historical insurance data organization process, which may specifically be: In order to realize the automated feature engineering process, it is necessary to pre-organize the historical trajectory data, that is, the historical insurance data in each data table. The historical insurance data in each data table is field-type log data. The database of a general insurance company does not directly have these historical insurance data, so it is necessary to go through the following process transformation and establish a log data update mechanism.

[0039] Combination Figure 3 As shown, a. different categories of insurance data and newly added incremental insurance data are introduced from multiple databases (business databases); different categories of insurance data and incremental insurance data are called historical insurance data.

[0040] b. Process different categories of historical insurance data to generate field-level log data, thereby forming data tables of different categories.

[0041] c. Assign the corresponding data type to the log data involved in the automated feature engineering in each data table. The data table may not have an index, but must have a datetime_index (a time series index, also called a datetime index). The significance of setting the data type is to calculate the corresponding log data (historical insurance data) according to the feature operators of different data types in the subsequent feature derivation process.

[0042] It should be noted that: (1) Log data that has not been assigned a data type will not participate in subsequent feature derivation (automatic feature engineering). For example, a garbled field has no reference significance for subsequent feature derivation, so the garbled field is not assigned a corresponding data type.

[0043] (2) Different preset data types are set in the database, and different log data types can be selected manually through the options. Figure 4 As shown, the options for different preset data types may include numeric type, time series type (date and time), ID type (Id identifier) ​​and custom type, etc. In other words, you can manually set an original data type, or automatically inherit the existing data type in the options.

[0044] Furthermore, if the existing data type is in the automatic inheritance option, it means that when the log data of this data type is calculated according to the characteristic operator, not only the operator that uses the log data of this data type as the input parameter is applicable, but also the operator whose parent class of the log data of this data type is used as the input parameter (in accordance with the inheritance rules of object-oriented programming).

[0045] The cost of organizing log data in this way is relatively low, and it can usually be transformed from the original database. The data type of log data can also usually be obtained by mapping the data type in the data dictionary and database products. Compared with the traditional data middle platform architecture, the cost is significantly reduced.

[0046] In some embodiments, the method further includes a historical insurance data cleaning stage, specifically including: In the process of organizing data tables, there may be a situation where multiple data tables have the same primary key. In this case, the target data table with the same primary key needs to generate a preservation serial number according to the configured serial number generation rule, and the preservation serial number is used as the current primary key of the target data table, and an index table of the primary key of each target data table is created. At this time, the index table is a bridge when the target data table is associated with the subsequent insurance sample table, or when the target data table is associated with other data tables.

[0047] This process can also be understood as follows: after being organized into log data, the ID of the corresponding category is no longer suitable as the primary key. For example, the primary keys of multiple data tables are all policy numbers. At this time, the primary key of the data table is changed to the preservation serial number (information change serial number). The policy number is no longer suitable as the primary key, which means that the policy number cannot reflect the uniqueness of each data table.

[0048] There is a necessity for the data table to use ID as the primary key: in the stage of feature engineering path generation, the table association logic is defined to associate the primary key and the foreign key (any non-primary key ID, which can be part of the joint primary key); generally speaking, the joint primary key of the insurance sample table is a certain important ID + the cutoff calculation time. Therefore, if there is no data table with the ID as the primary key, the insurance sample table will become an island and cannot be associated with any other data table.

[0049] Finally, the original primary key of the target data table is expanded into an index table, which only contains the original primary key. Afterwards, the index table is used as an intermediate bridge to associate the subsequent insurance sample table with the target data table, avoiding the insurance sample table from becoming an island.

[0050] In another embodiment, synonyms and synonyms of an ID are set: The function of synonyms is to record the synonymous IDs of all data tables but field combinations of different words.

[0051] For example, ID can be applicant ID, insured person ID, etc., which means that applicant ID and insured person ID can both represent customers, and synonyms of applicant ID and insured person ID can be set as customer ID.

[0052] The function of synonyms is to record the field combinations of the same word but different meanings of ID in all data tables.

[0053] For example, fee_ID and item_ID that exist in multiple data tables do not use a set of encoding rules, that is, the IDs corresponding to the same word are different.

[0054] This method can establish a more robust relationship between tables through synonyms, and avoid establishing invalid relationships between tables through antonyms, thereby avoiding invalid generation paths after determining the undirected graph.

[0055] In another embodiment, (1) for a data table without a primary key defined, if a row_ID needs to be added as the primary key, this means generating a unique identifier for each log record and setting it as the primary key. The purpose of this is to ensure that each record has a unique identifier to facilitate data query, update, and delete operations.

[0056] (2) Deleting historical insurance data that exists in the data table but has not been assigned a data type means that these historical insurance data will not participate in feature derivation.

[0057] (3) Deleting historical insurance data that has been set as a data type but does not exist in the data table means that the historical insurance data has been set incorrectly by humans.

[0058] (4) Data standardization: During the data exploration process, it was found that some categorical data similar to numerical data ended with ".0", which means that these categorical data were once treated as float types upstream and then forcibly converted to strings. Here, the ".0" is removed.

[0059] (5) Standardize time series data to avoid introducing historical insurance data of a time series type that cannot be recognized by the database. For example, some databases have valid time ranges.

[0060] (6) Each data table is deduplicated based on the determined primary key. Specifically, data tables with the same primary key are deduplicated.

[0061] (7) Delete historical insurance data with no information entropy from each data table. Historical insurance data with no information entropy is empty, none, or nan; there is only one unique value; the deletion process skips historical insurance data related to the primary key.

[0062] After the historical insurance data at the field level in each data table in the above database is set, step S210 may specifically include: The manually configured insurance sample table and multiple data tables including historical insurance data stored in the database are obtained through the terminal.

[0063] The insurance sample table organization stage may include: the insurance sample table may be a table including different types of IDs and cutoff calculation times; the data in the insurance sample table represents the samples to be used when establishing a machine learning model.

[0064] For example, different types of IDs can be customer numbers, policy numbers, agent numbers, etc. The specific ID depends on the content of the research object of this modeling. The data type of different types of IDs is ID type.

[0065] The calculation cutoff time is the observation cutoff time of the historical insurance data, which is also the prediction time node for subsequent modeling research. The time series data is used to constrain the historical insurance data introduced in this task to prevent time travel from occurring.

[0066] For example: Study the amount of auto insurance purchased by customers with customer numbers between 11...110 and 11...210 before December 31, 2023; December 31, 2023 is the calculation deadline and also the prediction time (a starting time node).

[0067] Step S220: Determine an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table according to the primary key of the insurance sample table and the foreign keys of each data table.

[0068] Specific, combined Figure 7 As shown, first, the data cleaning stage in the data table and the insurance sample table may include: Standardize the time series data in the obtained insurance sample table to obtain the processed time series data. This method can prevent the import of unrecognizable dates from affecting the calculation of time series data.

[0069] Also, deduplicate the data with the same primary key in the insurance sample table, that is, the data with the same ID type in the insurance sample table. This method can avoid unnecessary calculations in subsequent automated feature engineering. If there is a problem with the weight of the samples (ID type data) in the insurance sample table, there is no need to deduplicate the primary key in the sample table.

[0070] In some embodiments, in order to reduce calculation time, all data tables may be screened according to the configured screening rules and the synonyms set for the ID-type data in the insurance sample table to obtain a plurality of screened data tables.

[0071] Combination Figure 7 As shown, before preparing for path generation, it is necessary to delete redundant foreign keys of all data tables or multiple filtered data tables, which can include: It can be understood that all data tables are also multiple data tables after deduplication in step S210.

[0072] For any data table after deduplication, combine Figure 5 As shown, when the primary key of the data table is the foreign key of other data tables except the data table in multiple data tables (that is, the primary key of table A exists in table B, and the primary key of table A is also the foreign key of table B or a part of the joint primary key of table B), and the foreign key of the data table is the foreign key of other data tables (that is, the foreign key in table A is also a foreign key in table B and is only a foreign key and cannot be a part of the joint primary key), the corresponding foreign keys of other data tables are determined as target foreign keys, and the target foreign keys in other data tables are deleted to obtain the data table with the target foreign keys deleted.

[0073] It should be noted that the primary key in some data tables can be a composite primary key.

[0074] The foreign keys mentioned above do not actually exist as foreign keys in traditional databases, but mean that the historical insurance data corresponding to these foreign keys can be associated with certain tables. It can be considered that their function is only to associate data.

[0075] Due to normalization and denormalization, the original database may have added some key IDs to too many data tables during the design and continuous use and update process for query performance (fewer associated tables), and in the case of an imperfect data system, such as: the insurance type or liability level preservation, and the claim business table also has policy IDs. These operations are called denormalization operations here. These policy IDs can indeed facilitate query and improve query performance (the preservation and claim information under the insurance type or liability involved in the policy can be directly queried through the policy number, instead of first going to the insurance type main table to query the insurance type number corresponding to the policy), but these redundant foreign keys also bring too many redundant calculations to the subsequent generation path generation process, so these redundant foreign keys are removed before the generation path is generated through the above method, which is the reverse operation of the denormalization operation, the normalization operation.

[0076] The reasons why redundant foreign keys were not processed before obtaining the insurance sample table include: 1. The task level has a function to select data table ranges, which may cause some important data tables not to be selected. In this case, these redundant foreign keys are no longer redundant in the task. 2. The computational complexity of the normalization operation is very low, O(T⋅F⋅N), and the overhead of placing it at the task level is not high.

[0077] After basic processing of the data in the data table and the insurance sample table, the association relationship between the insurance sample table and the data table is established, including: Based on the primary keys and foreign keys (all ID-type data) of the insurance sample table and each data table, all possible relationships between tables are generated: for example, if the primary key of table A exists in table B as a possible foreign key, then it is considered that there is a possible relationship between table A and table B.

[0078] Afterwards, check the association relationship; specifically: when generating all possible association relationships between tables, confirm whether there is a possible association relationship between the insurance sample table and each data table. If there is not at least one data table associated with the insurance sample table, generate a prompt message of ID-type data error in the insurance sample table. When the ID-type data is a customer number, generate a prompt message of customer number error.

[0079] If it exists, determine the accuracy of at least one data table associated with the insurance sample table; for example: the joint primary key in the insurance sample table is customer number + cutoff calculation time, then the only table associated with the insurance sample table must be a data table with customer number as the primary key, and this data table is the index table with customer number as the primary key generated in the global data cleaning phase. The generation logic of this data table means that the table contains all the company's customers; if the customer number provided by the insurance sample table cannot be found in the index table, it indicates that there may be some errors in these customer numbers, and a prompt message including the customer number error is generated.

[0080] That is, an undirected graph with the insurance sample table as the starting table and at least one data table as the branch table is determined.

[0081] In this method, the insurance sample table is associated with multiple data tables to obtain an undirected graph. Since the undirected graph is compatible with the association of low-level tables to high-level tables (aggregation operation) and the association from high-level tables to low-level tables (migration operation), the undirected graph does not hinder the acquisition of information through aggregation operations or migration operations.

[0082] Step S230: Determine multiple generated paths based on the undirected graph and the configured maximum path depth.

[0083] Each generation path includes an insurance sample table and at least one data table.

[0084] Specifically, the search starts from the insurance sample table in the undirected graph, and stops when the search leads to the insurance sample table or the current path depth has reached the configured maximum path depth, so as to obtain multiple directed generation paths, wherein the directed generation paths can be understood as the existence of a connection order between the insurance sample table and the data table in each generation path, and between the data tables.

[0085] When the configured maximum path depth is 6, some examples of the obtained paths are as follows: The numbers represent the names of the insurance sample tables or data tables.

[0086] 1. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 10 (policy number); 2. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 21 (policy number); 3. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 26 (policy number); 4. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 28 (policy number); 5. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 31 (policy number); 6. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 6 (policy number); 7. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 13 (policyholder number) -> Table 13 (policy number); 8. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 1 (policyholder number) -> Table 17 (agent number) -> Table 17 (institution number); 9. Table 0 (customer number) -> Table 16 (customer number) -> Table 12 (customer number) -> Table 1 (policyholder number) -> Table 1 (agent number).

[0087] Step S240: For any generation path, according to the feature operators corresponding to the configured different data types, the historical insurance data of different data types involved in the data table corresponding to the generation path are calculated to obtain multiple target features, and the multiple target features are associated with the insurance sample table.

[0088] Specifically, for any generation path, the corresponding historical insurance data is calculated according to the configured feature operator to obtain multiple target features.

[0089] The configured feature operators include: Transformation, Aggregation, and custom operators.

[0090] Specifically, the user can choose which specific feature operator to use for a custom operator. The default is to use all of them. Further settings can also be made when choosing to use the corresponding feature operator. For example, a feature operator is only applicable to a field in a table. If no further settings are made, the feature operator will be applied to all historical insurance data corresponding to all suitable data types by default.

[0091] 1. Transformation: Transform or operate the data features of the same data type in a single table to generate new features, which may include: 1. Day: Get the date of the historical insurance data of time series type, ranging from 1 to 31. For example, the day of the insurance date of a certain insurance policy is 20.

[0092] 2. Month: Get the month of the time series historical insurance data, ranging from 1 to 12. For example, the month of the insurance date of a certain policy is 3.

[0093] 3. Weekday: Get the day of the week for time series historical insurance data, ranging from 1 to 7. For example, the weekday of the insurance date of a certain policy is 6.

[0094] 4. num_characters: Get the string length of string-type historical insurance data. For example, the num_characters of the policyholder address of a certain policy is 10.

[0095] 5. Vectorization: Convert string-type historical insurance data (such as customer feedback, claim instructions, and email content) into numerical (vectorized) representations. The specific approach is to introduce an open source pre-trained model (Bert) to extract meaningful features from historical insurance data, so that historical insurance data can be represented as a vector of fixed length (768 dimensions).

[0096] 6. Embedding+Transformer: Focuses on processing historical insurance data of categorical data in time series tables (such as the action type field in the customer behavior table). The specific approach is as follows: 6.1Embedding: First, the categorical data (such as action type) is mapped to a dense vector through Embedding. Specifically, each behavior type (such as "consultation", "insurance", "cancellation") will be converted into a dense vector. For example, if the dimension of Embedding is set to 64, the action type "insurance" can be represented as a 64-dimensional vector, such as 0.1, −0.2, 0.3, .... The behavior sequence after Embedding can be regarded as a sequence of behaviors, and each behavior becomes a dense vector, which is used as the input of Transformer.

[0097] 6.2 Transformer: Transformer is used to process these serialized dense vectors to capture the relationships and temporal dependencies between behaviors.

[0098] 6.2.1. Input: Assuming that each customer has T behavior records, and each record has been represented as a 64-dimensional vector through Embedding, the sequence size input to Transformer is T×64.

[0099] 6.2.2 Self-attention mechanism: Transformer uses the self-attention mechanism to calculate the correlation of each behavior at different time steps. For example, a customer’s “insurance” behavior and previous “consultation” behavior may have a strong correlation, and Transformer can capture this relationship through the self-attention mechanism.

[0100] 6.2.3. Output: After multiple layers of Transformer calculations, the output feature vector size is still T×64, which represents the new representation of each customer behavior after considering the impact of other behaviors.

[0101] 2. Aggregation: Summarize or count a set of historical insurance data in two levels of tables to generate statistical characteristics that describe the set of data.

[0102] 1.Sum: Calculate the comprehensive value of a set of historical insurance data. For example, the total premium paid by a customer is 15,000 yuan.

[0103] 2.Max: Find the maximum value in a set of historical insurance data. For example, the premium of a customer's largest insurance policy is 10,000 yuan; or calculate the maximum value of a multidimensional vector.

[0104] 3.Min: Find the minimum value in a set of historical insurance data. For example, the minimum premium for a certain customer's policy is 10 yuan; or calculate the minimum value of a multidimensional vector.

[0105] 4.Mean: Calculate the average value of a set of historical insurance data. For example, the average premium of all policies of a customer is 5,000 yuan; or calculate the mean of a multidimensional vector.

[0106] 5.Count: Count the number of elements in a set of historical insurance data. For example, a customer purchased a total of 5 insurance policies from the company.

[0107] 6.Num_unique: Calculates the number of unique values ​​in a set of historical insurance data. For example, a customer has purchased 5 insurance policies, and the weekday of the 5 insurance dates is 6, so the num_unique of the weekday of the insurance date is 1.

[0108] 7.Mode: Find the value with the highest frequency in a set of historical insurance data. For example, if a customer purchased 5 insurance policies, including 4 medical insurance policies and 1 critical illness insurance policy, then the mode of the policy type of all the policies purchased by the customer is medical insurance.

[0109] 3. Custom operators: Supports adding custom operators. The custom operator needs to specify the data types of input and output parameters. For example, if you add an aggregation operator to obtain the earliest transaction time of a customer, the input parameter quantity of the operator is 1, the data type is datetime, and the output parameter quantity is 1, the data type is also datetime.

[0110] Among them, if a custom operator is added, this operator will be applied to all time series historical insurance data when performing aggregation operations. Therefore, the addition of custom operators often leads to an exponential increase in the number of final features, and special caution is required unless it is only applied to a very narrow data type (for example, the premium is defined as the "premium" data type during data cleaning, and this data type inherits from the numeric type. In the task, in addition to applying operators with numeric types as input parameters, the premium field will also apply operators with the data type corresponding to "premium" as input parameters. This will not cause the number of target features to explode too much).

[0111] It should be noted that the corresponding feature operator is selected according to the data type of each historical insurance data in each generation path.

[0112] For example, the total transaction amount generated by each customer is obtained through sum in aggregation, and the output data type is clearly numeric.

[0113] The month of a transaction generated by a certain customer is obtained through the month (get month) in the transformation operator. The output data type is clearly a categorical type with order.

[0114] The customer information is redundantly added to the transaction table through the direct operator, and the output data type is clearly referenced to the original historical insurance data.

[0115] Recursively calculate the corresponding data according to the feature operator until all features are summarized (direct operator) and (aggregation operator) into the insurance sample table.

[0116] In some embodiments, for any generated path, a time window is added during the calculation process according to the configured feature operator.

[0117] It should be noted that the above steps in step 230 are only execution plans, not actual operations.

[0118] After obtaining the feature execution plan of each generation path, that is, which features are to be constructed, these execution plans are sampled according to the sampling ratio set by the user at the task level. For example, when the user sets the sampling ratio to 10%, it means that only 10% of the features of each path will be calculated. For example: suppose a generation path is expected to generate 100 features after simulation calculation by the feature operator, but the calculation has not actually started at this time, only the calculation is planned. At this time, 10% of the plan is sampled, that is, only the calculation plan of 10 target features is retained.

[0119] It should be noted that the number of samples is rounded up (ceil), which means that each generated path is guaranteed to have corresponding features retained, retaining a large amount of information content in the path dimension.

[0120] The sampling method defaults to random without replacement, but the sampling method settings are also open to users for customization.

[0121] Afterwards, the historical insurance data involving different data types (and time windows) involved in the data table corresponding to the generated path can be calculated according to the feature operators (and time windows) corresponding to the configured different data types to obtain multiple target features.

[0122] Specifically, time window calculation is not performed by default, which means that only the historical insurance data of each data table in each generation path within the entire historical time range will be calculated; in order to meet more needs, the corresponding time window can be configured to filter some historical insurance data corresponding to the time window: For example, if you add a 1_month time window, the target feature of each sample time_index one month before will be calculated in this time window.

[0123] Time windows of different lengths can be added according to different scenarios.

[0124] In this method, each time a time window is added, the features that need to be calculated will double.

[0125] The target features finally obtained can be: BD_T_CUSTOMER_LOG_CUSTOMER_ID_INDEX.MEAN(BD_T_CONTRACT_MASTER_LOG.BD_T_CONTRACT_MASTER_LOG_POLICY_ID_INDEX.MIN(BD_T_POLICY_PRODUCT.STAND_PREM)) represents the average of the lowest standard premiums among all policies for each customer.

[0126] BD_T_CUSTOMER_LOG_CUSTOMER_ID_INDEX.MAX(BD_T_CONTRACT_MASTER_LOG.BD_T_CONTRACT_MASTER_LOG_POLICY_ID_INDEX.NUM_UNIQUE(BD_T_POLICY_PRODUCT.MONTH(INSERT_TIME))) represents the maximum number of different inserted months in all policies of each customer.

[0127] BD_T_CUSTOMER_LOG_CUSTOMER_ID_INDEX.MAX(BD_T_CONTRACT_MASTER_LOG.BD_T_CONTRACT_MASTER_LOG_POLICY_ID_INDEX.MIN(BD_T_POLICY_PRODUCT.TAX_PREM)) represents the maximum value of the minimum after-tax premium among all policies for each customer.

[0128] BD_T_CUSTOMER_LOG_CUSTOMER_ID_INDEX.MEAN(BD_T_CONTRACT_MASTER_LOG.BD_T_CONTRACT_MASTER_LOG_POLICY_ID_INDEX.NUM_UNIQUE(BD_T_POLICY_PRODUCT.WEEKDAY(START_PAY_DATE))) represents the average number of days of the week with different payment start dates in all policies for each customer.

[0129] BD_T_CUSTOMER_LOG_CUSTOMER_ID_INDEX.NUM_UNIQUE(BD_T_CONTRACT_MASTER_LOG.DAY(END_DATE)) indicates the number of different end dates in all policies for each customer.

[0130] BD_T_CUSTOMER_LOG_CUSTOMER_ID_INDEX.MEAN(BD_T_CONTRACT_MASTER_LOG.BD_T_CONTRACT_MASTER_LOG_POLICY_ID_INDEX.NUM_UNIQUE(BD_T_POLICY_PRODUCT.DAY(WEAK_END))) represents the average number of different end dates in all policies of each customer.

[0131] In the process of determining multiple target features, all tables in the generation path are linked by inner joins based on the insurance sample table and the data table, as well as the relationship between the data tables, to achieve the purpose of filtering data. This results in different calculation results (target features) of the two generation paths even if they have the same features at a relatively shallow level. This is because although the calculation logic is exactly the same, the data ranges applied are different (for example, both generation paths calculate how many insurance policies a customer has, and the calculation logic is simply counted from the policy layer to the customer layer, but because of the different paths, the policy layer may only have data on policies that have been claimed and data on policies that have been manually underwritten).

[0132] Combination Figure 6 As shown, the multiple target features obtained are the corresponding feature matrices. At this point, each path will obtain a feature matrix, that is, different target features can be obtained through different operators; after that, all target features are associated with the insurance sample table to obtain a complete multiple target features to enter the subsequent model, which is the model for subsequent processing of the target features.

[0133] It should be noted that, for any generation path, when calculating the historical insurance data of the corresponding data type according to the characteristic operator, the historical insurance data to be calculated may not exist in the data table at the bottom of the generation path. The data table at the bottom is only a restriction condition to filter out some historical insurance data. At this time, the insurance sample table and the data table in the generation path, as well as the connection order between the data tables, should be followed in the opposite direction to search for some historical insurance data of the data type corresponding to the characteristic operator in turn, so as to calculate the filtered part of the historical insurance data according to the characteristic operator. For example, the generation path is Table 0 (customer number + deadline calculation time) -> Table 15 (customer number) -> Table 16 (insured person + policy ID + policy amount) -> Table 12 (surrender policy ID); when the feature operator is to count the total surrender amount within the corresponding deadline calculation time, Table 12 can only filter out the surrender policy ID, but the table does not store the policy amount, so the total surrender amount cannot be calculated; at this time, in the opposite direction of the generation path, the policy amount corresponding to the surrender policy ID corresponding to the feature operator is filtered out in Table 16, so as to calculate the total surrender amount. In short, the feature operator is used to calculate the corresponding part of the historical insurance data filtered out in the opposite direction of the generation path to obtain the target feature.

[0134] An embodiment of the overall steps of the present application may be: A total of 30 original business tables (data tables) are introduced from the core system of the insurance company, including contract_log (policy basic information change record table: records the change track of all policies that have changed), contract_master (policy master table, records the current status of all policies), customer_log (customer information change record table, records the change track of all customers that have changed), and customer (customer information table, records the current status of all customers). Insurance companies have complex businesses, usually with many basic business tables, and the range of tables used in machine learning modeling is usually large, which cannot be listed one by one. The following will also focus on the processing steps for data tables related to policies and customers.

[0135] The log data of the manually determined insurance policy is collated from contract_log and contract_master. The specific logic is that the historical track of the insurance policy that has changed will come from contract_log, and the information of the insurance policy that has not changed will come from the contract_master table; the log data of the customer is collated from customer_log and customer tables. The specific logic is that the historical track of the customer that has changed will come from customer_log, and the information of the customer that has not changed will come from the customer table. Then manually determine which data the information consists of, and whether these data need additional processing when they come from different data tables to ensure the consistency of the data format. The above data organization method includes insurance data and newly added incremental insurance data. The frequency of increments depends on the modeling scenarios that the project will support in the future. For example, if the modeling scenario is to predict the fraud risk of the insurance policy that took effect on the previous day every day, the incremental frequency needs to be at least daily batches.

[0136] Set the data type of each historical insurance data in the policy log table and customer log table that will participate in feature derivation. Due to too much data, only part of the data in the policy log table is described, as shown in Table 1 below: Table 1

[0137] It can be seen that the historical insurance data in Table 1 all use the preset data type, where CHANGE_ID and POLICY_ID constitute a joint primary key, which means that this table will save a piece of data every time a preservation change occurs in the policy.

[0138] After the organization of the policy log table and the customer log table is completed, because the historical track needs to be reflected, the policy number and the customer number are no longer used as the primary key. Therefore, an index table with the policy number and the customer number as the primary key is generated based on these two tables, and the incremental logic is preset to ensure the connection between other data tables and the insurance sample table.

[0139] Create a synonym dictionary: [customer_ID, applicant_ID, insured_ID]; Create a synonym dictionary: {Item_ID:[insurance type table, questionnaire, physical examination form]}; Note: item_ID in the insurance type table represents a type of insurance in the policy; item_ID in the survey table represents an event in the survey scenario; item_ID in the physical examination table represents a physical examination item in the physical examination scenario.

[0140] After setting up the data table as before step S210, the user selects the sample granularity to organize the insurance sample table according to the modeling scenario: for example, in the repurchase scenario of an insurance company, the requirement is to predict in advance at the beginning of each month the customers who have the intention to repurchase or add to the purchase in the next month for precision marketing. At this time, the sample granularity is the customer number (ID-type data). Each customer number and the timestamp of the beginning of each month since the customer contacted the company form a sample. The timestamp represents the cutoff calculation time of the sample (to control time travel), and the actual repurchase situation of the customer in each month in history is the label (whether it is a repurchase).

[0141] The time series data in the data table and the insurance sample table are standardized as in step S220, and the data with the same ID type in the insurance sample table is deduplicated.

[0142] Afterwards, in the redundant foreign key deletion phase, it is found that the insurance type table redundantly stores the policyholder ID data, so the policyholder ID is deleted. In addition, some redundant data is deleted after being discovered, which will not be described here.

[0143] The relationship between the insurance sample table, customer index table, customer history track table, policy index table, and policy history track table is as follows: Figure 8 shown.

[0144] It can be seen that the customer index table and the policy history track table are connected through the customer number (customer_id) and the insured number (applicant_id) because of the setting of the synonym dictionary.

[0145] At this time, according to Figure 8 The relationship between tables shows that the insurance sample table is normally connected to other data tables, confirming that there is a possible association between the insurance sample table and the data table, indicating that the machine learning modeling based on the insurance sample table can proceed normally.

[0146] The insurance sample table for this task consists of customer numbers and calculation deadlines, so the correctness of these customer numbers is checked here. Figure 8 It can be seen that the insurance sample table is only connected to the customer index table. The verification is implemented through the index table. Specifically, it is to check whether there is a customer number in the insurance sample table but not in the customer index table (theoretically including all the customer numbers of the company). If such a customer number does not exist, a prompt message will be thrown to the customer. Users can check whether there is a problem in the construction of their insurance sample table based on this prompt message.

[0147] Assuming that the maximum path depth is configured to be 4 (in actual work tasks, a deeper maximum path depth is often used), according to Figure 8 The relationship between tables in Fig. 9 and Fig.10As shown, the first generation path and the second generation path are generated (there are also many generation paths for actual work tasks, and the relationship between the tables used as an example here is relatively simple, with only 2 generation paths).

[0148] When inner joins are performed based on the relationship between tables. In the first generation path, because no special business scenarios are involved, no historical insurance data is actually screened and eliminated; in the second generation path, at the customer level, customers who have been policyholders in the past will be screened, which means that customers who have been other entities (such as insured persons or beneficiaries) in the past will be eliminated, and at the policy level, only policies with these customers as policyholders will be screened.

[0149] For any generation path, a target feature, that is, a feature matrix, can be generated according to each feature operator, and then the target features corresponding to all feature operators are sequentially associated with the insurance sample table in the reverse direction of the generation path to obtain a complete feature matrix that can enter the model.

[0150] The present application provides a data science automated feature engineering method for the insurance industry, the method comprising: obtaining an insurance sample table and multiple data tables including historical insurance data; determining an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table according to the primary key of the insurance sample table and the foreign keys of each data table; determining multiple generation paths based on the undirected graph and the configured maximum path depth; for any generation path, calculating historical insurance data of different data types involved in the data table corresponding to the generation path according to feature operators corresponding to different configured data types, obtaining multiple target features, and linking the multiple target features to the insurance sample table.

[0151] This application can improve scalability, flexibility, and ease of use while improving feature engineering efficiency, increasing the scale of processed data, improving feature quality and model performance, and reducing human bias. The specific reasons are as follows: Manual feature engineering is very time-consuming and complex, requiring data scientists to have deep domain knowledge and data processing capabilities. This application can greatly shorten the time from data preparation to model training and improve development efficiency.

[0152] Modern data sets are often large and diverse, and manually processing these data and extracting meaningful features may not be completed in a timely manner. Through this application, useful features can be better identified and extracted when processing structured, unstructured, and semi-structured data.

[0153] Automated feature engineering can mine potential high-quality features from data that may be difficult to discover with manual methods. This application uses automated feature operators to systematically select the most predictive features, reduce noise, and improve the accuracy and stability of the model.

[0154] The automated engineering of this application reduces subjective bias in the feature selection and processing process, making the results more objective and reliable, ensuring that each step of the feature engineering is standardized and repeatable, and reducing errors caused by manual operations.

[0155] The automation engineering of this application can be easily expanded to handle the growing volume and complexity of data. It can be adapted to different data sets and problem domains to provide a universal solution.

[0156] This application enables non-data or domain experts to perform effective feature engineering, promoting the popularization and application of data science.

[0157] Corresponding to the above method, the embodiment of the present application also provides an insurance industry data science automated feature engineering device, such as Fig.11 As shown, the device comprises: An acquisition unit 1110, configured to acquire an insurance sample table and a plurality of data tables including historical insurance data; The determining unit 1120 is used to determine the insurance sample table as the starting point according to the primary key of the insurance sample table and the foreign keys of each data table. An undirected graph in which the initial table and at least one data table are branch tables; and, based on the undirected graph and the configured maximum path depth, determining a plurality of directed generation paths; each generation path comprising the insurance sample table and at least one data table; A calculation unit 1130 is used to calculate, for any generation path, historical insurance data of different data types involved in a data table corresponding to the generation path according to the configured feature operators corresponding to different data types, to obtain multiple target features; The associating unit 1140 is used to sequentially associate the multiple target features with the insurance sample table in the reverse direction of the generation path.

[0158] The functions of the various functional units of the data science automated feature engineering device for the insurance industry provided in the above-mentioned embodiment of the present application can be achieved through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in the data science automated feature engineering device for the insurance industry provided in the embodiment of the present application will not be repeated here.

[0159] The present application also provides an electronic device, such as Fig.12 As shown, it includes a processor 1210 , a communication interface 1220 , a memory 1230 and a communication bus 1240 , wherein the processor 1210 , the communication interface 1220 , and the memory 1230 communicate with each other via the communication bus 1240 .

[0160] Memory 1230, used for storing computer programs; The processor 1210 is used to implement the following steps when executing the program stored in the memory 1230: Obtaining an insurance sample table and multiple data tables including historical insurance data; According to the primary key of the insurance sample table and the foreign keys of each data table, determining an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table; Based on the undirected graph and the configured maximum path depth, a plurality of directed generation paths are determined; each generation path includes the insurance sample table and at least one data table; For any generation path, according to the feature operators corresponding to the configured different data types, the historical insurance data of different data types involved in the data table corresponding to the generation path are calculated to obtain multiple target features, and the multiple target features are sequentially associated with the insurance sample table in the reverse direction of the generation path.

[0161] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0162] The communication interface is used for communication between the above electronic device and other devices.

[0163] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0164] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0165] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 2 The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0166] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes an insurance industry data science automated feature engineering method described in any of the above embodiments.

[0167] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute an insurance industry data science automated feature engineering method as described in any one of the above embodiments.

[0168] Those skilled in the art will appreciate that the embodiments in the present application can be provided as methods, systems, or computer program products. Therefore, the embodiments in the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments in the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0169] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0172] Unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which the present invention belongs. "First", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect", "couple" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0173] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the present application embodiments are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application embodiments.

[0174] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application are within the scope of the embodiments in the present application and their equivalents, the embodiments in the present application are also intended to include these modifications and variations.

Claims

1. A data science automated feature engineering method for the insurance industry, characterized in that: The method comprises: Obtaining an insurance sample table and multiple data tables including historical insurance data; According to the primary key of the insurance sample table and the foreign keys of each data table, determining an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table; Based on the undirected graph and the configured maximum path depth, a plurality of generated paths are determined; each generated path includes the insurance sample table and at least one data table; For any generation path, according to the feature operators corresponding to the configured different data types, the historical insurance data of different data types involved in the data table corresponding to the generation path are calculated to obtain multiple target features, and the multiple target features are connected to the insurance sample table.

2. The method according to claim 1, characterized in that Before determining the undirected graph, the method further includes: Standardize the time series historical insurance data in each data table to obtain processed time series historical insurance data; Furthermore, data tables with the same primary key and the same foreign key are deduplicated to obtain multiple deduplicated data tables.

3. The method according to claim 2, characterized in that Before determining an undirected graph with the insurance sample table as a starting table and at least one data table as a branch table according to the primary key of the insurance sample table and the foreign keys of each data table, the method further comprises: For any data table after deduplication, when the primary key of the data table is a foreign key of other data tables except the data table in multiple data tables, and the foreign key of the data table is the same as the target foreign key of other data tables, the target foreign keys in the corresponding other data tables are deleted to obtain a data table with the target foreign keys deleted, so as to determine an undirected graph with the insurance sample table as the starting table and at least one data table as a branch table based on the primary key of the insurance sample table and the foreign keys of each data table.

4. The method according to claim 1, characterized in that The feature operators include: aggregation operators and conversion operators.

5. The method according to claim 4, characterized in that After determining a plurality of directed generation paths, the method further includes: For any generation path, according to the configured feature operators and time windows corresponding to different data types, the historical insurance data in the different data types and time windows involved in the data table corresponding to the generation path are calculated to obtain multiple target features.

6. The method according to claim 1, characterized in that The insurance sample table is composed of ID-type data of different categories and time series-type calculation deadlines.

7. The method according to claim 1, characterized in that When an undirected graph with the insurance sample table as the starting table and at least one data table as the branch table is determined based on the primary key of the insurance sample table and the foreign keys of each data table, if the insurance sample table is not associated with at least one data table, a prompt message of ID-type data error in the insurance sample table is generated.

8. An insurance industry data science automated feature engineering device, characterized in that: The device comprises: An acquisition unit, used for acquiring an insurance sample table and a plurality of data tables including historical insurance data; A determination unit, configured to determine an undirected graph having the insurance sample table as a starting table and at least one data table as a branch table according to a primary key of the insurance sample table and foreign keys of each data table; and, based on the undirected graph and the configured maximum path depth, determining a plurality of directed generation paths; each generation path comprising the insurance sample table and at least one data table; A calculation unit, for calculating, for any generation path, historical insurance data of different data types involved in a data table corresponding to the generation path according to feature operators corresponding to different data types configured, to obtain multiple target features; An associating unit is used to associate multiple target features to the insurance sample table in sequence along the reverse direction of the generation path.

9. An electronic device, characterized in that: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

Images