Policy risk analysis methods, systems, and storage media based on association rule mining
By employing feature extraction, discretization coding, and dynamic support threshold filtering, the problem of misjudgment in policy risk analysis caused by differences in incidence rates across different data sources was solved, thereby improving the reliability and accuracy of policy risk analysis.
Patent Information
- Application Number
- CN202611208604.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-11
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-08-11
AI Technical Summary
Existing technologies in policy risk analysis suffer from inconsistent cross-source itemset comparison benchmarks due to differences in the incidence rates of data sources. This leads to sparse risk factor combinations being structurally filtered or misjudged as strongly correlated rules, reducing the credibility of policy risk analysis.
By extracting features and performing interval discretization coding on multi-source policy data, customer behavior data, and historical claims data, and combining marginal incidence rate dynamic minimum support threshold screening and contraction correction processing, a corrected correlation score of a strongly correlated candidate set is generated for determining the policy risk level.
This improves the reliability of identifying sparse but real strong risk correlation patterns in multi-source heterogeneous data, ensuring the accuracy and credibility of policy risk analysis results.
Smart Images

Figure CN122736788B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of policy risk control technology, and in particular to a policy risk analysis method, system and storage medium based on association rule mining. Background Technology
[0002] Association rule mining, an important branch of data mining, centers on treating each data record as a transaction and its various attribute values as items. By statistically analyzing the frequency of items and itemsets across all transactions, strong association rules are identified, where the presence of a certain set of preceding items often coincides with the presence of a certain following item. Support, confidence, and lift are used as quantitative indicators to measure the strength of these rules. When applying this approach to policy risk analysis, current techniques typically treat each policy as a transaction, using features extracted from policy elements, customer behavior data, and historical claims data as items. After feature extraction and discretization of multi-source policy data, customer behavior data, and historical claims data, an Apriori-like algorithm is used to generate candidate itemsets layer by layer and calculate their support. Frequent itemsets with support not lower than a preset threshold are retained. Candidate rules are then generated from these frequent itemsets and further filtered using confidence and lift. Finally, the selected strong association rules are used to label risk factor combinations in policies, serving as a basis for determining policy risk levels or issuing abnormal claims warnings.
[0003] Existing technologies have significant limitations in setting support thresholds. If a uniform minimum support threshold is applied to all candidate item sets, risk factor combinations derived from structurally sparse data sources such as claims data will be systematically lowered due to the significantly lower marginal occurrence rate of claims data compared to policy and behavioral data sources. Even if these combinations do reflect genuine strong risk associations, their original co-occurrence frequency will be systematically reduced because of the low occurrence rate of the data sources themselves. This makes it difficult to achieve the same support level as itemsets from the same source as policy and behavioral data sources, thus leading to direct filtering by the uniform threshold. While existing technologies have improved by setting differentiated support thresholds based on the importance of the itemsets themselves or their historical frequency, these differentiated settings target the importance of the items themselves and do not adjust the support calculation benchmark based on the structural factor of differences in the marginal occurrence rate of the data sources to which the items belong. This fails to fundamentally eliminate the problem of inconsistent comparison benchmarks across cross-source itemsets caused by different occurrence rates of data sources.
[0004] After identifying the issue of cross-source itemsets being structurally masked by differences in data source occurrence rates, resolving this problem while preserving the ability to identify sparse cross-source risk itemsets requires modifying the support calculation method itself. This would ensure that the support evaluation benchmark for cross-source itemsets matches the marginal occurrence rate of their respective data sources, rather than relying on a uniform or fixed threshold based on item importance. However, this normalization method, based on the marginal occurrence rate of the data source, can lead to cross-source itemsets with extremely low co-occurrence rates receiving inflated relative correlation evaluations simply because the normalization denominator is small. Such correlation evaluations are actually statistically random phenomena caused by insufficient sample size and lack true statistical reliability. If the normalized relative correlation evaluation results are not corrected according to their corresponding absolute co-occurrence frequencies, a large number of pseudo-strong correlation combinations lacking statistical reliability will be misjudged as strong correlation rules and included in subsequent policy risk analysis results, reducing the credibility of the policy risk analysis results. Summary of the Invention
[0005] This application provides a policy risk analysis method, system, and storage medium based on association rule mining. It solves the problems of cross-source risk factor combinations being structurally filtered due to differences in marginal occurrence rates of data sources, and small sample combinations being misjudged as strong association rules after normalization. It improves the reliability of the policy risk analysis method in identifying sparse but real strong risk association patterns in multi-source heterogeneous data.
[0006] Firstly, this application provides a policy risk analysis method based on association rule mining, the policy risk analysis method based on association rule mining including: Step S1: Perform feature extraction processing on multi-source policy data, customer behavior data, and historical claims data to obtain the original feature set with source labels; Step S2: Perform interval discretization encoding on the original feature set to obtain a transaction itemset database; Step S3: Based on the marginal occurrence rate of each code in the transaction item set database, the candidate item set is subjected to dynamic minimum support threshold screening to obtain the candidate item set that passes the initial screening. Then, based on the actual co-occurrence frequency of the candidate item set, the conditional support ratio of the candidate item set is subjected to shrinkage correction to obtain the corrected correlation score of the strongly correlated candidate item set. Step S4: Generate a policy risk association rule set from the strongly correlated candidate set, and perform risk level determination processing on the policy to be analyzed according to the policy risk association rule set to obtain the policy risk analysis result.
[0007] Secondly, this application provides a policy risk analysis system based on association rule mining, the policy risk analysis system based on association rule mining includes: The extraction module is used to perform feature extraction processing on multi-source policy data, customer behavior data and historical claims data to obtain the original feature set with source labels; The encoding module is used to perform interval discretization encoding on the original feature set to obtain a transaction item set database; The correction module is used to perform dynamic minimum support threshold filtering on candidate itemsets based on the marginal occurrence rate of each code in the transaction itemset database to obtain candidate itemsets that pass the initial screening. Then, based on the actual co-occurrence frequency of the candidate itemsets, the conditional support ratio of the candidate itemsets is subjected to shrinkage correction to obtain the corrected correlation score of the strongly correlated candidate itemsets. The determination module is used to generate a policy risk association rule set from the strongly correlated candidate set, and to perform risk level determination processing on the policy to be analyzed according to the policy risk association rule set to obtain the policy risk analysis result.
[0008] Thirdly, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, which, when run on a computer, cause the computer to execute the above-described policy risk analysis method based on association rule mining.
[0009] The technical solution provided in this application extracts features from multi-source policy data, customer behavior data, and historical claims data to obtain an original feature set with source labels. Then, the feature set is subjected to interval discretization encoding to obtain a transaction itemset database. These two technical features enable the three types of heterogeneous data—policy data, behavior data, and claims data—which have significant differences in business nature, collection frequency, and numerical distribution, to be uniformly converted into item codes with the same structural form while retaining their respective source identifiers. This provides a data foundation for the subsequent unified processing of risk factor combinations across data sources within the same set of association rule mining framework. The value of this approach lies in the inherent multi-source heterogeneity of the policy risk analysis scenario. The policy elements determined at the time of insurance purchase, the behavioral data generated after the purchase, and the claims data generated after an incident are all different in terms of data generation logic. If all features are processed indiscriminately without retaining source labels, subsequent steps will be unable to distinguish whether the original occurrence rate of a certain code is due to the business attributes of that item itself or to the collection characteristics of its data source. This is precisely the prerequisite for the association rule mining algorithm to generate effective discrimination ability in the application scenario of insurance risk analysis. It shows that the two technical features of feature extraction and discretization coding are not simply the application of general data preprocessing steps, but a processing method specifically designed for the data characteristics of the policy risk analysis scenario.
[0010] The candidate item set is dynamically screened based on the marginal occurrence rate of each code in the transaction item set database, and then the conditional support ratio is shrunk and corrected based on the actual co-occurrence frequency of the candidate item set to obtain the corrected correlation score of the strongly correlated candidate item set. These two technical features are the core contributions of this application at the algorithm level. The dynamic minimum support threshold is calculated based on the marginal occurrence rate of each code contained in the candidate item set. This allows risk factor combinations from naturally low-occurrence data sources such as claims data to be compared with high-occurrence risk factor combinations from policy data and behavioral data using an evaluation benchmark that matches the characteristics of their respective data sources, rather than applying a uniform fixed value. This algorithm feature directly corresponds to the business reality that the occurrence rate of claims data sources is structurally low in the policy risk analysis scenario. If this algorithm feature is applied to application areas where there is no difference in occurrence rates among multiple sources, its technical significance will not be apparent, indicating that the effectiveness of this algorithm feature is closely integrated with the data characteristics of the specific application scenario of policy risk analysis. The shrinkage correction process introduced on this basis further adjusts the normalized conditional support ratio based on the absolute co-occurrence frequency of the candidate options set, ensuring that the evaluation results are not distorted due to insufficient sample size, and overcoming the limitation of simply relying on normalization processing, which can easily introduce statistically unreliable combinations. A set of policy risk association rules is generated from the strongly correlated candidate options set, and the risk level of the policies to be analyzed is determined accordingly. This transforms the rules obtained from the aforementioned algorithm into risk level determination results that can be directly used in specific business processes such as policy underwriting and claims review. Attached Figure Description
[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 This is a schematic diagram of an embodiment of the policy risk analysis method based on association rule mining in this application. Figure 2 This is a schematic diagram comparing the numerical distribution before and after shrinkage correction in an embodiment of this application; Figure 3 This is a schematic diagram comparing the number of strongly correlated candidate options identified under different support filtering methods in the embodiments of this application. Detailed Implementation
[0013] This application provides a method, system, and storage medium for policy risk analysis based on association rule mining. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0014] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the policy risk analysis method based on association rule mining in this application includes: Step S1: Perform feature extraction processing on multi-source policy data, customer behavior data, and historical claims data to obtain the original feature set with source labels; Specifically, feature extraction processing refers to extracting specific fields related to policy risk assessment from the three types of original business records. Each extracted field value, along with its data source, field number, and the policy to which it belongs, is organized into a record called the original feature quadruple. The source label in the quadruple indicates whether the field originates from policy data, behavioral data, or claims data; the feature number identifies the field within its category; and the feature value is the specific numerical value or category of the field on that policy. Since not every policy has claims records, for policies without claims records, the values of claims-related fields are filled in using a pre-agreed default method, ensuring that each policy has a complete record location across the three data sources without structural omissions. The quadruples corresponding to all policies are merged according to their respective policies to obtain the original feature set with source labels. This set is the object of subsequent discretization coding.
[0015] Step S2: Perform interval discretization encoding on the original feature set to obtain the transaction itemset database; Specifically, discretization coding transforms continuously varying numerical fields and categorical fields with limited value ranges into item codes that facilitate counting and comparison. Numerical fields are sorted by value size and divided into several intervals. This interval division method ensures that the number of policies falling within each interval is roughly equal, preventing statistical instability due to insufficient samples in individual intervals. Categorical fields, with their limited value ranges, directly use their original codes as interval numbers. Source tags, feature numbers, and interval numbers are concatenated in a fixed order to form item codes. All item codes corresponding to the same policy are organized into a transaction, and the set of transactions corresponding to all policies constitutes a transaction item set database. Based on this database, the frequency of each item code appearing in all transactions is counted and divided by the total number of transactions to obtain the marginal occurrence rate of that item code. The marginal occurrence rate reflects the basic proportion of a specific risk factor value appearing in all policies and serves as a reference benchmark for subsequently judging whether a combination has abnormal correlation properties.
[0016] Step S3: Based on the marginal occurrence rate of each code in the transaction itemset database, the candidate itemsets are subjected to dynamic minimum support threshold screening to obtain the candidate itemsets that pass the initial screening. Then, based on the actual co-occurrence frequency of the candidate itemsets, the conditional support ratio of the candidate itemsets is subjected to shrinkage correction to obtain the corrected correlation score of the strongly correlated candidate itemsets. Specifically, the candidate set is a combination of multiple item codes. Its expected co-occurrence frequency is calculated by multiplying the marginal occurrence rates of each item code in the combination by the total number of transactions. This represents the number of times these item codes should theoretically occur under the assumption of no correlation. The dynamic minimum support threshold is obtained by multiplying the expected co-occurrence frequency by a global baseline coefficient less than one. This coefficient is less than one to allow for a certain margin of error based on the expected frequency, avoiding misjudging combinations that just reach the expected level but fluctuate slightly as not meeting the condition. After comparing the actual co-occurrence frequency of the candidate set with this threshold, combinations whose actual frequency reaches or exceeds the threshold are retained, while those that do not are eliminated. For the retained combinations, the conditional support ratio is obtained by dividing the actual co-occurrence frequency by the expected co-occurrence frequency. A ratio greater than one indicates that the actual co-occurrence of these item codes is more frequent than that calculated under the independent assumption. Since the actual co-occurrence frequency of some combinations is very small, directly using the conditional support ratio is easily distorted by random factors. Therefore, a shrinkage weight is introduced. The shrinkage weight is obtained by dividing the actual co-occurrence frequency by the sum of the actual co-occurrence frequency and a preset shrinkage smoothing constant. The larger the actual co-occurrence frequency, the closer the shrinkage weight is to one, and the conditional support ratio itself dominates. The smaller the actual co-occurrence frequency, the closer the shrinkage weight is to zero, and the final score is closer to the benchmark value of one, which represents the unrelated state. The result obtained by weighted summation of the shrinkage weight and the conditional support ratio is the corrected correlation score. Combinations exceeding the preset threshold are identified as strongly correlated candidate combination sets.
[0017] Step S4: Generate a policy risk association rule set from the strong association candidate set, and perform risk level determination on the policy to be analyzed based on the policy risk association rule set to obtain the policy risk analysis results.
[0018] Specifically, when the strong association candidate set contains item codes representing claims results, these item codes are used as the consequent of the rule, and the remaining item codes in the combination are used as the antecedents of the rule. The confidence level is calculated by the ratio of the frequency of the antecedent's individual occurrences to the actual co-occurrence frequency of the combination. The confidence level reflects the proportion of the consequent occurring in policies where the antecedent occurs. The lift level is calculated by the ratio of the confidence level to the consequent's actual occurrence rate. The lift level is used to determine the extent to which the occurrence of the antecedent changes the baseline occurrence rate of the consequent. A ratio greater than one indicates that the antecedent has a positive effect on the occurrence of the consequent. Rules that meet the confidence and lift level screening conditions constitute the policy risk association rule set. For the policy to be analyzed, after converting the corresponding item code combination using the same feature extraction and discretization method, it is compared with the antecedents of each rule in the rule set. The corrected association score, confidence level, and lift level of the matched rule are used in subsequent comprehensive calculations to finally obtain the risk assessment result for the policy.
[0019] In one specific embodiment, step S1 includes: Data is collected and processed from policy data sources, customer behavior data sources, and historical claims data sources to obtain raw policy data, raw behavior data, and raw claims data. Feature extraction processing is performed on the original policy data, behavioral data, and claims data to obtain the original feature quadruple. The original feature quadruple includes the policy identifier, source tag, feature number, and feature value. Based on the original claims data, the feature values of the number of claims and the feature values of the cause of claims in the original feature quadruple corresponding to policies for which there are no corresponding claims records are completed to obtain the completed feature quadruple; The feature quadruples are merged based on the policy identifier to obtain the original feature set with source labels.
[0020] Specifically, the fields collected from the policy data source include insured age, occupation category, sum insured, premium payment method, and insurance period. The fields collected from the customer behavior data source include the insurance channel, number of policy changes, number of overdue premium payments, and frequency of logins to online service platforms. The fields collected from the historical claims data source include the number of claims, cause of claim, payout amount, and payout conclusion. Source tags are represented using letters and numbers: policy data source is labeled S1, customer behavior data source is labeled S2, and historical claims data source is labeled S3. Feature numbers also use letters and numbers, numbered sequentially according to the fixed arrangement of fields within each data source; for example, the feature number corresponding to insured age is A01, and the feature number corresponding to the number of claims is C01. The original feature quadruple is a combination of four data items: the value of a policy in a certain field, the source tag of that field, the feature number, and the policy identifier corresponding to that policy. These quadruples are stored independently for easy retrieval and statistical analysis according to different dimensions.
[0021] For policies with no claims records, the number of claims feature is padded to zero, and the cause of claims code feature is padded to the pre-agreed code 999. The reason for choosing zero instead of other values is that the absence of claims records itself means that no claims actually occurred during the observation period. The value of zero accurately reflects this objective fact rather than being artificially estimated or interpolated, and will not introduce bias into subsequent statistical calculations based on this field. Code 999 is a reserved code in the cause of claims category field that is not occupied by actual business reasons. It is used to mark that this field is in a no-claims state, and is distinguished from other codes that indicate specific claims reasons, so as not to cause confusion with the actual business meaning. After the completion process, the completed feature quadruples are merged together with the original feature quadruples corresponding to the policies with existing claims records according to the policy identifier. All quadruples from the three types of data sources under the same policy identifier, covering different source labels and feature numbers, are organized together to form the original feature set with source labels. Each policy in this set has complete corresponding field records under the three types of data sources, and there are no structural gaps caused by missing claims records.
[0022] In one specific embodiment, step S2 includes: The numerical features in the original feature set are subjected to equal-frequency binning to obtain the numerical feature interval numbers, and the categorical features in the original feature set are subjected to encoding and reuse processing to obtain the categorical feature interval numbers. The item code is obtained by concatenating the source label, feature number, numerical feature interval number, and categorical feature interval number; The item codes are merged based on the policy identifier to obtain transactions. All transactions corresponding to all policies are merged to obtain a transaction item set database. Based on the transaction item set database, the occurrence frequency of each code in all transactions is statistically processed to obtain the marginal occurrence rate corresponding to each code.
[0023] Specifically, the equal-frequency binning processing of numerical features refers to the following steps for fields with continuous value characteristics, such as insured age, sum insured, premium payment method corresponding to premium amount, and compensation amount. First, the values of this field in all policy samples are sorted from smallest to largest. Then, the sorted value sequence is divided into five intervals. The division method ensures that the number of policy samples falling into each interval is roughly the same. After the interval division is completed, each interval is assigned a numerical number, which is arranged sequentially from one to five. This numerical number is the numerical feature interval number. The reason for dividing into five intervals instead of more or fewer intervals is that too few intervals will result in a large range of values within the same interval, masking the original risk differences between different value segments. Too many intervals will result in too few policy samples falling into a single interval, leading to unstable marginal incidence rates obtained based on the subsequent statistics of that interval. Five intervals achieve a balance between the two. Category-based feature code reuse refers to the direct use of the original business code values of fields with limited value types and existing business coding systems, such as occupational categories, insurance channels, causes of accidents, and claims conclusions, as interval numbers without additional interval division calculations. For example, the original business codes such as teacher codes and doctor codes in the occupational category field can be directly used as the interval numbers of the corresponding category-based features of the field.
[0024] Source tags, feature numbers, and numerical or categorical feature interval numbers are concatenated in a fixed order: source tag first, feature number in the middle, and interval number last, separated by hyphens to form an item code. For example, if the feature number corresponding to the insured age field in a policy data source is A01, and the value of this field in a certain policy falls into the second interval, the concatenated item code is denoted as S1-A01-2. All item codes obtained under the same policy identifier through the aforementioned concatenation process are grouped together to form a transaction. One transaction corresponds to the complete item code combination of all fields discretized under the three data sources for one policy. The transactions corresponding to all policies participating in this analysis are merged one by one to form a transaction item set database. The total number of transactions in this database is equal to the total number of policies participating in the analysis. For each item code that appears in the transaction itemset database, count the number of times that item code appears in all transactions. Divide the number of times by the total number of transactions, and the resulting ratio is the marginal occurrence rate corresponding to that item code. The smaller the marginal occurrence rate, the rarer the field value represented by that item code appears in all policies.
[0025] In one specific embodiment, step S3 involves performing dynamic minimum support threshold filtering on the candidate set to obtain a candidate set that passes the initial screening, including: Extract the marginal occurrence rate corresponding to each code in the candidate item set from the marginal occurrence rate, multiply the marginal occurrence rates corresponding to each code, and then multiply the result of the multiplication process according to the total number of transactions in the transaction item set database to obtain the expected co-occurrence frequency of the candidate item set. The expected co-occurrence frequency of the candidate options set is weighted according to the preset global benchmark coefficient to obtain the dynamic minimum support threshold of the candidate options set. The transaction itemset database is scanned and processed to obtain the actual co-occurrence frequency of candidate itemsets; The actual co-occurrence frequency of the candidate options is compared with the dynamic minimum support threshold of the candidate options to obtain the candidate options that pass the initial screening.
[0026] Specifically, the candidate set is composed of two or more item codes that co-occur within at least one transaction in the transaction itemset database. The combination methods are listed sequentially, including pairing item codes one by one and joint pairing of multiple item codes. Each pairing combination is treated as an independent candidate set for subsequent screening. For a given candidate set, the marginal occurrence rate value corresponding to each item code included in the candidate set is extracted from the marginal occurrence rate. These values are then multiplied sequentially to obtain a product. This product represents the theoretical proportion of these item codes appearing simultaneously under the assumption that each code appears independently and does not affect the others. Multiplying this product by the total number of transactions recorded in the transaction itemset database yields the expected co-occurrence frequency of the candidate set. The expected co-occurrence frequency represents the number of times the candidate set should appear in all policies, calculated based on the independence assumption. The global baseline coefficient is set to 0.8. The value is less than one because the expected co-occurrence frequency is a theoretical value calculated based on the independence assumption. It is normal for the actual co-occurrence frequency to fluctuate around the theoretical value. If the expected co-occurrence frequency is used as the screening threshold, candidate sets that are slightly lower than the expected value due to normal statistical fluctuations will be eliminated. The value of 0.8 leaves a 20% fluctuation space on the basis of the expected co-occurrence frequency. The dynamic minimum support threshold of the candidate set is set as the product of the global baseline coefficient and the expected co-occurrence frequency.
[0027] Scanning the transaction itemset database involves reading each transaction record in the database one by one, determining whether the transaction simultaneously contains all the item codes involved in a candidate itemset. If so, the count is incremented by one. The total count obtained after traversing all transactions is the actual co-occurrence frequency of the candidate itemset. The actual co-occurrence frequency is compared one by one with the previously calculated dynamic minimum support threshold. Candidate itemsets whose actual co-occurrence frequency reaches or exceeds the dynamic minimum support threshold are retained, while candidate itemsets whose actual co-occurrence frequency is lower than the dynamic minimum support threshold are eliminated. The candidate itemsets retained after comparison are the candidate itemsets that pass the initial screening. The candidate itemsets that pass the initial screening are recorded together with their corresponding actual co-occurrence frequency and expected co-occurrence frequency.
[0028] In one specific embodiment, step S3 involves calculating the conditional support ratio of the candidate option set, including: From the processing results corresponding to the candidate sets that passed the initial screening, extract the actual co-occurrence frequency and the expected co-occurrence frequency of the candidate sets; The ratio of the actual co-occurrence frequency of the candidate options set to the expected co-occurrence frequency of the candidate options set is calculated to obtain the conditional support ratio of the candidate options set. The candidate sets that pass the initial screening are labeled according to the conditional support ratio of the candidate sets to obtain the candidate sets with conditional support ratio. The candidate options set with conditional support ratios is associated with and stored with the actual co-occurrence frequency of the candidate options set to obtain the input data for shrinkage correction.
[0029] Specifically, the conditional support ratio is calculated by dividing the actual co-occurrence frequency of the candidate sets that passed the initial screening by the expected co-occurrence frequency of the same candidate set. This ratio reflects the deviation between the number of times the candidate sets appear simultaneously in real policy data and the theoretical number calculated based on the independent assumption of each code. A ratio greater than one indicates that the multiple codes included in the candidate set appear simultaneously in actual policies more frequently than predicted by the independent assumption. A ratio equal to one indicates that the actual occurrence of the candidate set matches the prediction result of the independent assumption and does not show any signs of association beyond statistical randomness. A ratio less than one indicates that the actual occurrence of the candidate set is lower than the level predicted by the independent assumption. The annotation process involves adding a data field to the existing records of the candidate sets that passed the initial screening. The calculated conditional support ratio is then filled into this field, so that each candidate set that passed the initial screening has corresponding records with three values: actual co-occurrence frequency, expected co-occurrence frequency, and conditional support ratio. After annotation, candidate sets with conditional support ratios are obtained.
[0030] Association storage processing refers to organizing the candidate options set with conditional support ratio and the corresponding actual co-occurrence frequency into the same record. Both values are stored together in the same storage unit, forming the input data required for shrinkage correction processing. The reason why the actual co-occurrence frequency needs to be retained together with the conditional support ratio rather than just the ratio itself is that the conditional support ratio only reflects the relative degree of deviation and cannot reflect the absolute number of times the candidate options set is observed in the original data. The shrinkage correction process needs to judge the statistical reliability of the conditional support ratio based on the absolute magnitude of the actual co-occurrence frequency. Therefore, the two values must be retained in pairs and passed to the next processing stage together. The absence of either one will cause the shrinkage correction process to fail.
[0031] In one specific embodiment, step S3 involves performing a shrinkage correction process on the conditional support ratio of the candidate options set to obtain the corrected relevance score of the strongly correlated candidate options set, including: The shrinkage weights of the candidate sets are obtained by calculating the ratio between the actual co-occurrence frequency of the candidate sets and the preset shrinkage smoothing constant. The conditional support ratio of the candidate options set is weighted according to the shrinkage weight of the candidate options set to obtain the first weighted value. The preset benchmark value is weighted according to the difference between 1 and the shrinkage weight to obtain the second weighted value. The first weighted value and the second weighted value are summed to obtain the corrected relevance score of the candidate set; The correction correlation scores of the candidate options are compared and processed according to the preset correction correlation score threshold to obtain a strongly correlated candidate options.
[0032] Specifically, the shrinkage smoothing constant is set to 20. The shrinkage weight is obtained by dividing the actual co-occurrence frequency of the candidate item set by the sum of the actual co-occurrence frequency and the constant. The basis for setting the value to 20 is that in the policy transaction item set database involved in this application, the actual co-occurrence frequency of a single candidate item set is generally distributed in the range of several to hundreds of times. The constant 20 is on the same order of magnitude as the low-to-mid range of this distribution range. When the actual co-occurrence frequency of the candidate item set is much less than 20, the shrinkage weight is close to zero. At this time, the conditional support ratio of the item set is mainly calculated based on a very small number of policy records, which is not statistically reliable. When the actual co-occurrence frequency of the candidate item set is much greater than 20, the shrinkage weight is close to one. At this time, the conditional support ratio is supported by a sufficient number of policy records, and the statistical result tends to be stable. The order of magnitude of the constant 20 allows the shrinkage weight to form a continuous and gradual change as the actual co-occurrence frequency increases, rather than a sudden jump. The preset baseline value is 1. The basis for this value is that it corresponds to the theoretical level that the conditional support ratio should present when the codes contained in the candidate options are independent and there is no correlation. This value will bring the candidate options with insufficient actual co-occurrence frequency and insufficient statistical reliability closer to this value. It is equivalent to reverting the correlation of these candidate options to the baseline state of no correlation when there is insufficient evidence, rather than arbitrarily retaining their values that may be too high due to accidental factors.
[0033] After the shrinkage weight is determined, the shrinkage weight is multiplied by the conditional support ratio of the candidate option set to obtain the first weighted value; the difference obtained by subtracting the shrinkage weight from 1 is multiplied by the preset benchmark value 1 to obtain the second weighted value; the first weighted value and the second weighted value are added together to obtain the corrected relevance score of the candidate option set. This score is essentially a weighted fusion of the conditional support ratio and the benchmark value 1 according to the shrinkage weight. The larger the shrinkage weight, the closer the score is to the conditional support ratio itself, and the smaller the shrinkage weight, the closer the score is to the benchmark value 1. The threshold for the corrected relevance score is set at 1.2. The reason for setting the value higher than the baseline value of 1 is that when the score is exactly equal to 1, it means that the candidate set is still at the baseline level of no relevance after shrinkage and fusion. If the threshold is also set to 1, a large number of candidate sets that are judged as strongly related only because of their large shrinkage weight and slightly higher conditional support ratio will be included. A considerable number of these sets are actually weakly related. The threshold is set to 1.2, which requires that the corrected relevance score of the candidate set significantly exceeds the baseline level after overcoming the effect of shrinkage and fusion pulling it towards the baseline value of 1, in order to be judged as a strongly related candidate set. The corrected relevance score is compared with this threshold one by one. Candidate sets with a score of 1.2 or higher are retained, and candidate sets with a score lower than 1.2 are removed. The retained candidate sets are the strongly related candidate sets, and their corresponding corrected relevance scores are recorded along with the candidate sets.
[0034] Figure 2This is a schematic diagram comparing the numerical distribution before and after shrinkage correction in this embodiment of the application. The dashed lines in the figure represent the cumulative distribution of the conditional support ratios of each candidate option set in all candidate option sets before shrinkage correction. The solid lines represent the cumulative distribution of the corrected correlation scores of each candidate option set in all candidate option sets after shrinkage correction. The vertical dotted lines in the figure mark the numerical positions of the preset corrected correlation score threshold. As can be seen from the figure, the numerical distribution after shrinkage correction is more concentrated in the area near and below the threshold compared to before the treatment. This reflects the regulating effect of shrinkage correction on candidate option sets with low actual co-occurrence frequency and high conditional support ratios due to insufficient sample size, bringing them closer to the benchmark value.
[0035] Figure 3 This diagram illustrates a comparison of the number of strongly correlated candidate sets identified under different support filtering methods in this application. The horizontal axis represents four categories: policy element combinations, behavioral feature combinations, cross-source claims association combinations, and total combinations, categorized by item coding source labels. The vertical axis represents the number of strongly correlated candidate sets identified under the corresponding category. The diagram uses three different filled bars to represent the number of identifications under each category: a unified fixed support threshold method, a hierarchical support threshold method, and the dynamic threshold contraction correction method described in this application. As can be seen from the diagram, in the policy element combinations and behavioral feature combinations categories, which are combinations with a single item coding source, the number of identifications obtained by the three methods is relatively close. However, in the cross-source claims association combination category, the number of identifications obtained by the method described in this application is significantly higher than the other two existing methods, demonstrating the difference between the dynamic threshold contraction correction method described in this application and existing technologies in the ability to identify cross-source risk factor combinations originating from structured sparse claims data sources.
[0036] In one specific embodiment, step S4 includes: From the strongly correlated candidate set, select the strongly correlated candidate set that contains the risk result label item code, take the risk result label item code as the latter item, and take the remaining item codes in the corresponding strongly correlated candidate set except for the latter item as the former item to form the candidate rule; The frequency of the preceding item in the transaction item set database is statistically processed to obtain the frequency of the preceding item. The confidence of the candidate rule is obtained by calculating the ratio between the actual co-occurrence frequency of the corresponding strongly associated candidate item set and the frequency of the preceding item. The confidence of the candidate rule is compared with the preset minimum confidence threshold to obtain the candidate rule that passes the confidence screening. Extract the marginal occurrence rate corresponding to the subsequent term from the marginal occurrence rate. Calculate the ratio between the confidence level of the candidate rule selected through confidence level screening and the marginal occurrence rate corresponding to the subsequent term to obtain the lift of the candidate rule. Compare the lift of the candidate rule according to the preset minimum lift threshold to obtain the policy risk association rule set. The item codes corresponding to the policy to be analyzed are matched with the preceding terms of each candidate rule in the policy risk association rule set to obtain matched candidate rules. The corrected correlation score, confidence score, and lift score of each matched candidate rule are multiplied to obtain multiple candidate rule scores, where each candidate rule corresponds to one candidate rule score. The multiple candidate rule scores are summed, and then the ratio is calculated based on the total number of matched candidate rules to obtain a comprehensive risk score. The comprehensive risk score is compared with a preset risk level threshold to obtain the policy risk analysis result.
[0037] Specifically, the risk outcome label code includes two values: high claim risk and low claim risk. This value is determined by whether a historical policy made a claim during the set observation period and whether the corresponding payout amount when a claim was made exceeds the 80th percentile value after sorting the total payout amounts of all claiming policies from smallest to largest. Policies that did not make a claim, or those that made a claim but whose payout amount did not exceed this percentile value, have a low claim risk label code. Policies that made a claim and whose payout amount exceeded this percentile value have a high claim risk label code. The reason for choosing the 80th percentile instead of other percentiles is that the payout amounts in historical claim data show a distribution characteristic where a few policies have higher payouts and most policies have lower payouts. The 80th percentile value can distinguish policies whose payout size is at the tail end of the distribution and has a greater impact on the overall claim expenditure, serving as the target risk outcome to be identified in subsequent rule mining. The combination containing the risk outcome label code in the strongly correlated candidate set is split into an antecedent and a consequent. The consequent is fixed as the risk outcome label code, and the antecedent is the set of codes of the remaining items in the combination except for the consequent. The number of times the antecedent appears alone in the transaction item set database is the antecedent frequency. The confidence of the candidate rule is obtained by dividing the actual co-occurrence frequency of the combination by the antecedent frequency. The confidence is reflected in the policy containing the risk factor combination represented by the antecedent and is also determined as the proportion of the corresponding claim risk outcome.
[0038] The preset minimum confidence threshold is set at 0.6. This value is chosen because it is higher than the level of random guessing. This allows us to retain candidate rules that we are confident in determining as the corresponding claim risk outcome after the occurrence of the preceding item, while preventing the large-scale elimination of candidate rules with medium to high confidence that still have practical reference value due to an excessively high threshold. Candidate rules selected through confidence level screening are further divided by the marginal occurrence rate of the subsequent term to obtain the lift. The lift reflects the extent to which the occurrence of the preceding term improves the occurrence of the subsequent term relative to the basic occurrence rate of the entire policy. The preset minimum lift threshold is set to 1. The reason for setting the value to 1 is that a lift of 1 means that the occurrence of the preceding term does not have a positive or negative impact on the occurrence of the subsequent term. There is no correlation between the preceding and subsequent terms that exceeds the basic proportion. Only candidate rules with a lift greater than 1 are retained. This distinguishes between rules where the preceding term has a positive effect on the claim risk outcome and rules where the co-occurrence of the preceding term and the claim risk outcome is merely a coincidence. The candidate rules obtained after dual screening by confidence level and lift constitute the policy risk association rule set. For each policy to be analyzed, its corresponding item code combination is compared one by one with the preceding item of each candidate rule in the policy risk association rule set. If the item code combination of the policy to be analyzed completely contains the preceding item of a certain candidate rule, then that candidate rule is determined to be a hit. For all the hit candidate rules, the corrected association score, confidence score, and lift score corresponding to each candidate rule are multiplied together, summed, and then divided by the total number of hit candidate rules. The resulting value is the policy comprehensive risk score for the policy to be analyzed. Policies with a comprehensive risk score greater than 2 are determined to be high-risk, policies with a comprehensive risk score greater than 1 but not exceeding 2 are determined to be medium-risk, and policies with a comprehensive risk score not exceeding 1 are determined to be low-risk. The policy risk analysis result consists of three parts: the risk level of the policy, the list of hit candidate rules, and the comprehensive risk score.
[0039] The above describes the policy risk analysis method based on association rule mining in the embodiments of this application. The following describes the policy risk analysis system based on association rule mining in the embodiments of this application. One embodiment of the policy risk analysis system based on association rule mining in the embodiments of this application includes: The extraction module is used to perform feature extraction processing on multi-source policy data, customer behavior data and historical claims data to obtain the original feature set with source labels; The encoding module is used to perform interval discretization encoding on the original feature set to obtain a transaction item set database; The correction module is used to perform dynamic minimum support threshold filtering on candidate itemsets based on the marginal occurrence rate of each code in the transaction itemset database to obtain candidate itemsets that pass the initial screening. Then, based on the actual co-occurrence frequency of the candidate itemsets, the conditional support ratio of the candidate itemsets is subjected to shrinkage correction to obtain the corrected correlation score of the strongly correlated candidate itemsets. The determination module is used to generate a policy risk association rule set from the strongly correlated candidate set, and to perform risk level determination processing on the policy to be analyzed according to the policy risk association rule set to obtain the policy risk analysis result.
[0040] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the policy risk analysis method based on association rule mining.
[0041] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for analyzing the risk of an insurance policy based on association rule mining, characterized in that, The method includes: Step S1: Perform feature extraction processing on multi-source policy data, customer behavior data, and historical claims data to obtain the original feature set with source labels; Step S2: Perform interval discretization encoding on the original feature set to obtain a transaction itemset database; Step S3: Based on the marginal occurrence rate of each code in the transaction itemset database, perform dynamic minimum support threshold filtering on the candidate itemsets to obtain candidate itemsets that pass the initial screening. Then, based on the actual co-occurrence frequency of the candidate itemsets, perform shrinkage correction processing on the conditional support ratio of the candidate itemsets to obtain the corrected correlation score of the strongly correlated candidate itemsets. This includes: extracting the marginal occurrence rate corresponding to each code in the candidate itemsets from the marginal occurrence rate, multiplying the marginal occurrence rates corresponding to each code, and then calculating the corrected correlation score based on the total number of transactions in the transaction itemset database. The product of the product processing results is performed to obtain the expected co-occurrence frequency of the candidate itemsets; the expected co-occurrence frequency of the candidate itemsets is weighted according to a preset global benchmark coefficient to obtain the dynamic minimum support threshold of the candidate itemsets; the transaction itemset database is scanned to obtain the actual co-occurrence frequency of the candidate itemsets; the actual co-occurrence frequency of the candidate itemsets is compared with the dynamic minimum support threshold of the candidate itemsets to select candidate itemsets that pass the initial screening; the actual co-occurrence frequency of the candidate itemsets is extracted from the processing results corresponding to the candidate itemsets that pass the initial screening. The actual co-occurrence frequency and the expected co-occurrence frequency of the candidate options set are calculated; the ratio of the actual co-occurrence frequency of the candidate options set to the expected co-occurrence frequency of the candidate options set is calculated to obtain the conditional support ratio of the candidate options set; the candidate options sets that pass the initial screening are labeled according to the conditional support ratio of the candidate options set to obtain candidate options sets with conditional support ratios; the candidate options sets with conditional support ratios are associated and stored with the actual co-occurrence frequency of the candidate options set to obtain the input data for shrinkage correction processing; the actual co-occurrence frequency of the candidate options set and the preset shrinkage are then used to calculate the conditional support ratio of the candidate options set. A smoothing constant is used to calculate the ratio, resulting in the shrinkage weight of the candidate antecedents. The conditional support ratio of the candidate antecedents is then weighted according to the shrinkage weight to obtain a first weighted value. A preset benchmark value is then weighted according to the difference between the first and second weighted values to obtain a second weighted value. The first and second weighted values are summed to obtain the corrected relevance score of the candidate antecedents. The corrected relevance scores of the candidate antecedents are compared against a preset corrected relevance score threshold to filter out strongly correlated candidate antecedents. Step S4: Generate a policy risk association rule set from the strongly correlated candidate set, and perform risk level determination processing on the policy to be analyzed according to the policy risk association rule set to obtain the policy risk analysis result.
2. The policy risk analysis method based on association rule mining according to claim 1, characterized in that, Step S1 includes: Data is collected and processed from policy data sources, customer behavior data sources, and historical claims data sources to obtain raw policy data, raw behavior data, and raw claims data. The original policy data, the original behavior data, and the original claims data are subjected to feature extraction processing to obtain an original feature quadruple, which includes a policy identifier, a source tag, a feature number, and a feature value. Based on the original claims data, the number of claims and the cause of claims coding features in the original feature quadruples corresponding to policies for which there are no corresponding claims records are completed to obtain the completed feature quadruples. The complete feature quadruples are merged based on the policy identifier to obtain the original feature set with source labels.
3. The policy risk analysis method based on association rule mining according to claim 1, characterized in that, Step S2 includes: The numerical features in the original feature set are subjected to equal-frequency binning to obtain numerical feature interval numbers, and the categorical features in the original feature set are subjected to encoding reuse processing to obtain categorical feature interval numbers. The item code is obtained by concatenating the source label, feature number, numerical feature interval number, and categorical feature interval number; The item codes are merged according to the policy identifier to obtain transactions. All transactions corresponding to all policies are merged to obtain a transaction item set database. Based on the transaction item set database, the occurrence frequency of each code in all transactions is statistically processed to obtain the marginal occurrence rate corresponding to each code.
4. The policy risk analysis method based on association rule mining according to claim 1, characterized in that, Step S4 includes: From the strongly correlated candidate set, a strongly correlated candidate set containing risk result label item codes is selected. The risk result label item codes are used as the latter item, and the codes of the remaining items in the corresponding strongly correlated candidate set except for the latter item are combined as the former item to form candidate rules. The frequency of occurrence of the preceding item in the transaction item set database is statistically processed to obtain the preceding item frequency. The ratio of the actual co-occurrence frequency of the corresponding strongly associated candidate item set to the preceding item frequency is calculated to obtain the confidence level of the candidate rule. The confidence level of the candidate rule is compared with the preset minimum confidence level threshold to obtain the candidate rule that passes the confidence level screening. Extract the marginal occurrence rate corresponding to the subsequent term from the marginal occurrence rate, calculate the ratio between the confidence level of the candidate rule selected by confidence level and the marginal occurrence rate corresponding to the subsequent term to obtain the lift of the candidate rule, compare the lift of the candidate rule with the preset minimum lift threshold, and filter to obtain the policy risk association rule set; The item codes corresponding to the policy to be analyzed are matched with the preceding terms of each candidate rule in the policy risk association rule set to obtain matched candidate rules. The corrected correlation score, confidence score, and lift score of each matched candidate rule are multiplied to obtain multiple candidate rule scores, where each candidate rule corresponds to one candidate rule score. The multiple candidate rule scores are summed, and then the ratio is calculated based on the total number of matched candidate rules to obtain a comprehensive risk score. The comprehensive risk score is compared with a preset risk level threshold to obtain the policy risk analysis result.
5. A policy risk analysis system based on association rule mining, characterized in that, For implementing the policy risk analysis method based on association rule mining as described in any one of claims 1-4, the policy risk analysis system based on association rule mining includes: The extraction module is used to perform feature extraction processing on multi-source policy data, customer behavior data and historical claims data to obtain the original feature set with source labels; The encoding module is used to perform interval discretization encoding on the original feature set to obtain a transaction item set database; The correction module is used to perform dynamic minimum support threshold filtering on candidate item sets based on the marginal occurrence rate of each code in the transaction item set database to obtain candidate item sets that pass the initial screening. Then, based on the actual co-occurrence frequency of the candidate item sets, it performs shrinkage correction on the conditional support ratio of the candidate item sets to obtain the corrected correlation score of the strongly correlated candidate item sets. This includes: extracting the marginal occurrence rate corresponding to each code in the candidate item set from the marginal occurrence rate; multiplying the marginal occurrence rates corresponding to each code; and then, based on the total transaction frequency in the transaction item set database... The expected co-occurrence frequency of the candidate itemsets is obtained by multiplying the results of the consecutive product operations. The expected co-occurrence frequency of the candidate itemsets is then weighted according to a preset global benchmark coefficient to obtain the dynamic minimum support threshold of the candidate itemsets. The transaction itemset database is scanned to obtain the actual co-occurrence frequency of the candidate itemsets. The actual co-occurrence frequency of the candidate itemsets is compared with the dynamic minimum support threshold of the candidate itemsets to filter out the candidate itemsets that pass the initial screening. From the processing results corresponding to the candidate itemsets that pass the initial screening, the [missing information - likely related to data extraction or processing] of the candidate itemsets is extracted. The actual co-occurrence frequency and the expected co-occurrence frequency of the candidate options set are calculated. The ratio of the actual co-occurrence frequency to the expected co-occurrence frequency of the candidate options set is calculated to obtain the conditional support ratio of the candidate options set. The candidate options sets that pass the initial screening are labeled according to the conditional support ratio to obtain candidate options sets with conditional support ratios. The candidate options sets with conditional support ratios are associated and stored with the actual co-occurrence frequency of the candidate options set to obtain the input data for shrinkage correction processing. The actual co-occurrence frequency of the candidate options set and the preset... The shrinkage smoothing constant is used to calculate the ratio, resulting in the shrinkage weight of the candidate option set. The conditional support ratio of the candidate option set is then weighted according to its shrinkage weight to obtain a first weighted value. A preset benchmark value is then weighted according to the difference between the first and second weighted values to obtain a second weighted value. The first and second weighted values are summed to obtain the corrected relevance score of the candidate option set. The corrected relevance scores of the candidate option set are compared according to a preset corrected relevance score threshold to filter out strongly correlated candidate option sets. The determination module is used to generate a policy risk association rule set from the strongly correlated candidate set, and to perform risk level determination processing on the policy to be analyzed according to the policy risk association rule set to obtain the policy risk analysis result.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by a processor, it causes the processor to execute the policy risk analysis method based on association rule mining as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Method and apparatus used for mining association rules among car insurance risk factors
CN108460685A
MapReduce-based parallel frequent item set incremental data mining method
CN112925821A
