Coherent independent Bayesian classifier

Abstract

The invention provides a method and a system for classifier-based prediction. The method includes: receiving a multivariable input having a characteristic variable; performing prediction based on a coherent independent Bayesian classifier (CIBer) by executing a conditional probability quality function on the feature variables; and outputting the prediction category. Prediction based on the coherent independent Bayesian classifier (CIBer) includes joint coding to manually insert an order in classification features. The joint coding is configured to maintain intrinsic dependencies between the classification features.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. application No. 63 / 487,282, filed on February 28, 2023, which is hereby incorporated by reference in its entirety, including any tables, figures, or drawings contained therein. Background Art

[0003] In the field of machine learning, various methods have been developed for regression tasks. Linear regression is a fundamental technique that uses linear functions to model the relationship between input features and a continuous target variable. Support Vector Regression (SVR) utilizes support vector machines to find the hyperplane that best fits the data points while minimizing the deviation from the target variable. Decision trees (including regression trees) recursively partition the feature space based on threshold conditions to create a predictive model. Gradient boosting is an ensemble method that sequentially builds a strong regression model by combining weak learners. Neural networks, particularly deep learning architectures such as multilayer perceptrons (MLPs), have garnered significant attention due to their ability to learn complex relationships between inputs and outputs. However, even with high accuracy, one drawback of these methods is their potential lack of interpretability, which can lead to unexplainable results. This is particularly problematic as many regulatory bodies and ethical considerations require transparency and interpretability of machine learning models. A robust alternative approach is needed that combines accuracy and interpretability.

[0004] The extension of Naive Bayes to regression has been discussed for a long time.

[19] discussed the possibility of extending the commonly used Naive Bayes classifier to the regression setting, but the performance was quite unsatisfactory.

[0005] Over the past few decades, the concept of comonotonicity has been widely used by researchers in actuarial science, finance, statistics, and economics. [4], which describes a special kind of dependence between several random variables, namely that whenever the realization of one variable changes, the realizations of all the other variables change in the same direction. Synchronous motion exhibits extreme positive dependence, which generally corresponds to the extreme behavior of a random system involving variables with fixed marginal distributions but facing unknown dependencies. For example, classic results in the actuarial-financial context show that homogeneity characterizes the worse and riskiest dependence structure among several loss variables, as measured by the tail value at risk of the total loss (or other risk measures that conform to the convex order) as shown in [3]. In the face of dependency uncertainty, it is important to identify such worse dependencies because appropriate action needs to be taken if one is to act prudently. In addition to modeling dependency structures, other contemporary applications of homogeneity range from risk management of derivatives to life insurance pricing and provisioning. [3,4] provide more discussion of how homogeneity is related and applied to actuarial science and finance.

[0006] First, we review some basic definitions and useful concepts of homology. For any two vectors x, y∈R d , x≤y, if for all i=1,2,…,d, x i ≤y i , then the same definition applies if x ≥ y.

[0007] Definition 1.1. If for any x, y∈S, x≥y or x≤y, then the set is homological. If for a homological set P(X∈S)=1, then R d The random vectors X in are homologous.

[0008] In other words, the homology set S is a totally ordered set. The following proposition (as shown in [5,6]) illustrates how to describe the joint probability under homology.

[0009] Proposition 1.1. Suppose X=(X1,…,X d ) is a random vector, and its marginal cumulative distribution functions are The following four statements are equivalent:

[0010] 1.(X1,…,X d ) are homophonic;

[0011] 2. There exists a random variable W and increasing functions h1,…,h d , making

[0012] 3. For any U~U(0,1),

[0013] 4. For all x=(x1,…,x d ),

[0014] where for any i∈{1,…,d},

[0015] The third statement in Proposition 1.1 shows that a single uniformly distributed random variable U~U(0,1) can be obtained from its corresponding inverse marginal distribution Before continuing with the discussion of sampling methods for homological variables, we review the well-known inverse transformation method, which allows us to generate random numbers U~U(0,1) and use the cumulative distribution function (CDF) F X :R→[0,1] samples the random variable and then calculates the sample value Specifically, if X is a discrete random variable, the interval [0,1] can be split into k disjoint subintervals:

[0016] [0,F X (s1)),[F X (s1),F X (s2)),…,[F X (s k-1 ),1],

[0017] where s1 <s2<…<s k , and it is obvious that the event {X=s i}By the interval [F X (s i-1 ),F X (s i )) represents the probability given by the length of the interval. When it comes to a set of homological random variables, according to Proposition 1.1, generate random numbers U~U(0,1), then for all i∈{1,…,d}, the set The data generated by this sampling process follows a joint distribution with homological structure. Summary of the Invention

[0018] There remains a need in the art for improved designs and techniques for methods and systems for making predictions based on homology classifiers.

[0019] According to an embodiment of the present invention, a method for making predictions based on a classifier comprises: receiving a multivariate input having feature variables; making predictions based on a comonotone independence Bayes classifier (CIBer) by executing a conditional probability mass function on the feature variables; and outputting a predicted category. In addition, making predictions based on the comonotone independence Bayes classifier (CIBer) comprises joint encoding to artificially insert sorting in the classification features. The joint encoding is configured to maintain the inherent dependencies between the classification features. Maintaining the inherent dependencies between the classification features comprises obtaining the grouping frequency of the values of the feature variables. Further, the joint encoding is configured to encode the classification features to obtain the inherent dependencies for classification. The method may further comprise performing Laplace smoothing on zero probabilities before outputting the predicted category.

[0020] In a certain embodiment of the present invention, a method for prediction based on a homologically independent Bayesian classifier (CIBer) is provided, comprising step 1: finding the combination of values of two feature variables with the highest frequency, and encoding the corresponding feature value as 1; step 2: finding the combination with the highest frequency among the combinations of unencoded values of the feature variables, and encoding the corresponding value with the next unassigned integer; step 3: repeating step 2 until only one unencoded value remains for the feature variable, and then automatically assigning the next unassigned integer to it; and step 4: if the condition in step 3 is true for both feature variables, assigning the same integer to the values of the two feature variables; if not true, and if the remaining feature variables have two or more unencoded values, their remaining values are encoded in descending order of their corresponding margins.

[0021] In some embodiments of the present invention, a non-transitory computer-readable medium is provided, wherein the non-transitory computer-readable medium stores program instructions, wherein the program instructions are executable by a computing system to cause the computing system to perform a method for making predictions based on a classifier. The method comprises: receiving a multivariate input having feature variables; making predictions based on a homogeneous independent Bayesian classifier (CIBer) by applying a conditional probability mass function to the feature variables; and outputting a predicted class. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figures 1A-1C A common discretization method for 9 sample data according to an embodiment of the present invention is shown.

[0023] Figure 2 A comparison of different ROC curves, their corresponding AUC values, and test accuracy rates in the presence or absence of classification features according to an embodiment of the present invention is shown.

[0024] Figure 3 FIG. 4 shows the test error rate of the classifier for cross-sale prediction obtained through five-fold cross validation according to an embodiment of the present invention.

[0025] Figure 4A and 4B The test error rates for different training sizes for cross-sell prediction according to an embodiment of the present invention are shown.

[0026] Figure 5 ROC curves and associated AUC values of various classifiers in cross-selling prediction according to an embodiment of the present invention are shown.

[0027] Figure 6 The figure shows the test error rate of the classifier for default premium payment prediction obtained by five-fold cross validation according to an embodiment of the present invention.

[0028] Figure 7A and 7B The test error rates for different training sizes for default premium payment prediction according to an embodiment of the present invention are shown.

[0029] Figure 8 ROC curves and associated AUC values of various classifiers in default premium payment prediction according to an embodiment of the present invention are shown.

[0030] Figure 9 FIG. 4 shows the test error rate of a classifier for travel insurance claim prediction obtained through five-fold cross validation according to an embodiment of the present invention.

[0031] Figure 10A and 10B The test error rates for different training sizes for travel insurance claim predictions according to an embodiment of the present invention are shown.

[0032] Figure 11 Shown are ROC curves and associated AUC values for various classifiers in travel insurance claim prediction according to an embodiment of the present invention.

[0033] Figure 12 FIG. 4 shows the test error rate of a classifier for vehicle insurance claim prediction obtained through five-fold cross validation according to an embodiment of the present invention.

[0034] Figure 13A and 13B The test error rates for different training sizes for vehicle insurance claim prediction according to an embodiment of the present invention are shown.

[0035] Figure 14ROC curves and associated AUC values of various classifiers in vehicle insurance claim prediction according to an embodiment of the present invention are shown.

[0036] Figure 15 Shown are ROC curves and associated AUC values for various classifiers in travel insurance claim prediction according to an embodiment of the present invention.

[0037] Figure 16 FIG. 4 shows the test error rate of a classifier for vehicle insurance claim prediction obtained through five-fold cross validation according to an embodiment of the present invention.

[0038] Figure 17A and 17B The test error rates for different training sizes for vehicle insurance claim prediction according to an embodiment of the present invention are shown.

[0039] Figure 18 ROC curves and associated AUC values of various classifiers in vehicle insurance claim prediction according to an embodiment of the present invention are shown.

[0040] Figure 19 A K-means clustering method for continuous response variables according to an embodiment of the present invention is shown.

[0041] Figure 20 A box plot of the mean absolute error of each regressor in the house rent dataset according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] Embodiments of the present invention relate to a method and system for prediction based on homology independent Bayesian classifiers (CIBer).

[0043] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used herein, the term "and / or" includes any combination and all combinations of one or more items in the associated listed items. As used herein, unless the context clearly indicates otherwise, the singular forms "one / a kind of" and "said" are intended to include plural forms as well as singular forms. It should be further understood that when used in this specification, the term "comprising" specifies the presence of stated features, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, steps, operations, elements, components and / or their groups.

[0044] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. It should be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly defined as such herein.

[0045] When the term "about" is used in conjunction with a numerical value herein, it is understood that the value may be within the range of 90% to 110% of the value, i.e., the value may be + / - 10% relative to the stated value. For example, "about 1 kg" means 0.90 kg to 1.1 kg.

[0046] When describing the present invention, it should be understood that many techniques and steps are disclosed. Each of these techniques and steps has its own benefits, and each technique and step can also be used in combination with one or more of other disclosed techniques, or in some cases all of them. Therefore, for the sake of clarity, this specification will avoid repeating each possible combination of the various steps in an unnecessary manner. However, the specification and claims should be read under the understanding that such combinations are fully within the scope of the present invention and the claims.

[0047] In real-world datasets, we usually encounter three types of features, namely categorical, continuous, and discrete feature variables.

[0048] The Homologously Independent Bayesian Classifier (CIBer) is a Bayesian classifier developed and extended to the regression setting. It consistently outperforms regression trees across a wide range of datasets.

[0049] This achievement highlights the shortcomings of previous attempts to extend Naive Bayes to regression, as Naive Bayes fails to model the probabilistic structure of the feature variables. This naive approach may still be suitable for classification tasks, where the focus is on determining the maximum probability between classes. However, when applied to regression, this no longer holds true.

[0050] According to an embodiment of the present invention, a homology-independent Bayesian classifier (CIBer) makes predictions by executing a conditional probability mass function on a given feature variable. To this end, discretization of continuous features can facilitate a unified approach to different features without sacrificing the large amount of hidden information embedded in the features. On the other hand, homology dependency structures are only considered for feature variables with naturally ordered values. For categorical feature variables such as race, gender, and nationality, a joint encoding method is provided to artificially insert sorting in the categorical features.

[0051] Formally, a classifier accepts multivariate inputs, the components of which are called feature variables or simply features, and outputs a predicted class. Assuming there are d features in the dataset, the term “feature space” refers to the R space containing all possible combinations of feature values. d The elements in the feature space are X=(X1,…,X d ) represents the eigenvector. As mentioned above, only “discrete features” are studied, and any component X I The sample space is defined as where for each i=1, 2, ..., d, k i ∈Z + In other words, the sample space represents X i Without loss of generality, assume that for each i = 1, 2, ..., d, there is a natural order X I Any observed value x I All expressed as where for each i=1, 2, ..., d, m i ∈{1,2,…,k i For simplicity, the features can be represented by their corresponding indices, i.e. (1,…,d) represents (X1,…,X d ). This set of symbols can be used interchangeably according to the context without causing too much ambiguity. Finally, for some T∈Z + , and the “category space” is represented as where for all t=1,…,T, C t is the set of all elements belonging to a class with label / type t, and the observed class label is further denoted as c(x j )∈{1,…,T}, where for j=1,…,n, x j =(x j,1 ,…,x j,d ) is the jth observed eigenvector.

[0052] 1.1 Discretization of continuous features

[0053] Intuitively, the same category C t Individuals in should share similar characteristics, and thus the values of their corresponding continuous features are also expected to cluster together. To this end, it is natural to apply discretization to some continuous variables before any classifier transfers the task.

[0054] Figures 1A-1CThree common discretization methods for continuous feature variables are shown, including (1) equal width; (2) equal frequency; and (3) equal distribution quantiles, where the nine sample data are divided into (hyperparameters) Boxes:

[0055]

[0056] Assume that Represents the set of the b-th bin:

[0057] 1. Equal width:

[0058]

[0059] where l (d) is the width of each bin:

[0060]

[0061] Therefore, in In the case of and So that:

[0062]

[0063] 2. Equal frequency: set is x (d) The sorted ascending values of , and is the total number of samples desired in each bin.

[0064]

[0065] Therefore, in And when N=9,

[0066]

[0067] Equal distribution quantile: Assume x (d) The samples in are iid and follow a parametric distribution F with cdf θ , making Return to F θ quantile of .

[0068]

[0069] Hypothetical Sample in Calculated and therefore,

[0070]

[0071] Multi-interval discretization (MID): based on entropy minimization

[0072] First find The "optimal" binary partition of

[0073] Find the “optimal” binary partition for each of the resulting subsets;

[0074] Repeat the above steps until the stopping criterion, namely the Minimum Description Length Principle (MDLP)[8], is met.

[0075] 5. Proportional k-interval discretization (PKID): an equal frequency discretization method, where Subsets of Order Statistics

[0076]

[0077] 6. Non-disjoint discretization (NDD): constructed by PKID, using (4) in the case of atomic boxes b′, for

[0078]

[0079] In practice, especially when the number of observations is limited, if too many bins are constructed for a continuous feature variable, some of these bins may end up containing few or no observations and are therefore unlikely to provide any reasonably good empirical approximation to the underlying conditional marginal density function. In a practical implementation, the number of bins for a continuous feature variable can be manually adjusted whenever the number of observations in each bin exceeds some pre-specified threshold (e.g., 100).

[0080] Through preliminary experiments on empirical data, the above discretization method shows much better robust performance than other conventional discretization methods, such as discretization into equal-width bins based on the range of eigenvalues, at the cost of longer running time.

[0081] 1.2 Joint Encoding of Categorical Features

[0082] Unlike ordinary continuous or discrete features, which are inherently governed by arithmetic, the association measures defined in Section 1.4.2 cannot be directly applied to categorical features that are purely ordinal in nature. Furthermore, since different categorical features may not exhibit any natural ordering, the concept of homology is not immediately applicable to such variables. In this section, we present a novel encoding scheme for categorical features that simultaneously preserves the inherent dependencies between them by configuring the frequencies with which the various feature values are grouped. This approach of encoding several feature values simultaneously is called joint encoding.

[0083] First, we use a simple example to illustrate the effectiveness of joint encoding. Consider the following data set with two categorical variables, whose contingency table is shown in Table 1. Borrowing the notation from Section 1.4.2, Y and Z represent two categorical feature variables.

[0084]

[0085] Table 1: Contingency table for a sample dataset with two categorical variables.

[0086] To determine the degree of dependence between Y and Z, numerical values are assigned to the possible eigenvalues they can take, and then the association measure introduced in Section 1.4.2 is calculated. Due to the nature of this dataset, i.e., the large number of tied observations (y i ,z i ), compared with other (Pearson's ) is a more appropriate metric. First consider an arbitrary encoding scheme for Y and Z (i.e., the value of Y). For i = 1, 2, 3, y i Coded as i, and for the value of Z, for i = 1, 2, z i Encoded as i. According to this scheme,

[0087]

[0088] Its obtained

[0089]

[0090] And using the formula in Section 1.4.2, Pearson It can be calculated as follows:

[0091]

[0092] It is very close to 0 and therefore gives no hint about any possible (linear) relationship between Y and Z.

[0093] However, as can be observed from Table 1, the units (y1, z2), (y2, z2), and (y3, z2) have higher frequencies than the other units. This non-uniform distribution of data points must indicate a higher degree of correlation between Y and Z, which is not fully reflected by the arbitrary coding scheme discussed above. This, in turn, suggests that a more systematic coding method must be adopted to reveal as much as possible any hidden connections between Y and Z.

[0094] To this end, the following heuristics are provided:

[0095] 1. Find the combination of Y and Z values with the highest frequency and encode the corresponding eigenvalue as 1;

[0096] 2. Find the most frequent combination of uncoded values of Y and Z and encode the corresponding value with the next unassigned integer;

[0097] 3. Repeat step 2 until the feature variable has only one uncoded value left, then automatically assign it the next unassigned integer. If this is the case for both feature variables, the same integer is assigned to both values. Otherwise, if the remaining feature variable has two or more uncoded values, its remaining values are coded sequentially in descending order of their corresponding marginal frequencies, as if the previously removed feature variable had never existed.

[0098] Following this scheme, y2 and z1 are first encoded as 1 by noting that (y2, z1) has a maximum frequency of 250. Since z2 is now the only value of Z that has not been encoded, and this is not the case for Y, it is encoded as 2. Finally, for y1 and y3, they are encoded in descending order according to the marginal frequencies listed in the bottom row of Table 1, i.e., y1 is encoded as 2 and y3 is encoded as 3, respectively. Based on this encoding result, we can calculate

[0099]

[0100] Its obtained

[0101]

[0102]

[0103] And finally

[0104]

[0105] This is a significant increase compared to any of the above solutions, mainly due to the fact that The empirical covariance of the numerator of is greatly increased.

[0106] The above heuristic idea can be extended to d as follows c Categorical feature variables The general high-dimensional case of . Recall that for i = 1, ..., d c , X i The finite sample space of For ease of implementation, it is assumed that the feature variables have been expressed in such a way that Now, consider the size of d c dimensional hyperrectangle Each cell has a "coordinate" given by The combination of eigenvalues that can be used represents, more precisely, for m i =1,…,k i And i=1, ..., d c , they are Depend on The value of each cell in the hyperrectangle represented by is the observed frequency of the corresponding combination in the training dataset. All cells in the hyperrectangle are initially marked as available, which means that they can be freely selected. The pseudo code for the joint encoding is shown in Algorithm 1 below.

[0107] Algorithm 1 Joint encoding of classification features

[0108]

[0109]

[0110] The operation of Algorithm 1 is demonstrated using an illustrative dataset of annual sales records (in thousands) of vehicles of two specific colors (silver and red) from five different brands (A, B, C, D, E) in three countries (China, Japan, and the United States). The three categorical feature variables (i.e., brand, color, and country) are first sorted in non-ascending order by the number of possible feature values. For this particular dataset, X1 represents brand, X2 represents country, and X3 represents color. Assume that the hyperrectangle of dimensions 5×3×2 in Table 2 is generated from the data set, where each of the two charts represents a specific value of X3, as indicated in the upper left grid, and the rows and columns in each chart correspond to different values of X2 and X1, respectively. All cells in are initially marked as available (with a white background). The implementation of Algorithm 1 is as follows:

[0111] First iteration: Among all available cells, cell (C, Japan, Red) has the highest frequency of 45. Therefore, r1(C)=r2(Japan)=r3(Red)=1 is encoded, and the cells with X3=Red, the rows with X2=Japan, and the columns with X1=C in the chart are marked as unavailable. These operations result in Table 3. Note that the relevant highest frequencies are shown in bold and the unavailable cells are colored gray; the same set of display rules is used in the following steps. Since only one value of X3 (i.e., silver) has not been encoded, it is automatically encoded as random feature 2, and then X3 is fully encoded. By completely removing the cells with and then adding the count values to the same combination of X2 (country) and X1 (brand), we get dimensionality reduction, thus obtaining If the corresponding values of X1 and X2 are not encoded, the resulting hyperrectangle The cells in are marked as available, for example, as in the new 5×3 rectangle in the left table of Table 4 shown.

[0112] Second iteration: Of all the available cells in , cell (B, USA) has the highest frequency of 45. It should be noted that due to the joint encoding nature of the algorithm, silver (color), B (brand) and USA (country) must be "jointly encoded", and since silver (color) is encoded as 2, the same number is also used to encode B (brand) and USA (country), that is, r1(B)=r2(USA)=2. Subsequently, the cells in the rows where X2=USA and the columns where X1=B are marked as unavailable, as shown in the right table of Table 4. Since only one value of X2 (i.e. China) remains unencoded, it is automatically encoded as 3, and then the random feature X2 is also fully encoded. By again performing the dimensionality reduction procedure in the same way as the first iteration, Reduced to a one-dimensional "line" of length 5, the eigenvalue size of X1 The only available units are A, B, and D, as shown in the left table of Table 5.

[0113] The third iteration (as shown in Table 5): Of the remaining three available cells shown in the table on the left, cell D has a maximum value of 50. Similarly, according to the joint encoding property mentioned in the second iteration, Japan (counter) and D (brand) must be "jointly encoded", and since the former is encoded as 3, the same number is used to encode r1(D) = 3. Cell D is then marked as unavailable, while the other cells do not need to be marked as unavailable because they represent different values of X1. It should also be noted that X1 has more than one value that has not yet been encoded, so no automatic encoding or dimensionality reduction is performed in this iteration.

[0114] The last iteration (as shown in Table 6): Of the two remaining available cells in X1, cell E has a higher frequency of 27, so r1(E) is set to 4 and cell E is marked as unavailable. Now, A becomes the only remaining value of X1 that has not been encoded, so r1(A) is set to 5.

[0115] The entire algorithm ends here. Therefore, by using Algorithm 1, the mapping r1, r2, r3 is expressed as follows:

[0116] r1(C)=1, r1(B)=2, r1(D)=3, r1(E)=4, r1(A)=5;

[0117] r2(Japan)=1, r2(United States)=2, r2(China)=3; r3(red)=1, r3(silver)=2.

[0118]

[0119] Table 2: Initial hyperrectangle

[0120]

[0121] Table 3: Encoded hyperrectangles in the first iteration

[0122]

[0123] Table 4: Rectangles obtained after dimensionality reduction in the first iteration and subsequent encoding in the second iteration

[0124]

[0125] Table 5: The resulting “lines” after dimensionality reduction in the second iteration and subsequent encoding in the third iteration

[0126]

[0127]

[0128] Table 6: The resulting “lines” after dimensionality reduction in the third iteration and subsequent encoding in the fourth and final iteration

[0129] The above method is simple, but it usually takes up more memory than necessary because the eigenvalues Some specific combinations of will not appear in the dataset, but Algorithm 1 still requires cells, even if the count value is 0, for each of all possible combinations, for example, cells (A, China, silver) and (A, Japan, red) are still created in Table 2. This concern may be particularly important in the case of a large number of categorical variables with sparse observations. However, the algorithm needs to search for the largest available cell on the entire hyperrectangle in each iteration, which is particularly time-consuming when a large number of categorical feature variables are involved. To this end, the following algorithm is considered, which performs exactly the same as Algorithm 1 but requires much less memory, especially for very sparse data. Specifically, for the categorical variables that have been sorted as in Algorithm 1 The data matrix of all categorical variables in the training dataset Depend on The purpose is to construct a reduced frequency-enhanced data matrix As input to the algorithm. In fact, first we form a reduced data matrix The different features of the rows identify all distinguishable rows of observation combinations. For example, Its corresponding cumulative frequencies form a column vector The values of , which together produce the dimension The reduced data matrix of , and then obtained by sorting the rows in descending order according to frequency for different distinguishable rows For example, Table 7 shows the training dataset The first 7 entries of the three categorical features in , and based on these 7 observations, the resulting reduced frequency augmented data matrix is depicted in Table 8 Its dimensions are 4×3+4×1. As a comment, for multiple rows with the same frequency, no further tie-breaker mechanism is considered, as this situation rarely occurs in practical considerations. Even if a tie occurs, the ordering between them has little effect on the final mapping adopted. The output is a transformation from the original sample space to the set of positive integers {1,2,…,k i} (For i=1, ..., d c ) c The pseudo code of this algorithm is shown in Algorithm 2 below.

[0130] Algorithm 2 Joint encoding of classification features

[0131]

[0132]

[0133] <![CDATA[X1]]> <![CDATA[X2]]> <![CDATA[X3]]> A China red B Japan silver A China red C USA red E USA silver A China red C USA red

[0134] Table 7: Training dataset The subsample data matrix contains the first 7 entries of the three categorical features in .

[0135] <![CDATA[X1]]> <![CDATA[X2]]> <![CDATA[X3]]> frequency A China red 3 C USA red 2 B Japan silver 1 E USA silver 1

[0136] Table 8: Reduced frequency enhancement data matrix from Table 7

[0137] Algorithm 2 is demonstrated using the same illustrative dataset of annual vehicle sales records. As in Algorithm 1, X1, X2, and X3 correspond to brand, country, and color, respectively. A 28×3+28×1 frequency-enhanced data matrix is obtained by sorting the rows of the data matrix in descending order according to their corresponding frequencies. As shown in Table 9, It should be noted that among the 5×3×2=30 possible combinations of eigenvalues, two of them have zero frequency, namely (B, United States, silver) and (A, Japan, red), which is why There are only 28 lines in the algorithm. The implementation of Algorithm 2 is as follows:

[0138] · Row 1 of : Since none of the three eigenvalues have been encoded, assign 1 to each eigenvalue, i.e., r1(C) = r2(Japan) = r3(Red) = 1. So far, silver is the only unencoded value of X3, and it is encoded as 2, and the encoding of X3 is complete.

[0139] ·based on A new data matrix is constructed by first removing the third column of X3, then adding the count values for the same combination of values from X1 and X2, but possibly different values from X3, and then augmenting this new data matrix with the corresponding frequencies, and next sorting the rows in descending order of frequency. Obtaining a 15×2+15×1 frequency augmented data matrix As shown in the extract of Table 10 for the first 4 rows. Continue coding by restarting at the first line of

[0140] · Lines 1, 2, and 3 in the table are not coded because X1 = C and X2 = Japan. , but not all feature values of these two feature variables are encoded.

[0141] · Line 4: Since both X1 = B and X2 = USA are uncoded, they are assigned the smallest unassigned integer 2. That is, r1(B) = r2(USA) = 2. So far, China is the only uncoded value of X2, and it is coded as 3. The coding of all eigenvalues of X2 is then complete.

[0142] Next, remove the The second column of X2 is then combined and sorted in the same way as before, resulting in a new 5×1+5×1 frequency-augmented data matrix As shown in Table 11. Next, by Continue coding by restarting at the first line of

[0143] · Lines 1 and 2 in : are not encoded because both X1=C and X1=B have been encoded.

[0144] · Line 3 of : Since X1=D has not been encoded yet, it is assigned the smallest possible unassigned integer 3. That is, r1(D)=3.

[0145] · Line 4 in : Since X1 = E has not yet been encoded, it is assigned the smallest unassigned integer 4, that is, r1(E) = 4. Finally, A is the only unencoded value of X1, which is encoded as 5. Thus, the complete encoding of all values of X1 is completed.

[0146] In summary, the mapping obtained by Algorithm 2 is as follows:

[0147] r1(C)=1, r1(B)=2, r1(D)=3, r1(E)=4, r1(A)=5; r2(Japan)=1, r2(USA)=2, r2(China)=3; r3(red)=1, r3(silver)=2,

[0148] This is exactly the same as the result of Algorithm 1.

[0149] <![CDATA[X1]]> <![CDATA[X2]]> <![CDATA[X3]]> frequency C Japan red 45 B Japan silver 25 C USA silver 25 C USA red 25 ... ... ... ...

[0150] Table 9: The first 4 rows of .

[0151] <![CDATA[X1]]> <![CDATA[X2]]> frequency C Japan 60 C USA 50 B Japan 49 B USA 45 ... ... ...

[0152] Table 10: The first 4 rows of .

[0153] <![CDATA[X1]]> frequency C 127 B 103 D 50 E 27 A 18

[0154] Table 11: OK.

[0155] Theorem 1.1. Algorithms 1 and 2 are equivalent.

[0156] It shows that the three seemingly different steps in the two algorithms actually produce exactly the same effect. Therefore, the two algorithms are equivalent. c , X i The mapping is from to {1,…,k i}, so each such map is initialized by a default value, for which 0 is chosen since it should not appear in the final output.

[0157] For the first iteration, by building The first line is has the maximum frequency among all possible combinations of the values of . Therefore, in Algorithm 2 The first line of encoding is equivalent to the hyperrectangle in Algorithm 1 The unit with the highest frequency is encoded.

[0158] Regarding subsequent iterations, The row j ofmax , r i (x j,i )=0, then This means that the cells in the hyperrectangle with the corresponding value combination are available; otherwise, when When , there must exist i0 such that coded, which is equivalent to the unavailable state of the corresponding cell in the hyperrectangle. Therefore, the row of the 9th row of the execution algorithm 2 in the reduced frequency enhanced data matrix corresponds to the cell with the maximum frequency among the available cells in the hyperrectangle.

[0159] For the dimensionality reduction step, Algorithm 1 removes all The count value is then compared with This is equivalent to reconstructing the hyperrectangle using only the remaining eigenvalues. On the other hand, Algorithm 2 removes Column, compare the count value with Summing the same combination of eigenvalues of and then sorting the newly obtained rows in descending order by frequency is equivalent to reconstructing the reduced frequency-augmented data matrix using only the remaining eigenvariates. Therefore, both algorithms produce the same dimensionality reduction effect. Therefore, it can be concluded that the two algorithms are equivalent.

[0160] Joint encoding is a heuristic algorithm that encodes categorical features in a way that can reveal inherent dependencies and use them for classification. After applying joint encoding, categorical features are also considered discrete and even sortable in addition to being only ordinal features.

[0161] 1.3 Laplace smoothing for zero probability

[0162] Since the conditional marginal probability The calculation is based on the empirical distribution of the training data set, so for X i and C t Some combinations of may end up with zero probability due to the lack of some relevant information in the finite sample. To overcome this problem, a well-known additive smoothing is used, which is called Laplace smoothing. Laplace smoothing assumes that the training dataset is large enough so that in class C t In the feature variable X i Adding a small value to the number of observations for each possible value of , such as the tuning parameter α>0, should be about t These observations in X i On the other hand, this approach can successfully solve the zero probability problem.

[10] More precisely, for a feature space The characteristic variable X i, additive smoothing works as follows:

[0163]

[0164] in It has characteristic value Category C t The number of observations in , N c(X)＝t Category C t The number of observations in . By setting α = 1, we adopt the so-called plus-one smoothing, which is an interesting special application of Laplace's succession rule and is one of the most convenient smoothing techniques. Further, as in the training phase of Naive Bayes, we follow the usual smoothing method, i.e., for problems with zero probability (i.e., In the training set, for some t and some Any given feature X i , for any right Perform additive smoothing.

[0165] 1.4 Coherent Independent Bayesian Classifier (CIBer)

[0166] In addition to independence (a special case of connectives), a Bayesian classifier incorporating homology is provided to characterize the dependency structure between features. At the same time, homology properties between subcategories of feature variables are extracted by a heuristic search method inspired in the context of cluster analysis (i.e., the clustering nesting algorithm of

[13] ). In practice, the classifier mainly focuses on discrete features with numerical values or simply "discrete features". For this purpose, it is assumed that the feature engineering in Section 1, i.e., the discretization of continuous features and the joint encoding of categorical features, has been performed so that all feature variables are now discrete feature variables.

[0167] 1.4.1 Empirical Conditional Joint Distribution Modeling

[0168] Although for all i=1, 2, ..., d, it can be obtained from its corresponding inverse CDF In practice, as long as the data set is large enough, the empirical distribution function can be used instead.

[0169]

[0170] where x1, ..., x n is a random sample observation, and 1 A is the indicator function of the set A. However, for simplicity, we use to replace

[0171] However, for the conditional case, the value of each feature variable is first divided based on its category, and then both the empirical conditional distribution of each feature variable and the joint conditional distribution of the feature vector given the category label are calculated as follows:

[0172] Definition 1.2. Let X = (X1,…,X d ) is the feature vector, and c(X) is its corresponding category label. For each component X in X i , given c(X), the empirical conditional cumulative distribution function is given by:

[0173]

[0174] where N A is the total number of observations in event A; the empirical conditional joint cumulative distribution function of X given c(X) is given by:

[0175]

[0176] Similar to the fourth statement in Proposition 1.1, the following conditional homology is defined as shown in [2,12].

[0177] Definition 1.3. For all x=(x1,…,x d ) and for each t=1,…,T, conditional on c(X), if and only if

[0178]

[0179] When the random variables X1, ..., X d It is in sync.

[0180] Analogous to the fact that classical homology implies conditional homology, we obtain the following homology analogy of Proposition 1.1 for the empirical and conditional cases, as shown in [2,12].

[0181] Proposition 1.2. Suppose X=(X1,…,X d ) is a random vector, and its marginal cumulative distribution functions are given by c(X). The following two statements are equivalent:

[0182] 1. Given c(X), X1,…,X d is conditionally homologous,

[0183] 2.(X1,…,X d |c(X)=t)

[0184] where for any i∈{1,…,d},

[0185] Based on Proposition 1.2, in a given category C t Calculate X1,…,X d The conditional joint probability of is equivalent to finding the largest measurable subset of [0,1] such that for any u in this subset, Similarly, in practice, Also use its empirical version To approximate. Recalling that the cumulative distribution function of discrete random variables divides the interval [0,1] into k disjoint subintervals, a calibration method for the conditional empirical joint probability under homology is obtained.

[0186] Assume X=(X1,…,X d ) is a homological random vector. For any component X i , let X i have The sample space of Then, in a given category In the case of P como The conditional joint probability mass of X in the homology setting represented by (X=s|c(X)=t) is such that

[0187]

[0188] where for any i=1, 2, ..., d, And every time use

[0189] Formula 5 shows that the conditional joint probability of homology features is equal to the probability that each observation The intersection length of all intervals of the conditional marginal probability mass of . In the next step, the focus is on determining the homological dependencies between features. The method described is named "cluster homology".

[0190] 1.4.2 Cluster Homology

[0191] Treating all features individually as homogeneous may be too coarse, so a more appropriate approach for classification problems is to partition the features into several subgroups and treat the features within the same subgroup as homogeneous, while treating all features in one subgroup as independent of all features in another subgroup. However, finding the optimal partition is time-consuming because the search complexity grows exponentially with the increase in the number of features. Therefore, an efficient heuristic search is needed to approximate the optimal partition instead of traversing all possibilities, and this problem can be solved by clustering. From a machine learning perspective, "clustering" is an unsupervised learning algorithm for partitioning unlabeled data into groups where the data objects in the same group are as similar as possible

[11] . A metric is chosen to measure the distance between any two features, and any two features with a smaller distance metric value are clustered into the same group. Some popular metrics include Euclidean distance, Manhattan distance, and Minkowski distance. [9] . In general, there are four statistical indicators that can be used to measure homological dependence, namely normalized mutual information U, Pearson r, Kendall's τ, and Spearman's ρ. In practice, these four statistical indicators are regarded as hyperparameters, among which the one with the best performance is always selected. The four commonly used statistical indicators will be recalled even in their natural empirical form. Here, Y and Z are observation pairs (y1, z1), ..., (y n ,z n ), then:

[0192] 1. Normalized Mutual Information (NMI) [16,24]: Originating from probability theory and information theory, this metric quantifies the dependency between two features through information entropy:

[0193]

[0194] in The empirical mutual information between Y and Z is expressed as follows:

[0195]

[0196] in represents the corresponding empirical probability mass function, and Represent the sample spaces of Y and Z respectively. and are the empirical entropies of Y and Z, which are defined as:

[0197]

[0198] 2. Pearson

[20] : It measures the magnitude and direction of the linear association between two continuous features that together follow a multivariate normal distribution:

[0199]

[0200] in are the sample means of Y and Z respectively.

[0201] 3. Kendall

[14] : This metric is non-parametric and quantifies the dependency between two features Y and Z. Statistically, for i≠j, an observation pair (y i ,z i ) and (y j ,z j ) is said to be consistent with:

[0202] sgn(y i -y j )=sgn(z i -z j );

[0203] Otherwise, it is said to be discordant. Kendall τ calculates the difference between the number of concordant pairs, represented by C, and the number of discordant pairs, represented by D, and then divides the difference by the total number of pair combinations

[0204]

[0205] 4. Spearman

[23] : It measures the rank correlation between two variables, or equivalently, the dependence between the ranks of two variables, since the value of Spearman's ρ between Y and Z is equal to R Y With R Z (the ranking variables of Y and Z, respectively). Mathematically, let represents the ranking of observation pairs, then Spearman is defined as:

[0206]

[0207] in and are the average rankings of Y and Z respectively.

[0208] To facilitate unified study, for any two random variables Y and Z in the rest of this paper, the sample value of any of the above statistical dependence measures is called the sample association measure For classification purposes, if the sample association measure is greater than a predetermined threshold δ∈[0,1] (also a hyperparameter), then any two different features X i and X j are considered homologous. However, all the above statistical dependence indicators are measures of "similarity" rather than "dissimilarity" commonly used in clustering. To this end, the concept of distance matrix A is introduced as follows:

[0209] Definition 1.4. Given a d-dimensional random vector X = (X1,…,X d ), if the d×d matrix A: = (a i,j ) is called a distance matrix if its entries satisfy the following conditions:

[0210] a i,j :=1-|asso(X i ,X j )|, for i, j = 1, ..., d,

[0211] where asso(X i ,X j ) is X as indicated above i and X j The theoretical counterpart of the sample association measure .

[0212] In a similar way, the empirical distance matrix can be defined by appropriately using an empirical association measure as where for i, j = 1, ..., d, the entry This matrix is useful in the following hierarchical clustering algorithm first proposed in

[13] , called Agglomerative Nested Clustering (AGNES), which is used to cluster the d input features. The implementation of this algorithm relies on the additional notion of a distance metric between two disjoint clusters of features, rather than just the distance metric of one feature to another, where the association metric described above is used. Consider two disjoint subsets of features Make Where m1 and m2 are and The number of features in .

[0213] Select Average Link as and The distance metric between is ,since it takes into account all features within each cluster; it is defined as the average distance between all possible pairs of cross-cluster features, as in

[10] :

[0214]

[0215] As a bottom-up hierarchical clustering method, the AGNES algorithm starts with all d features as singletons, i.e., the initialized clusters are {1}, ..., {d}. Then, two clusters (e.g., two disjoint subsets of {1, ..., d}, i.e. and ) According to them If the number of features is smaller, it is iteratively merged into one. Therefore, the total number of clusters is reduced by one in each iteration until the termination condition determined by the maximum acceptable distance for merging is met. Otherwise, the algorithm terminates with only one cluster remaining (i.e., {1, …, d}), meaning that all features are cluster-homological. The pseudocode for AGNES is shown in Algorithm 3 below.

[0216] Algorithm 3 Agglomerative Nesting algorithm (AGNES)

[0217]

[0218] After dividing the features into different homology clusters, the conditional joint probability density function of the clustering homology measure is calculated.

[0219] Definition 1.5. Suppose X=(X1,…,X d ) is divided into n c Clusters or equivalently divided into the following groups (For ν=1,…,n c ). Given feature i, category C t and the resulting eigenvalues For i = 1, ..., d, the conditional joint probability mass is calculated as follows:

[0220]

[0221] Among them, P como is the joint probability of the homologous random variables defined in formula (5).

[0222] 1.4.3 Probability Calibration of Highly Feature-Dependent

[0223] As mentioned in Section 1.4.2, highly dependent features (including both highly positively correlated features (correlation measure close to 1) or highly negatively correlated features (correlation measure close to -1)) are grouped into a cluster so that they can be considered homologous through modeling. For example, consider a pair of feature variables X that form a cluster i and X j (For i≠j), then according to formula (5), in a given category C t In the case and The joint probability is:

[0224]

[0225] On the one hand, if X i and X j With close to 1 The empirical distribution function is positively correlated with the large value of and In the independent variable and is monotonic in the same direction and therefore has the smallest possible gap due to well-defined changes, all of which means that (6) is very likely to be non-zero. Therefore, we do not usually encounter the zero probability problem in this case. On the other hand, if X i and X j is strongly negatively correlated, which means If the value of is very close to -1, then an increase in the empirical distribution function, such as middle increases, while the other decreases, such as in middle is reduced, so formula (6) allows a higher probability of zero values; since this zero probability problem endangers the implementation of the classification task, it can be overcome by the following calibration procedure to convert high negative correlation into high positive correlation. For example, suppose that within the cluster, the two feature variables X i and X j Empirically, there is a negative correlation, i.e. From the implementation point of view, suppose we select a feature variable as the reference object, such as X i , and for the alternative characteristic variable X j Execute X j to X′ j :=-X j Then (6) becomes:

[0226]

[0227] This procedure converts negative associations into positive ones, so the problem of encountering zero probability problems may be greatly reduced through the previous discussion.

[0228] Generally speaking, for clusters in characteristic variables Case (for ν=1、…、n c ), if they are all highly positively correlated, then (5) can be evaluated directly without any feature transformation; otherwise, the feature variables can be divided into two subgroups, such as and The characteristic variables within the same subgroup are highly positively correlated with each other, but the two groups are highly negatively correlated, and one subgroup is selected as the reference group, such as And then perform the same above transformation on all feature variables in another subgroup, i.e. In this way, all features within a cluster will be positively correlated and the zero probability problem is suppressed.

[0229] 2.1 Experimental Setup and Training and Testing Procedures

[0230] We first describe the setup and experimental procedures used to study each of the four datasets. While each dataset studied in this section contains tens or even hundreds of thousands of records, a small but representative subset of data points from each dataset is sufficient for illustration purposes. To this end, a stratified sampling approach was employed, and a subset of 10,000 points was extracted from the original dataset. This subset is referred to as the "population dataset" and is treated as the full dataset for the purposes of the corresponding empirical studies. It should also be noted that the feature engineering procedures described in Section 1 were performed using the population dataset, followed by clustering of feature variables based on carefully chosen association metrics. Specifically, Spearman's ρ was chosen for all four datasets in this section because clustering based on this metric outperformed the performance achieved using other association metrics. Furthermore, the hyperparameters of all candidate classifiers were manually tuned to maximize their performance. Specifically, a grid search is performed to determine the threshold for joint homology by setting the threshold to 0.1, 0.2, ..., 0.9 and selecting the one that gives the best performance for CIBer. While this procedure is rather ad hoc, the performance of CIBer has been demonstrated using the resulting thresholds, so only this one-step grid search is sufficient. Furthermore, due to the natural imbalance of relevant datasets, after performing a training-test split on the overall dataset, rebalancing techniques (i.e., oversampling and undersampling) can be applied to the training dataset to avoid overfitting and favor the dominant class. Finally, for fair comparison, each candidate algorithm always uses the same training and test datasets.

[0231] The empirical study aims to achieve three goals: i) validate the performance of candidate models using conventional model validation methods; ii) investigate the robustness of candidate models at different training ratio levels; and iii) compare the predictive abilities of candidate models at various training ratio levels.

[0232] For the first objective, a standard five-fold cross-validation method is used, where the overall dataset is randomly divided into five equally sized subsets in a stratified manner. For the kth iteration, where k = 1, ..., 5, the kth subset is used as the test set, while the classifier is trained using the remaining four subsets together as the training set. The trained classifier is then configured to make predictions on the selected test set, and the corresponding test error rate is calculated accordingly. For a robust classifier, the average test error should be as small as possible, and the error rates across the five iterations should be comparable.

[0233] For the second target, perform the following procedure:

[0234] 1. Extract a complete training dataset T from the overall dataset, wherein the training dataset contains 70% of the observations in the original dataset, and the remaining observations together form a test dataset;

[0235] 2. For each i=1, ..., 7, extract the alternative subset Ti from the complete training dataset T as the true training dataset, which contains the observations in the complete training dataset This corresponds to a training ratio of i × 10%;

[0236] 3. Using each real training dataset Ti obtained in step 2, fit the candidate model accordingly, then use each fitted model to predict the test dataset and calculate the corresponding error rate;

[0237] 4. Repeat steps 1-3 N = 100 times (the number of iterations was chosen because the results become stable after 100 or more iterations), and for each i = 1, ..., 7, obtain the test error rate For each candidate model, based on this, the corresponding 95% confidence interval of the test error rate is calculated using each candidate model, which is given by:

[0238]

[0239] in and The sample mean and sample standard And t 0.975,99 = 1.9842 is the 97.5th percentile of the Student's t distribution with N-1 = 99 degrees of freedom.

[0240] For the third objective, the model can be fitted using the complete training dataset T, and then a receiver operating characteristic (ROC) curve is plotted for each classifier, which depicts the relationship between the true positive rate and the false positive rate at different decision thresholds. In particular, the ROC curve of the effective classifier is configured as close as possible to the upper left corner of the unit square, while the ROC line located below or above the main diagonal of the unit square corresponds to the least efficient random classifier, as shown in

[30] . In addition, the area under the curve (AUC) of each ROC curve obtained is also calculated accordingly, and the larger the AUC value, the better the predictive ability of the model.

[0241] 2.2 Insurance Product Cross-Selling Forecast

[0242] The health insurance dataset used in this section records the biometrics and vehicle status of 380,000 health insurance policyholders, each of whom owns a single car. The dataset contains 10 feature variables: gender, age, driver's license, area code, previous insurance, vehicle age, vehicle damage, annual premium, policy sales channel, and year (the number of days the insured has been associated with the company). The response records whether the individual considered purchasing car insurance and therefore represents the label. It should be emphasized that over 50% of the feature variables are categorical in nature: gender, driver's license, previous insurance, vehicle damage, area code, and policy sales channel. The first four are assumed to be binary by default. The confidence intervals are derived from a one-sample t-test on the population mean with unknown variance.

[0243] This dataset is first used to demonstrate the nature of including categorical feature variables in the classification procedure. To this end, in addition to CIBer, the complete training dataset T with or without categorical feature variables is used to fit several representative classifiers, namely Naive Bayes, Decision Tree, and Multilayer Perceptron. The corresponding ROC curves are then plotted, and the AUC values and accuracy rates are also calculated accordingly, as shown in Figure 2. Figure 2 As shown, each color represents a classifier, and the solid (or dashed) curve corresponds to the presence (or absence) of the categorical variable in the dataset. Figure 2 As shown in Figure 3, CIBer achieves the best performance among the four classifiers in either of the two settings, regardless of the presence of categorical feature variables. Another noteworthy observation is that including categorical feature variables not only improves the accuracy and predictive power of CIBer, but also improves the accuracy and predictive power of all other classifiers. This result further demonstrates the necessity of including categorical feature variables in this section and in real-world classification tasks.

[0244] Next, we examined the clustering results of the feature variables after feature engineering, which provided economic insights into the theoretical basis behind this clustering. Using Spearman's ρ as a measure of association and a joint monotonicity threshold of 0.6, determined through grid search, we obtained two homologous clusters of the feature variables in this dataset: (vehicle age, vehicle age, policy sales channel) and (previous insurance, vehicle damage). Specifically, the homologous structure among the feature variables in the first cluster is somewhat natural, as older adults are more likely to have owned a vehicle for a longer period of time than younger adults, and younger generations are more inclined to purchase insurance products in modern ways, such as through online platforms, while older generations prefer to consult an insurance agent in person before signing a contract. On the other hand, the strong negative dependence observed in the dataset between previous insurance and vehicle damage is plausible given the underwriting requirements of the insurance industry. For careless drivers who have experienced multiple accidents and caused significant damage to their vehicles during their driving history, underwriting procedures may result in insurance companies either denying insurance or setting very high premiums that may be unaffordable. This may explain why damage records may indicate a low probability of past insurance. By jointly studying the feature variables within each of the two clusters, CIBer naturally yields stronger predictive power than other classifiers due to their built-in strong or weak dependencies. Furthermore, CIBer can reveal the strong dependency structure between feature variables in an intuitive and natural way. Other classifiers, including decision trees and multilayer perceptrons, either ignore this dependency structure, such as Naive Bayes, which relies heavily on the conditional independence assumption, or only implicitly utilize dependencies, making it infeasible to reveal the dependency structure from their results, let alone provide a valid justification from an economic perspective. Specifically, for decision trees, the sets of feature variables along the various branches of a decision rule are more likely to differ from one another, making it inconvenient to reason about any dependency structure. For multilayer perceptrons, linear dependencies between features or hidden variables in each intermediate layer are easily revealed, as reflected by corresponding coefficients with similar values and signs. However, once nonlinear activations are applied to the next layer, the dependency structure suddenly becomes less apparent. Further analysis below demonstrates the advantages of using CIBer for classification tasks compared to all other representative classifiers.

[0245] The results of the five-fold cross validation are presented in Figure 3 where NB stands for Naive Bayes, MLP stands for Multilayer Perceptron, DT stands for Decision Tree, and LR stands for Logistic Regression. Figure 3It can be observed that CIBer achieves the lowest test error rate among the candidate classifiers considered here, and the error rate is quite stable over the five iterations with the corresponding random initial seeds. It is also worth noting that the categorical variable policy sales channel can take more than 100 possible values, and given that the size of the entire training dataset T is 7,000, the training data points are quite sparse, which may have a negative impact on the prediction accuracy of the classifier and thus provide a reason for CIBer's average test accuracy of only about 85%. In summary, this result already clearly demonstrates the excellent performance of CIBer under conventional standards. Next, the robustness of CIBer is compared with other classifiers at different training ratios following the procedure described in Section 2.1. When the performance of the classifier stabilizes after 100 or more iterations, steps 1-3 are repeated 100 times. The 95% confidence intervals of the test error rate for training sizes of 0.1, ..., 0.7 are then plotted, as shown in Figure 4A and 4B As shown in Figure 3 . It can be observed that the confidence intervals are quite narrow at all training sizes, which applies not only to CIBer but also to all other classifiers. As mentioned above, this verifies the stable performance of all classifiers after 100 iterations. Among all candidate models, CIBer achieves the lowest test error rate when the training size is less than 40%. At larger training sizes, it remains quite competitive with other learners that are generally considered strong enough, with the difference in test error compared to MLP and decision tree being less than 1%. Overall, CIBer is more robust than other representative classifiers, including the recognized top and widely used classifiers mentioned above.

[0246] As another measure of predictability, the ROC curve using the complete training dataset T (i.e., the presence of categorical features) was also plotted, and the associated AUC values were calculated accordingly, as shown in Figure 5 As shown in Figure 1, CIBer's predictive power surpasses all other classifiers, as its corresponding orange ROC curve is closest to the upper left corner of the unit square. This finding is further supported by CIBer's AUC value of 0.93, which exceeds that of all other classifiers. As previously mentioned, a key reason for the relatively poor predictability of other classifiers is the lack of clustering of feature variables with strong dependencies, especially for categorical variables. Consequently, these classifiers are unable to reveal and utilize hidden connections between all feature variables, resulting in weaker performance compared to CIBer.

[0247] 2.3 Default Premium Payment Forecast

[0248] The insurance dataset records the personal profile details and premium payment history of 79,853 policyholders and has 10 feature variables: percentage (percentage of premiums paid in cash or by credit card), age (in days), underwriting score, monthly income, count 3-6 (count of premium payments delayed by 3-6 months), count 6-12 (count of premium payments delayed by 6-12 months), count over 12 (count of premium payments delayed by more than 12 months), premium on time (count of premium payments paid on time), purchasing channel, and residential area. Only the last two feature variables are categorical, with the former having five categorical values and the latter taking the binary value of "urban" or "rural." The binary response variable target takes the value 0 if the policyholder defaults on premium payment and 1 otherwise, which is, of course, the policyholder's label. As discussed in the previous section, some justification for the clustering results of correlated feature variables after feature engineering in an economic context is provided. Again, Spearman's ρ is used as the association measure, but the threshold for joint homogeneity is determined by grid search at 0.4. In this paper, only one homology cluster of feature variables was obtained in this dataset: (Counts 3-6, Counts 6-12, and Counts greater than 12). The strong dependency structure between these three variables is clearly understood, as customers who delay paying their premiums for at least three months are likely to experience some long-term financial hardship, leading to a higher probability of longer premium payment delays. It is also noteworthy that insureds can fall into financial hardship due to misfortune without any prior notice, regardless of their personal background recorded in the dataset, such as their salary level at the time of policy issuance. Interestingly, this may explain why other feature variables (such as monthly income) do not fall into this homology cluster.

[0249] As mentioned above, the results of the five-fold cross validation are as follows Figure 6 As shown in the figure, CIBer's superior performance compared to all other classifiers is evident, as it achieves the lowest error rate of 5% on average, and its error rate is stable across all five folds. In contrast, the decision tree, the best-performing classifier among the remaining classifiers, has a much higher error rate, approaching 10% overall. Therefore, it can be concluded that for this typical dataset, CIBer once again outperforms the other classifiers under five-fold cross-validation.

[0250] As for the comparison of the robustness of CIBer with other classifiers under different training ratios, the 95% confidence intervals of the test error rate when the training size is 0.1, ..., 0.7 are as follows Figure 7A and 7BAs shown in Figure 2; when the performance of the classifier stabilized after 100 or more iterations, steps 1-3 in Section 2.1 were repeated 100 times. Among all the candidate models, except for the decision tree, none of the models had a chance to challenge CIBer at a smaller training size, because when the training size was less than or equal to 30%, they all achieved almost the same test error rate, while for larger training sizes exceeding 40%, CIBer greatly outperformed all other classifiers, including the decision tree, as shown in Figure 2. Figure 7A and 7B This result convincingly demonstrates the superior robustness of CIBer over other classifiers.

[0251] Finally, the ROC curve of the candidate classifier using the full training dataset T and the associated AUC value is as follows Figure 8 As shown in Figure 2. Once again, CIBer's ROC curve (in orange) is closer to the upper left corner of the unit square than any other curve, and CIBer's AUC value of 0.97 is also the largest AUC value obtained by all classifiers. Therefore, it can be concluded that for this dataset, CIBer has the strongest predictive ability among all the classifiers considered.

[0252] 2.4 Insurance Claims Prediction

[0253] Claims prediction plays a crucial role in the pricing of insurance products, as the accuracy of such predictions is directly related to the insurance company's profits. A well-designed risk classification mechanism is crucial for more accurately predicting claims for specific policyholders. To this end, in this section, we compare the claims prediction performance of CIBer with other common representative classifiers for travel and vehicle insurance products.

[0254] 2.5 Travel Insurance

[0255] The travel insurance dataset used in this section contains personal characteristics and insurance-related information for approximately 18,000 policyholders of various travel insurance products. The dataset includes 10 feature variables: age, gender, trip duration, net sales (of travel insurance policies), agent commission, agent name, agent type, distribution channel, product name, and travel destination. The categorical feature variables are gender, agent name, agent type, distribution channel, product name, and travel destination. Gender is the only binary-valued feature variable. The binary response variable, target (now label), records whether the policyholder filed a claim.

[0256] For clustering the feature variables in this dataset, Spearman's ρ was used as the association measure, and a grid search was performed with a threshold of 0.8 for joint homogeneity. For this particular dataset, two clusters of homogeneous feature variables were identified: (net sales, agent commission) and (agent name, travel destination). The dependency structure within the first cluster is obvious, as the commission amount received by agents is typically a percentage of net sales in the insurance industry. The second cluster of feature variables is also reasonable, as travel insurance agencies often target attractive or exotic travel destinations. That is, one agency may only want to sell travel insurance products for Western European destinations, while another agency may focus primarily on products for travelers to North America. This difference may be the main reason for the strong dependency between the two feature variables in the second cluster.

[0257] Figure 9 The results of the five-fold cross-validation are shown, which again demonstrate the superior performance of CIBer compared to all other classifiers. Specifically, while competitive results were observed using the decision tree, its overall test error rate was indeed slightly higher than CIBer. Therefore, we can draw the same conclusion as in the previous sections: CIBer performs best among all candidate classifiers under the five-fold cross-validation.

[0258] Next, the 95% confidence interval of the test error rate when the training size is 0.1, ..., 0.7 is as follows Figure 10A and 10B As shown in the previous experiment, steps 1-3 in Section 2.1 were repeated 100 times, and the stable performance of the classifier was observed. Figure 10A and 10B It can be immediately noticed that the decision tree poses a significant challenge as it achieves a lower test error rate than CIBer, but only for smaller training sizes, which of course may depend on the influence of the chosen initial seed. Furthermore, given the nature of this dataset, where most of the feature variables are either discrete or categorical, it is reasonable that decision trees are suitable for this classification task. However, at larger training ratios, CIBer can learn from the training data points more efficiently due to the reduced sparsity of the data in the training set, so its test error rate undergoes a significant phase shift as it decreases more than any other classifier, and it even ends up surpassing the decision tree if the full training dataset T (instead of the entire dataset!) is used during the training phase. For this reason, even for this dataset, where decision trees should achieve the best performance, CIBer can still show higher robustness than other classifiers.

[0259] like Figure 11The ROC curves for the classifiers using the full training dataset T and the associated AUC values shown in Figure 2 provide further conclusive evidence of CIBer's superior performance. In fact, CIBer's orange ROC curve almost coincides with the "L" curve along the left and top of the unit square, and the corresponding AUC value of 0.97 is also very close to 1, indicating that it is a perfect classifier. Therefore, it is clear that CIBer has the highest predictive ability among these representative classifiers.

[0260] 2.6 Vehicle Insurance

[0261] The vehicle insurance dataset

[43] considered here contains approximately 590,000 data points and has a total of 57 feature variables, including 10 continuous feature variables, 33 discrete feature variables, and 14 categorical feature variables, and the label / response variable target records whether the insured filed a claim. The names of all feature variables encrypted by the insurance company are labeled here, so that only the type of each feature variable is identified without revealing their nature, let alone providing an economic interpretation of the clustering results. However, using Spearman ρ as the association measure and 0.5 as the threshold of joint homology determined by grid search, the clustering procedure revealed 5 clusters of homologous feature variables, namely (ind 12, ind14), (ind 16, ind18), (reg 01, reg 02, reg 03), (car 03, car 05), and (car12, car 13). Although their economic interpretation is still unclear, the following results do illustrate the importance of considering these clusters when improving classification accuracy.

[0262] As mentioned above, first Figure 12 The results of the five-fold cross-validation are shown in Figure 2. The test error rate of CIBer on the five-fold test is very stable and significantly lower than the error rates of all other classifiers. This observation clearly shows that CIBer outperforms all candidate classifiers under the five-fold cross-validation standard.

[0263] Plots the 95% confidence intervals for the test error rate for training sizes 0.1, ..., 0.7. Figure 13A and 13B We verified the significant robustness advantage of CIBer at different training ratios. Once the classifier's performance stabilized, steps 1-3 in Section 2.1 were repeated 100 times. All other candidate models only achieved a test error rate of at least 10%, regardless of the training size. However, as expected, CIBer's training error remained well below 5% for all training sizes.

[0264] In addition, if Figure 14As shown in Figure 2, a comparison of ROC curves using the full training dataset T and the corresponding AUC values indicates the predictive power of CIBer relative to other classifiers. Clearly, CIBer's orange ROC curve is closest to the upper left corner of the unit square, and it also achieves the maximum AUC value of 0.97. Therefore, CIBer's predictive power exceeds that of all other representative classifiers.

[0265] also, Figure 15 Shows the ROC curves and associated AUC values of various classifiers in travel insurance claim prediction; Figure 16 The test error rate of the classifier for vehicle insurance claim prediction obtained by five-fold cross validation is shown; Figure 17A and 17B shows the test error rate for different training sizes for vehicle insurance claim prediction; and Figure 18 Shown are the ROC curves and associated AUC values for various classifiers in vehicle insurance claim prediction.

[0266] From classification to regression

[0267] From a heuristic point of view, Bayesian classifiers cannot be extended to the context of regression. We provide a method for extending CIBer to regression. First, we refer to Figure 19 , the K-means clustering algorithm is used to discretize the continuous response variable into different clusters, where K is a hyperparameter used for tuning.

[0268] Based on the clustering results, the continuous response variable is transformed into a categorical variable. Then, since the continuous response variable has been transformed into a categorical variable, the "classification" problem is solved. Mathematically, given the continuous response in the training data The predicted value (as the posterior mean given x) is

[0269]

[0270] Real data application

[0271] CIBer is implemented on a house rental dataset that records 10 characteristic variables representing available rental properties. First, the experimental procedure adopted for the dataset is described. The dataset contains 4746 records, which is enough to undersample a small but representative portion of the original dataset, as the training dataset has a total of 3796 data points. Afterwards, 950 data points are randomly selected as the test dataset. This process is repeated 100 times, and the results are presented using a boxplot of the mean absolute error (MAE). Although these 100 experiments are not independent, this bootstrapped boxplot still allows us to understand the robustness and effectiveness of each regressor.

[0272] The performance of CIBer is compared with two commonly used regression methods, linear regression and regression tree. There are several categorical feature variables in this dataset, namely

[0273] Floor (3)

[0274] Region_Type(4)

[0275] Region_Location(5)

[0276] City (6)

[0277] ·Furniture_Status(7)

[0278] Tenant_Preferred (8)

[0279] Contact_point(10)

[0280] The continuous response variable is rent, which is the rental price of a house. Given the feature variables in the dataset, we need to predict the rental price. By implementing the procedure on the dataset, we get Figure 20 The box plot shown.

[0281] The box plot reveals that CIBer exhibits lower median mean absolute error (MAE) compared to regression trees and significantly outperforms linear regression. Although the range displayed by regression trees in the box plot is relatively narrow, it is worth noting that CIBer mostly outperforms regression trees in terms of MAE.

[0282] 6 Conclusion

[0283] To perform risk classification tasks (one of the most popular practices in the insurance industry), various machine learning algorithms have been investigated. However, some algorithms (such as decision trees) are designed for categorical feature variables and perform poorly with continuous feature variables. Other algorithms (such as MLP, SVM, and logistic regression) lack systematic treatment of categorical feature variables, complicate them with one-hot encoding, or even more extreme, discard them entirely. Furthermore, Naive Bayes, a traditional statistical learning algorithm for classification tasks, suffers from the critical drawback of ignoring any possible dependency structure between feature variables. To address this, we introduce a primitive feature engineering approach that systematically treats all continuous, discrete, and categorical feature variables simultaneously under the same method; in principle, all of these variables are converted to discrete variables. Based on this feature engineering, we present CIBer, which incorporates a simple yet powerful dependency structure by leveraging the concept of homology commonly used in actuarial science. CIBer's superior performance is demonstrated on several real-world insurance datasets covering various key aspects of the insurance industry, giving us confidence in its broad future application in InsurTech.

[0284] Beyond actuarial science, CIBer has direct applications in many other fields. For example, preliminary research indicates that CIBer has achieved significant results in representative scenarios across quantitative finance, production and logistics research, advertising and marketing science, teacher education research, and the social sciences and humanities. CIBer's superior performance over decision trees has been clearly demonstrated, and since random forests are enhanced versions of decision trees that incorporate ensemble learning techniques, this enhanced version of CIBer should outperform them.

[0285] Exemplary embodiments

[0286] Example 1. A method for prediction based on a classifier, comprising:

[0287] receiving a multivariate input having feature variables;

[0288] Making predictions based on a homology independent Bayesian classifier (CIBer) by performing a conditional probability mass function on the feature variables; and

[0289] Output the predicted category.

[0290] Embodiment 2. The method of embodiment 1, wherein making predictions based on the CIBer comprises joint encoding to artificially insert an ordering in the classification features.

[0291] Embodiment 3. The method of embodiment 2, wherein the joint encoding is configured to preserve the inherent dependencies between the classification features.

[0292] Embodiment 4. The method of embodiment 3, wherein maintaining the inherent dependencies between the categorical features comprises obtaining group frequencies of values of the feature variables.

[0293] Embodiment 5. The method of embodiment 2, wherein the joint encoding is configured to encode classification features to obtain inherent dependencies for classification.

[0294] Embodiment 6. The method of embodiment 1, further comprising performing Laplace smoothing on zero probabilities before outputting the predicted class.

[0295] Embodiment 7. The method of embodiment 1, wherein making predictions based on a homology independent Bayesian classifier (CIBer) comprises configuring the CIBer to perform regression.

[0296] Example 8. A method according to Example 7, wherein configuring the CIBer to perform regression includes: configuring a K-means clustering method to discretize continuous response input variables into different classes, where K is a hyperparameter for adjustment; and transforming the continuous response input variables into categorical variables based on the clustering results.

[0297] Example 9. A method according to Example 1, wherein prediction based on a homologically independent Bayesian classifier (CIBer) includes: optimally dividing the feature variables into multiple subgroups; and considering the feature variables within the same subgroup to be homological, and considering the feature variables in one subgroup to be independent of the feature variables in another subgroup.

[0298] Embodiment 10. The method according to embodiment 9, wherein the optimal partitioning is determined by a clustering method.

[0299] Example 11. A method for prediction based on a homology independent Bayesian classifier (CIBer), comprising:

[0300] Step 1: Find the combination of the two feature variables with the highest frequency and encode the corresponding feature value as 1;

[0301] Step 2: Find the combination with the highest frequency among the combinations of uncoded values of the feature variable, and encode the corresponding value with the next unassigned integer;

[0302] Step 3: Repeat step 2 until the feature variable has only one uncoded value left, then automatically assign it the next unassigned integer; and

[0303] Step 4: If the condition in step 3 is true for both feature variables, then the same integer is assigned to the values of both feature variables; if not true, and if the remaining feature variables have two or more uncoded values, then their remaining values are coded in descending order of their corresponding marginals.

[0304] Embodiment 12. A non-transitory computer-readable medium having program instructions stored therein, wherein the program instructions are executable by a computing system to cause the computing system to perform a method for performing prediction based on a classifier, the method comprising:

[0305] receiving a multivariate input having feature variables;

[0306] Making predictions based on a homology independent Bayesian classifier (CIBer) by performing a conditional probability mass function on the feature variables; and

[0307] Output the predicted category.

[0308] Embodiment 13. The method of embodiment 12, wherein making predictions based on the CIBer comprises joint encoding to artificially insert an ordering among classification features.

[0309] Embodiment 14. The method of embodiment 13, wherein the joint encoding is configured to preserve inherent dependencies between the classification features.

[0310] Embodiment 15. The method of embodiment 14, wherein maintaining the inherent dependencies between the categorical features comprises obtaining group frequencies of values of the feature variables.

[0311] Embodiment 16. The method of embodiment 13, wherein the joint encoding is configured to encode classification features to obtain inherent dependencies for classification.

[0312] Embodiment 17. The method of embodiment 12, further comprising performing Laplace smoothing on zero probabilities before outputting the predicted class.

[0313] Embodiment 18. The method of embodiment 12, wherein making predictions based on a homology independent Bayesian classifier (CIBer) comprises configuring the CIBer to perform regression.

[0314] Example 19. A method according to Example 18, wherein configuring the CIBer to perform regression includes: configuring a K-means clustering method to discretize a continuous response input variable into different classes, where K is a hyperparameter for adjustment; and transforming the continuous response input variable into a categorical variable based on the clustering result.

[0315] Example 20. A method according to Example 12, wherein prediction based on a homologically independent Bayesian classifier (CIBer) includes: optimally dividing the feature variables into multiple subgroups; and considering the feature variables within the same subgroup to be homological, and considering the feature variables in one subgroup to be independent of the feature variables in another subgroup.

[0316] All patents, patent applications, provisional applications, and publications mentioned or cited herein are hereby incorporated by reference in their entirety, including all figures and tables, to the extent they are not inconsistent with the explicit teachings of this specification.

[0317] It should be understood that the examples and embodiments described herein are for illustrative purposes only, and that various modifications or variations made in accordance with these examples and embodiments will be suggested to those skilled in the art and are intended to be included within the spirit and scope of this application. Additionally, any element or limitation of any invention disclosed herein or its embodiments may be combined with any and / or all other elements or limitations (individually or in any combination) or any other invention or embodiment disclosed herein, and all such combinations are intended without limitation with respect to the scope of the invention.

[0318] References

[0319] [1] M., Ayuso, M., and Guillen, M. Modelling different types of automobile insurance fraud behaviour in the spanish market. Insurance: Mathematics and Economics 24, 1-2 (1999), 67–81.

[0320] [2] Breiman, L. Description of chlorine tree development and use, 1978.

[0321] [3] Burges, CJ. A tutorial on support vector machines for pattern recognition. Datamining and knowledge discovery, 2, 2 (1998), 121–167.

[0322] [4] Catlett, J. On changing continuous attributes into ordered discrete attributes. European working session on learning (1991), Springer, pp. 164-178.

[0323] [5] Chen, Y., Cheung, K.C., Choi, H.M.C., and Yam, SCP. Evolutionary credibility risk premium. Insurance: Mathematics and Economics 93 (2020), 216–229.

[0324] [6] Cheung, KC. Characterizations of conditional comonotonicity. Journal of Applied Probability, 44, 3 (2007), 607–617.

[0325] [7] Cover, T. and Hart, P. Nearest neighbor pattern classification. IEEE transactions on information theory, 13, 1 (1967), 21–27.

[0326] [8] Deelstra, G., Dhaene, J. and Vanmaele, M. An overview of comonotonicity and its applications in finance and insurance. Advanced mathematical methods for finance. Springer, 2011, pp. 155–179.

[0327] [9] Denuit, M., Hainaut, D., and Trufin, J. Effective statistical learning methods for actuaries I: GLMs and Extensions. Springer, 2019.

[0328]

[10] Denuit, M., Hainaut, D., and Trufin, J. Effective statistical learning methods for actuaries III: Neural Networks and Extensions. Springer, 2019.

[0329]

[11] Denuit, M., Hainaut, D., and Trufin, J. Effective statistical learning methods for actuaries II: Tree-Based Methods and Extensions. Springer, 2020.

[0330]

[12] Dhaene, J., Denuit, M., Goovaerts, M.J., Kaas, R. and Vyncke, D. The concept of comonotonicity in actuarial science and finance: applications. Insurance: Mathematics and Economics, 31, 2 (2002), 133–161.

[0331]

[13] Dhaene, J., Denuit, M., Goovaerts, M.J., Kaas, R., and Vyncke, D. The concept of comonotonicity in actuarial science and finance: theory. Insurance: Mathematics and Economics 31, 1 (2002), 3–33.

[0332]

[14] Dhaene, J., Vanduffel, S., Goovaerts, M. J., Kaas, R., Tang, Q., and Vyncke, D. Risk measures and comonotonicity: a review. Stochastic Models, 22, 4 (2006), 573–606.

[0333]

[15] Dougherty, J., Kohavi, R. and Sahami, M. Supervised and unsupervised discretization of continuous features. Machine learning proceedings 1995. Elsevier, 1995, pp. 194–202.

[0334]

[16] Duda, R.O., Hart, P.E. and Stork, D.G. Pattern classification and scene analysis, vol. 3. Wiley New York, 1973.

[0335]

[17] Farkas, S., Lopez, O., and Thomas, M. Cyber claim analysis using generalized pareto regression trees with applications to insurance. Insurance: Mathematics and Economics 98(2021), 92–105.

[0336]

[18] Fayyad,U. and Irani,K. Multi-interval discretization of continuous-valued attributes for classification learning.

[0337]

[19] Frank, E., Trigg, L., Holmes, G., et al. Technical Note: Naive Bayes for Regression. Machine Learning, 41, 5–25 (2000). https: / / doi.org / 10.1023 / A:1007670802811

[0338]

[20] Friedman, J. H. A recursive partitioning decision rule for nonparametric classification. IEEE Trans. Computers 26, 4 (1977), 404–408.

[0339]

[21] Friedman, JH. A tree-structured approach to nonparametric multiple regression. Smoothing techniques for curve estimation. Springer, 1979, pp. 5-22.

[0340]

[22] Gan, G., Ma, C., and Wu, J. Data clustering: theory, algorithms, and applications. Society for Industrial and Applied Mathematics (SIAM), 2020.

[0341]

[23] Han, J., Pei, J. and Kamber, M. Data mining: concepts and techniques. Elsevier, 2011.

[0342]

[24] Jain, AK, Murty, MN and Flynn, PJ. Data clustering: a review. ACM computing surveys (CSUR), 31, 3 (1999), 264–323.

[0343]

[25] Jouini, E. and Napp, C. Conditional comonotonicity. Decisions in Economics and Finance, 27, 2 (2004), 153–166.

[0344]

[26] Kaufman, L. and Rousseeuw, P.J., Finding groups in data: an introduction to cluster analysis, vol. 344. John Wiley & Sons, 1990.

[0345]

[27] Kendall, MG. A new measure of rank correlation. Biometrika 30, 1 / 2 (1938), 81–93.

[0346]

[28] Kim, JH and Jeon, Y. Credibility theory based on trimming. Insurance: Mathematics and Economics 53, 1 (2013), 36–47.

[0347]

[29] Kotsiantis, S.B., Zaharakis, I. and Pintelas, P. Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering, 160, 1 (2007), 3–24.

[0348]

[30] Krzanowski, WJ and Hand, DJ. ROC curves for continuous data. Crc Press, 2009.

[0349]

[31] Kullback, S. Information theory and statistics. Courier Corporation, 1997.

[0350]

[32] Kumar, A. Health insurance cross sell prediction, 2020. Accessed: 2021-08-28, https: / / www.kaggle.com / anmolkumar / health-insurance-cross-sell-prediction.

[0351]

[33] Nasrudin, Z. Travel insurance, 2018. Accessed: 2021-08-28, https: / / www.kaggle.com / mhdzahier / travel-insurance.

[0352]

[34] Oh, R., Lee, Y., Zhu, D., and Ahn, J. Y. Predictive risk analysis using a collective risk model: Choosing between past frequency and aggregate severity information. Insurance: Mathematics and Economics 96 (2021), 127–139.

[0353]

[35] Pearson, K. Notes on regression and inheritance in the case of two parents proceedings of the royal society of london, 58, 240-242, 1895.

[0354]

[36] Pitselis, G. Quantile credibility models. Insurance: Mathematics and Economics 52, 3 (2013), 477–489.

[0355]

[37] Pitselis, G. Credible risk measures with applications in actuarial sciences and finance. Insurance: Mathematics and Economics 70 (2016), 373–386.

[0356]

[38] Pitselis, G. Risk measures in a quantile regression credibility framework with fama / french data applications. Insurance: Mathematics and Economics 74 (2017), 122–134.

[0357]

[39] Quinlan, JR. Discovering rules by induction from large collections of examples. Expert systems in the micro electronics age (1979).

[0358]

[40] Quinlan, JR. Induction of decision trees. Machine Learning, 1, 1 (1986), 81–106.

[0359]

[41] Rathi, P. Insurance company dataset: Premium default prediction, 2019. Accessed: 2021-08-28, https: / / www.kaggle.com / prakharrathi25 / insurance-company-dataset.

[0360]

[42] Schinzinger, E., Denuit, MM and Christiansen, MC. A multivariate evolutionary credibility model for mortality improvement rates. Insurance: Mathematics and Economics 69 (2016), 70–81.

[0361]

[43] Seguro, P. Porto Seguro's safe driver prediction, 2017. Accessed: 2021-08-28, https: / / www.kaggle.com / c / porto-seguro-safe-driver-prediction.

[0362]

[44] Shi, P. and Shi, K. Territorial risk classification using spatially dependent frequency-severity models. ASTIN Bulletin: The Journal of the IAA, 47, 2 (2017), 437–465.

[0363]

[45] Spearman, C. The proof and measurement of association between two things. The American Journal of Psychology, 15, 1 (1904), 72–101.

[0364]

[46] Sundt, B. Two credibility regression approaches for the classification of passenger cars in a multiplicative tariff. Bulletin of the International Association of Actuaries: Journal of the International Actuarial Association 17, 1 (1987), 41–70.

[0365]

[47] Verschuren, R. M. Predictive claim scores for dynamic multi-product risk classification in insurance. Bulletin of the International Association for Insurance Statistics: Journal of the International Actuarial Association 51, 1 (2021), 1–25.

[0366]

[48] Witten, IH, Frank, E., Hall, M.A., Pal, C., and Data, M. Practical machine learning tools and techniques. Data Mining (2005), vol. 2, p. 4.

[0367]

[49] Xacur, O.A.Q. and Garrido, J. Bayesian credibility for glms. Insurance: Mathematics and Economics 83 (2018), 180–189.

Claims

1. A method for prediction based on a classifier, comprising: receiving a multivariate input having feature variables; Making predictions based on a homology independent Bayesian classifier (CIBer) by performing a conditional probability mass function on the feature variables; and Output the predicted category.

2. The method of claim 1, wherein making predictions based on the CIBer comprises joint encoding to artificially insert an ordering in the classification features. The method according to claim 2 , wherein the joint encoding is configured to preserve the inherent dependencies between the classification features. The method of claim 3 , wherein maintaining the inherent dependencies between the categorical features comprises obtaining group frequencies of values of the feature variables. The method according to claim 2 , wherein the joint encoding is configured to encode classification features to obtain inherent dependencies for classification. The method of claim 1 , further comprising performing Laplace smoothing on zero probabilities before outputting the predicted class.

7. The method of claim 1, wherein performing prediction based on a homology independent Bayesian classifier (CIBer) comprises configuring the CIBer to perform regression.

8. The method of claim 7, wherein configuring the CIBer to perform regression comprises: configuring a K-means clustering method to discretize a continuous response input variable into different categories, wherein K is a hyperparameter for tuning; and transforming the continuous response input variable into a categorical variable based on a clustering result.

9. The method according to claim 1, wherein prediction based on a homology independent Bayesian classifier (CIBer) comprises: optimally dividing the feature variables into multiple subgroups; and considering the feature variables within the same subgroup to be homology, and considering the feature variables in one subgroup to be independent of the feature variables in another subgroup.

10. The method according to claim 9, wherein the optimal partition is determined by a clustering method.

11. A method for prediction based on a homology independent Bayesian classifier (CIBer), comprising: Step 1: Find the combination of the two feature variables with the highest frequency and encode the corresponding feature value as 1; Step 2: Find the combination with the highest frequency among the combinations of uncoded values of the feature variable, and encode the corresponding value with the next unassigned integer; Step 3: Repeat step 2 until the feature variable has only one uncoded value left, then automatically assign it the next unassigned integer; and Step 4: If the condition in step 3 is true for both feature variables, then the same integer is assigned to the values of both feature variables; if not true, and if the remaining feature variables have two or more uncoded values, then their remaining values are coded in descending order of their corresponding marginals.

12. A non-transitory computer-readable medium having program instructions stored therein, wherein the program instructions are executable by a computing system to cause the computing system to perform a method for performing prediction based on a classifier, the method comprising: receiving a multivariate input having feature variables; Making predictions based on a homology independent Bayesian classifier (CIBer) by performing a conditional probability mass function on the feature variables; and Output the predicted category.

13. The method of claim 12, wherein making predictions based on the CIBer comprises joint encoding to artificially insert an ordering among classification features. The method of claim 13 , wherein the joint encoding is configured to preserve inherent dependencies between the classification features.

15. The method of claim 14, wherein maintaining the inherent dependencies between the categorical features comprises obtaining group frequencies of values of the feature variables. The method of claim 13 , wherein the joint encoding is configured to encode classification features to obtain inherent dependencies for classification.

17. The method of claim 12, further comprising performing Laplace smoothing on zero probabilities before outputting the predicted class.

18. The method of claim 12, wherein performing prediction based on a homology independent Bayesian classifier (CIBer) comprises configuring the CIBer to perform regression.

19. The method of claim 18, wherein configuring the CIBer to perform regression comprises: configuring a K-means clustering method to discretize a continuous response input variable into different categories, wherein K is a hyperparameter for tuning; and transforming the continuous response input variable into a categorical variable based on a clustering result.

20. The method according to claim 12, wherein prediction based on a homology independent Bayesian classifier (CIBer) comprises: optimally dividing the feature variables into a plurality of subgroups; and considering the feature variables within the same subgroup to be homology, and considering the feature variables in one subgroup to be independent of the feature variables in another subgroup.

Images