Intuitive fuzzy double support vector machine classification method based on relative density and non-membership degree
By constructing the RD-IFTSVM model, the relative density and non-membership function are used to evaluate the importance of samples, combined with global information and regularization terms, the problems of category imbalance and noise impact in traditional methods are solved, and higher classification accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202510539494.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-08-08
AI Technical Summary
When traditional support vector machines process class imbalance data, they tend to be biased towards most class samples, and noise and outliers affect the classification effect. When calculating non-membership, traditional IFTSVM only relies on local neighborhood information and ignores global distribution, resulting in insufficient classification accuracy.
The RD-IFTSVM model is constructed, the sample importance is evaluated through relative density and non-membership functions, the support vector and noise are distinguished by global information, regularization terms are added to reduce the risk of overfitting, and the double hyperplane discrimination criterion is used for classification.
Effectively distinguishing support vectors from noise, improving the classification performance of a few types of samples, improving the robustness and generalization capabilities of the model, and reducing the misjudgment rate.
Smart Images
Figure CN120448942A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of machine learning and pattern recognition, and in particular relates to an intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership. Background Art
[0002] Classifying class-imbalanced data has always been a challenging problem in machine learning and pattern recognition. Traditional support vector machines (SVMs), designed to maximize the class margin, often favor samples from the majority class, resulting in poor classification performance for samples from the minority class. Furthermore, noise and outliers in the data can severely impact SVM classification performance. To address these issues, researchers have proposed various improvements.
[0003] Fuzzy support vector machines (FSVMs) are robust to noise and outliers by assigning a membership function to each sample. However, their membership function typically relies solely on the distance between the sample and the class center. This limitation of local information makes FSVMs prone to mislabeling support vectors as noise, affecting classification accuracy.
[0004] The Intuitionistic Fuzzy Twin Support Vector Machine (IFTSVM) introduces intuitionistic fuzzy sets and assigns membership and non-membership to each sample, which can better capture the uncertainty in the data. However, traditional IFTSVMs often only consider the local neighborhood relationships of samples when calculating non-membership, ignoring the global distribution of the data. This makes them incapable of distinguishing support vectors from noise.
[0005] To address the class imbalance problem, researchers have conducted extensive research at both the data and algorithm levels. Data-level methods, such as SMOTE, balance the data distribution by oversampling minority class samples, but this can introduce the risk of overfitting. Algorithmic-level methods assign higher weights to minority class samples by adjusting model parameters or loss functions. For example, IFTSVM-ID addresses the class imbalance problem by assigning weights based on the class distribution of samples. However, this method's non-membership function fails to fully utilize global information, and its robustness to noise and outliers needs to be improved. Summary of the Invention
[0006] To solve the above technical problems, the present invention provides an intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership, comprising:
[0007] Build and train the RD-IFTSVM model, and use the trained RD-IFTSVM model to perform data classification;
[0008] Training the RD-IFTSVM model includes:
[0009] S1: Collect sample data sets, extract sample features and perform standardization to obtain sample feature data sets, which are used as training data;
[0010] S2: Define the fuzzy membership function based on relative density and calculate the relative density membership value of the majority class samples;
[0011] S3: Construct a non-membership function based on the relative density membership value of the majority class samples, and calculate the non-membership function value of the majority class samples;
[0012] S4: construct a scoring function based on relative density membership values and non-membership values;
[0013] S5: Calculate the membership values of all training samples according to the scoring function;
[0014] S6: Based on the membership values of all training samples, an optimization problem is constructed to obtain the decision hyperplane in the linear case and the decision boundary in the nonlinear case;
[0015] S7: Determine the label of the sample based on the decision hyperplane in the linear case and the decision boundary in the nonlinear case.
[0016] Beneficial effects of the present invention:
[0017] The concept of relative density is introduced: the k-nearest neighbor probability density estimation technology is used to evaluate the importance of each training sample and assign higher weights to samples in high-density areas. A relative density membership function is proposed, which combines the ratio of heterogeneous samples in the local neighborhood to samples with the same density globally, avoiding misjudgment caused by traditional methods that rely only on the local neighborhood, printing noise and retaining valid samples.
[0018] New non-membership function: Incorporating global information, a new non-membership function is defined to effectively distinguish support vectors from noise and outliers, thus avoiding misclassification.
[0019] Membership function for imbalanced data: assign higher membership values to minority class samples, and adjust the weights of majority class samples according to the imbalance ratio to solve the class imbalance problem.
[0020] Regularization term: A regularization term is added to the objective function to reduce the risk of overfitting and improve the generalization ability of the model. Optimization problems are constructed separately for positive and negative classes, and independent regularization parameters and slack variables are introduced. Through membership weighting, key protection of core samples is achieved, and robustness to noise samples is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 The present invention is a flowchart of an intuitive fuzzy dual support vector machine classification method based on relative density and non-membership. DETAILED DESCRIPTION
[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0023] An intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership, such as Figure 1 Shown, including:
[0024] Build and train the RD-IFTSVM model, and use the trained RD-IFTSVM model to perform data classification;
[0025] Training the RD-IFTSVM model includes:
[0026] S1: Collect sample data sets, extract sample features and perform standardization to obtain sample feature data sets, which are used as training data;
[0027] S2: Define the fuzzy membership function based on relative density and calculate the relative density membership value of the majority class samples;
[0028] S3: Construct a non-membership function based on the relative density membership value of the majority class samples, and calculate the non-membership function value of the majority class samples;
[0029] S4: construct a scoring function based on relative density membership values and non-membership values;
[0030] S5: Calculate the membership values of all training samples according to the scoring function;
[0031] S6: Based on the membership values of all training samples, an optimization problem is constructed to obtain the decision hyperplane in the linear case and the decision boundary in the nonlinear case;
[0032] S7: Determine the label of the sample based on the decision hyperplane in the linear case and the decision boundary in the nonlinear case.
[0033] This paper proposes a novel relative density and non-membership-based intuitionistic fuzzy dual support vector machine (RD-IFTSVM) method for class imbalance learning. This method uses nearest neighbor-based probability density estimation to assess the importance of each training pattern. A novel non-membership-based intuitionistic scoring function is used to distinguish noise, outliers, and support vectors in the majority class training pattern. The majority class training pattern is assigned a score function that reflects the imbalance ratio, while the minority class training pattern is assigned a higher membership of 1.
[0034] Weight distribution of training data points:
[0035] 1. Fuzzy membership function based on relative density:
[0036] In high-dimensional space, determining the exact probability density of each training sample is both challenging and time-consuming. Therefore, the present invention focuses on approximating the proportional relationship between the probability density of any two training samples, expressed as relative density. The relative density is estimated using a k-nearest neighbor-based probability density estimation technique. For each training sample X i , identify its kth nearest neighbor of the same type and calculate the distance between them, recorded as Obviously, the distance The larger the sample X is, the i The smaller the density is. Since noise and outliers mostly exist in low-density areas, Evaluate the importance of each training sample. Convert to reciprocal That is X i The relative density of , assigning higher values to samples in high-density areas and lower values to samples in low-density areas. The relative density of the i-th sample relative to the j-th sample is:
[0037]
[0038] set up where N s For X i The total number of samples in the class to which it belongs. A fuzzy membership function based on relative density is now defined, assigning higher membership values to samples in high-density areas. To eliminate the influence of data distribution, a standardized fuzzy membership function is given:
[0039]
[0040] Among them, χ(x i ) represents the sample x i The relative density membership value of Represents sample x i The distance between the kth nearest neighbor sample of the same type, N s Represents sample x i The total number of samples of the class to which , Represents sample x j The distance between the kth nearest neighbor sample of the same type.
[0041] 2. Proposed non-membership function: To define the non-membership function, the following terms are first described:
[0042] (1) Positive class (C + ) and negative class (C - )Class center of the training point:
[0043]
[0044] in, (positive training set), (Negative training set).
[0045] (2) The distance from the sample to the category center:
[0046]
[0047] Among them, ψ(x) is the image of sample x mapped to the high-dimensional feature space through the kernel function.
[0048] (3) Positive (R + ) and negative (R - ) class radius:
[0049]
[0050] Now, consider the following two sets of corresponding positive samples:
[0051]
[0052] Negative and positive samples A set of samples whose distance in the feature space is less than or equal to δ.
[0053] Positive category and (Similar) A set of samples (i≠j) whose distance difference to the class center in the feature space is less than or equal to δ.
[0054] Among them, δ>0 is a threshold. Next, define the ratio as follows:
[0055]
[0056] The distribution of samples is characterized by ratios, and |·| represents the cardinality (number of elements) of a set. Similarly, for negative samples:
[0057]
[0058] Positive and negative samples A set of samples whose distance in feature space is less than δ.
[0059] Negative class and The set of samples whose distance from the negative class center is less than δ.
[0060] Define the ratio as follows:
[0061]
[0062] Based on the membership and the above two ratios, a new non-membership function is defined for each mode as follows:
[0063]
[0064] Among them, v(x i ) represents the sample x i The non-membership value of X(x i ) represents the sample x i The relative density membership value of represents the ratio of positive samples, Represents the ratio of negative samples, y i Represents sample x i The category label of .
[0065] 3. Scoring function: After assigning membership and non-membership to each training pattern, the scoring function is calculated using the following expression:
[0066]
[0067] This score function helps distinguish support vectors, noise, and outliers by utilizing membership and non-membership. Patterns surrounded by similar classes with high membership are considered significant patterns, while those close to different classes or far away from their class center are considered noise.
[0068] 4. Proposed membership function for unbalanced data:
[0069]
[0070] Among them, μ(x i ) represents the sample x i The membership value of s(x i ) represents the sample x i The scoring function is , and IR represents the class imbalance ratio.
[0071] Proposed RD-IFTSVM:
[0072] Linear case:
[0073] To classify linearly separable datasets, this method determines two optimal hyperplanes by solving the following optimization problem: and
[0074]
[0075] Among them, C i>0 (i=1,2,3,4) are penalty and regularization parameters, e1,e2 are dimensionally adapted all-one column vectors, ξ1,ξ2 are slack variables associated with class +1 and class -1, μ1,μ2 are column vectors of membership values set for the training patterns of class +1 and class -1, respectively. The first term of problem (13) aims to minimize the sum of the squared distances between the training patterns and their corresponding hyperplanes; the second term of the objective function acts as a regularization term, and the third term minimizes the weighted misclassification error. These constraints ensure that the minimum distance between the training patterns of the opposite class and the optimal hyperplane is unity. The Lagrangian function corresponding to the quadratic programming problem (13) (where λ1λ2 are positive Lagrangian multipliers) is expressed as:
[0076]
[0077] In addition, the KKT conditions are as follows:
[0078]
[0079] Combining (16), (17) and (18), we get formula (20:
[0080]
[0081] Assumptions:
[0082] G=[T1e1],H=[T2e2],w1=[u1b1] T ,
[0083] Formula (20) can be transformed into:
[0084] G t Gw1+C1w1+H t λ1=0
[0085] in:
[0086] w1=-(G t G+C1I) -1 H t λ1#(21)
[0087] It should be noted that since the matrix G t G is a semi-positive matrix and C1>0, so the matrix G t G+C1I is always reversible.
[0088] Similarly, for QPP (14) with β1, β2 as positive Lagrange multipliers, the following relationship can be obtained:
[0089] w2=(H t H+C3I) -1 G t β1#(22)
[0090] Where w2 = [u2b2] T .
[0091] Furthermore, the Wolff dual
[16] of the original problems (13) and (14) can be calculated as follows:
[0092]
[0093] Solve problems (23) and (24) to obtain vectors λ1 and β1. Then calculate w1 and w2 using equations (21) and (22) to determine the optimal hyperplane. and Finally, the new data point x is classified as positive or negative according to the function.
[0094]
[0095] Nonlinear case:
[0096] For nonlinearly separable training data, the kernel technique is used to map the data into a high-dimensional space. Then, the nonlinear decision boundary is determined by solving the following optimization problem: and (in, is the kernel function, ).
[0097]
[0098] Here, the definitions of all symbols are the same as those in the linear case. Following similar steps as in the linear case, we can obtain the nonlinear hyperplane:
[0099] q1=-(E t E+C1I) -1 F t λ1,(28)
[0100] q2=(F t F+C3I) -1 E t β1(29)
[0101] in, q1=[v1b1] t ,q2=[v2b2] t For new test data points The classification method is:
[0102]
[0103]
[0104]
[0105] Innovations of the proposed non-membership function:
[0106] (1) Intuitionistic fuzzy membership function based on global density:
[0107] Relative density membership function Combining heterogeneous samples in local neighborhoods Samples with the same density as the global This method can avoid misjudgment caused by traditional methods that rely only on local neighborhoods. It can also print noise and retain valid samples.
[0108] (2) Model innovation:
[0109] Optimization problems are constructed for positive and negative classes respectively, and independent regularization parameters (C1 / C3) and slack variables (ξ1 / ξ2) are introduced. Through membership weighting (μ1 / μ2), key protection of core samples is achieved and the robustness to noise samples is improved.
[0110] (3) Decision function:
[0111] A dual hyperplane discrimination criterion based on geometric distance is proposed, and the sample attribution is determined by the minimum distance to the positive and negative kernel hyperplanes.
[0112] In this embodiment, the method of the present invention was experimentally verified, and the experimental results are as follows:
[0113] The experiments were conducted in Python, and the results were calculated using the 10-fold cross-validation method. In this method, the dataset is divided into ten equal parts, of which nine folds are used for training and the remaining one fold is used for testing. This process is repeated ten times, and the average of the results is recorded. In addition, a grid search technique is used to determine the optimal parameters. The penalty parameters in all dual models are configured as C1 = C3 and C2 = C4 to simplify the calculation process and are selected from the range {10 -2 ,10 -1 ,...,10 3}. To classify nonlinear data, a Gaussian kernel (K(x1,x2)=exp{-σ||x1-x2|| 2}), the parameter σ is from the set {2 -2 ,2 -1 ,...,2 5 To analyze the behavior of the proposed method, the following metrics are used: AUC, F-value, G-mean, sensitivity, and specificity, which are defined as follows:
[0114]
[0115] where tp, tn, fp, and fn represent the number of true positive, true negative, false positive, and false negative samples, respectively.
[0116] Experiments on the UCI dataset and KEEL dataset:
[0117] The effectiveness of RD-IFTSVM was evaluated by comparing its performance with several state-of-the-art techniques on 13 imbalanced datasets. The calculated AUC values and standard deviations (SD) are given in Tables 1 and 2 for linear and nonlinear kernels, respectively. The highest AUC values for each dataset among all classifiers are highlighted in bold. The proposed RD-IFTSVM exhibits superior generalization performance on most datasets, achieving an average AUC of 91.65% across all benchmark datasets, surpassing the performance of all existing classifiers in the linear case. This is followed by CGFTSVM (average AUC of 89.63%), IFTSVM-ID (88.31%), IFTPSVM (87.52%), and TSVM (84.33%).
[0118] In the case of nonlinear kernels, the proposed RD-IFTSVM again shows the best performance, achieving an average AUC of 91.29%, followed by CGFTSVM with 89.76%, IFTPSVM, IFTSVM-ID, and TSVM with 86.92%, 87.54%, and 84.94%, respectively.
[0119] Table 1: AUC values and standard deviations (SD) of the proposed model and the baseline model on the benchmark dataset using the linear kernel
[0120]
[0121] Table 2: AUC values and standard deviations (SD) of the proposed model and the baseline model on the benchmark dataset using Gaussian kernel
[0122]
[0123] To thoroughly evaluate the performance of the classifiers, each model was ranked based on its AUC value. The best-performing model on each dataset received the lowest rank, and vice versa. The proposed RD-IFTSVM achieved the lowest average rank of 1.00 and 1.00 for the linear and Gaussian kernels, respectively. The results clearly demonstrate that the proposed RD-IFTSVM consistently outperforms existing methods.
[0124] Furthermore, experiments were conducted on contaminated datasets to evaluate the performance of the proposed RD-IFTSVM compared to baseline models. Noise and outliers were introduced into five randomly selected benchmark datasets by perturbing the feature columns while keeping the label columns intact. Gaussian noise was added to a small fraction of the feature values, with the proportion of affected entries being 10%, 20%, and 30% of the dataset, respectively. Furthermore, outliers were added to the datasets by replacing randomly selected feature values at proportions of 10%, 20%, and 30%. The results of these experiments are listed in Tables 3 and 4 for the linear and Gaussian kernels, respectively. The proposed RD-IFTSVM demonstrates superior performance in the presence of noise and outliers. The observed performance improvement stems from the method's ability to effectively distinguish between support vectors and outliers, thereby ensuring appropriate weight distribution.
[0125] The impact of noise and outliers:
[0126] Table 3: Impact of noise and outliers on benchmark datasets when using linear kernels
[0127]
[0128] Table 4: Impact of noise and outliers on benchmark datasets when using Gaussian kernel
[0129]
[0130] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.
Claims
1. An intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership, characterized in that: include: Build and train the RD-IFTSVM model, and use the trained RD-IFTSVM model to perform data classification; Training the RD-IFTSVM model includes: S1: Collect sample data sets, extract sample features and perform standardization to obtain sample feature data sets, which are used as training data; S2: Define the fuzzy membership function based on relative density and calculate the relative density membership value of the majority class samples; S3: Construct a non-membership function based on the relative density membership value of the majority class samples, and calculate the non-membership function value of the majority class samples; S4: construct a scoring function based on relative density membership values and non-membership values; S5: Calculate the membership values of all training samples according to the scoring function; S6: Based on the membership values of all training samples, an optimization problem is constructed to obtain the decision hyperplane in the linear case and the decision boundary in the nonlinear case; S7: Determine the label of the sample based on the decision hyperplane in the linear case and the decision boundary in the nonlinear case.
2. The intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership according to claim 1 is characterized in that: Define the fuzzy membership function based on relative density and calculate the relative density membership value of the majority class samples, including: Among them, χ(x i ) represents the sample x i The relative density membership value of Represents sample x i The distance between the kth nearest neighbor sample of the same type, N s Represents sample x i The total number of samples of the class to which , Represents sample x j The distance between the kth nearest neighbor sample of the same type.
3. The intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership according to claim 1, characterized in that: Construct a non-membership function based on the relative density membership value of the majority class samples, and calculate the non-membership value of the majority class samples, including: Among them, ν(x i ) represents the sample x i The non-membership value of χ(x i ) represents the sample x i The relative density membership value of represents the ratio of positive samples, Represents the ratio of negative samples, y i Represents sample x i The category label of .
4. The intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership according to claim 1 is characterized in that: The scoring function is constructed based on the relative density membership value and non-membership value, including: Among them, s(x i ) represents the sample x i The scoring function, v(x i ) represents the sample x i The non-membership value of χ(x i ) represents the sample x i The relative density membership value of .
5. The intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership according to claim 1 is characterized in that: Calculate the membership values of all training samples according to the scoring function, including: Among them, μ(x i ) represents the sample x i The membership value of s(x i ) represents the sample x i The scoring function is , and IR represents the class imbalance ratio.
6. The intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership according to claim 1, characterized in that: Based on the membership values of all training samples, an optimization problem is constructed to obtain the decision hyperplane in the linear case and the decision boundary in the nonlinear case, including: Linear optimization problem: Among them, u1 and u2 represent the hyperplane normal vectors set by the +1 and -1 training modes respectively, b1 and b2 represent the hyperplane bias terms of the +1 and -1 classes respectively, ξ1 and ξ2 represent the slack variables related to the +1 and -1 classes respectively, T1 and T2 represent the matrices of the +1 and -1 training sample data respectively, e1 and e2 represent the first and second all-one column vectors of dimension adaptation respectively, C i >0(i=1,2,3,4) represents the penalty and regularization parameters, Represents the transpose of the column vector of membership values set for the +1 and -1 training modes respectively; Optimal hyperplane for linear case: Nonlinear optimization problem: Among them, v1 and v2 represent the coefficient column vectors of the nonlinear hyperplane of +1 class and -1 class respectively. represents the kernel function, T t represents the transposed matrix of the training data matrix T, Respectively represent the transpose of the coefficient column vectors v1 and v2 of the nonlinear hyperplane; Decision boundary for nonlinear case:
7. The intuitionistic fuzzy dual support vector machine classification method based on relative density and non-membership according to claim 1 is characterized in that: The labels of the samples are determined based on the decision hyperplane in the linear case and the decision boundary in the nonlinear case, including: Linear case: Nonlinear case: Among them, y(x) represents the label of sample x, argmin represents the variable value function that obtains the minimum value, Represents the hyperplane normal vector u i The transpose of b i represents the bias term of the hyperplane, represents the kernel function, T t represents the transpose of the training data matrix T, The coefficient column vector v representing the nonlinear hyperplane i The transpose of .