Multi-category feature selection method for independent inspection of category specific conditions

Abstract

The invention discloses a multi-category feature selection method for independent inspection of category specific conditions. The method comprises the following steps: step 1, data preprocessing and category division; 2, class specific condition independence testing is carried out; step 3, carrying out class specific Markov Blanket discovery, and carrying out class specific Markov Blanket discovery; step 4, carrying out Markov Blanket structure optimization, and carrying out Markov Blanket structure optimization; step 5, carrying out cross integration on the class specific features; 6, training and verifying the model; according to the method, a class-specific condition independence test method is adopted, a sample subset is independently constructed for each class, and the condition dependence intensity is calculated, so that statistical deviation caused by inter-class data distribution difference is effectively avoided. By accurately capturing a correlation structure only existing in a specific category, a condition independence test result is more fit with a real structure of data, and the resolution is remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and data mining technology, specifically to a multi-class feature selection method for class-specific conditional independence testing. Background Technology

[0002] In machine learning and data mining systems, feature selection is a crucial preprocessing step for improving model performance, reducing computational complexity, and eliminating feature redundancy. Its core objective is to select a subset of features with strong discriminative power for the target variable from a high-dimensional candidate feature set, eliminating irrelevant and redundant features. Currently, mainstream feature selection algorithms are mainly divided into three categories:

[0003] Filtering feature selection methods: These methods select features based on the statistical correlation between features and target variables. They do not rely on specific classifiers and are characterized by high computational efficiency and strong generalization. A typical example is the mutual information-based method, which selects features by ranking them according to the mutual information values ​​between features and target variables.

[0004] Wrap-up feature selection method: Binds feature selection to classifier performance. It traverses feature subsets through search algorithms (such as genetic algorithms and greedy algorithms) and selects the optimal subset based on classifier performance. Although the screening effect is good, the computational complexity is high and it is suitable for low-dimensional data scenarios.

[0005] Feature selection methods based on causal discovery: These methods are based on Markov Blanket (MB) theory and use conditional independence tests to discover feature sets that have a direct causal relationship with the target variable. They have strong theoretical basis and interpretability. Typical algorithms include IAMB, Fast-IAMB, and Inter-AMB.

[0006] However, most existing feature selection algorithms default to using the same set of statistical tests and selection strategies for all categories, i.e., a uniform processing mode across categories. This mode ignores the inherent differences between different categories in multi-class classification tasks:

[0007] The distribution of feature values ​​may differ significantly for different categories. For example, in image classification tasks, the distribution of features such as color and shape of target objects of different categories is fundamentally different.

[0008] The same feature has different effects on different categories of target variables, and some features may only be discriminative for a specific category and have no significant effect on other categories.

[0009] In imbalanced tasks, the feature distribution of minority class samples is easily masked by the majority class, and a uniform processing mode will cause minority class features to be ignored.

[0010] The aforementioned differences lead to the following drawbacks in feature selection methods that unify processing across categories:

[0011] It cannot identify correlation structures that exist only within a specific category. The selected feature subset may contain redundant features that are irrelevant to some categories, or omit discriminative features that are key to a specific category.

[0012] Differences in data distribution between classes can interfere with the results of conditional independence tests, leading to a decrease in the reliability and resolution of the tests, and consequently affecting the accuracy of feature selection.

[0013] The selected feature subset cannot meet the discrimination requirements of different categories, especially in scenarios with imbalanced categories and multimodal high-dimensional data, where the generalization performance of the classification model is greatly reduced.

[0014] Uniformly selected features cannot reflect the specific relationship between different categories and features, making it difficult to explain the model's classification logic for different categories. Summary of the Invention

[0015] The purpose of this invention is to provide a multi-class feature selection method for independent testing under specific conditions, in order to solve the problems mentioned in the background art.

[0016] To achieve the above objectives, the present invention provides the following technical solution: a multi-class feature selection method for class-specific conditional independent testing, comprising the following steps:

[0017] Step 1: Data preprocessing and classification. Obtain the original dataset containing the target variable Y and the feature set F, and classify the data according to the values ​​of the target variable Y {y1, y2, ..., y3}. n The original dataset is divided into Y in-class sample subsets D. y1 D y2 D yn , where each class has a subset of samples D y It only includes samples where the target variable has a value of y;

[0018] Step 2: Class-specific conditional independence test, for each class-specific subset D of samples. y For any two features X in the feature set F i With X j Given a condition set S, calculate the class-specific conditional dependency strength, which is obtained through conditional mutual information and G. 2 This can be achieved through statistics or other measures of independence.

[0019] Step 3: Class-specific Markov Blanket discovery. Based on the conditional dependency strength results from Step 2, an initial Markov Blanket is discovered for each class y through an iterative process of forward selection, candidate set pruning, and reverse elimination.

[0020] Step 4: Markov Blanket structure optimization. Based on the initial Markov Blanket obtained in Step 3, the structure of the initial Markov Blanket is optimized through feature priority sorting and missing feature recovery strategies to obtain the optimized Markov Blanket for each category.

[0021] Step 5: Cross-integration of class-specific features. The feature contribution of the optimized Markov Blanket for all classes is evaluated. Redundant features are removed by cross-integration strategy, and class-specific and common features are retained to generate the final feature subset.

[0022] Step 6: Model training and validation. Input the final feature subset obtained in Step 5 into the classifier to train the model and validate its performance, thus completing the multi-class feature selection and classification task.

[0023] Preferably, step 1 specifically includes:

[0024] 1.1 Raw Data Acquisition: Collect datasets corresponding to multiple categories of tasks. The datasets include the target variable Y and the feature set F = {X1, X2, ..., X...} m}, where the target variable Y is a discrete categorical variable, and its value set is {y1, y2, ..., y}. n}, n≥2, the feature set F contains m candidate features, m≥1;

[0025] 1.2 Data cleaning: Remove missing value samples and outlier samples from the dataset. The outlier samples are identified by the 3σ criterion or box plot method. Numerical features are normalized for range, and discrete features are encoded using one-hot encoding.

[0026] 1.3 Classification: Based on the value of the target variable Y, the cleaned dataset is divided into n non-overlapping within-class sample subsets D. y1 D y2 D yn ,satisfy and i≠j, the sample subset D within each class y The target variable for all samples is y.

[0027] Preferably, in step 2, based on G 2 The specific method for calculating the class-specific conditional dependency strength of a statistic is as follows: for a subset D of samples within a class... y Any two features X in i With X j Given condition sets S and G 2 The formula for calculating the statistic is:

[0028]

[0029] Where, n y (x i ,x j ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i Feature X j The value is x j The frequency of samples when the condition set S takes the value s; n y (s) represents the subset of samples D within the class. y In the given condition set S, the sample frequency is the value of s; n y (x i ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i The frequency of samples when the condition set S takes the value s; n y (x j ,s) represents the subset of samples D within the class. y In the middle, feature X j The value is x j The sample frequency when the condition set S takes the value s;

[0030] The larger the value of X, the higher the feature X is within class y. i With X j Given a set of conditions S, the less independent the conditions are, the stronger the conditional dependency is.

[0031] The smaller the value of X, the higher the feature X is within class y. i With X j Given a condition set S, the closer the condition is to independence, the lower the strength of the conditional dependency.

[0032] Preferably, the method for calculating class-specific conditional dependency strength based on conditional mutual information in step 2 is as follows:

[0033] For the in-class sample subset D y Any two features X in i With X j Given a condition set S and conditional mutual information I y (X i ,X j The formula for calculating |S) is:

[0034]

[0035] Among them, P y (xi ,x j ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i Feature X j The value is x j The joint probability when the condition set S takes the value s. N y For the in-class sample subset D y Total number of samples; P y (x i ,x j |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X i The value is x i And feature X j The value is x j The conditional probability of P; y (x i |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X i The value is x i The conditional probability of P; y (x j |S) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X j The value is x j The conditional probability;

[0036] Conditional Mutual Information I y (X i ,X j The larger the value of |S), the stronger the feature X within class y. i With X j Given a condition set S, the stronger the condition dependency, the higher the strength; when I y (X i ,X j When |S)=0, it indicates that within class y, feature X i With X j They are independent of each other given a set of conditions S.

[0037] Preferably, step 3 specifically includes:

[0038] 3.1 Initialization parameters: For each category y, initialize the candidate feature set CanF = F.

[0039] 3.2 Forward Selection Stage: In the candidate feature set CanF, select the feature that makes the dependency dep(F) equal to the desired value.i Fbest is the largest feature of (Y|CurMB), and the dependency is obtained by the class-specific conditional dependency strength calculation method in step 2; if Fbest and Y have a significant conditional dependency given CurMB, and the dependency is greater than the preset threshold θ1, θ1∈(0,1), then Fbest is added to CurMB and removed from CanF.

[0040] 3.3 Candidate Set Pruning Stage: For each feature F in the candidate feature set CanF i If F i If F is independent of Y under given CurMB, and its dependency is less than or equal to a preset threshold θ0, where θ0∈(0,θ1), then F... i Remove from CanF;

[0041] 3.4 Reverse Elimination Phase: In the current CurMB, select the option that makes the dependency dep(F) equal to the value of F. j ,Y|CurMB{F j The smallest feature Fworst; if Fworst and Y are independent of each other given CurMB{Fworst} and their dependence is less than or equal to θ0, then Fworst is removed from CurMB;

[0042] 3.5 Iteration Termination: Repeat steps 3.2-3.4 until CurMB no longer changes, obtaining the initial Markov BlanketCurMB for each category y. y .

[0043] Preferably, step 4 specifically includes:

[0044] 4.1 Feature Priority Ranking: For each category y, the initial Markov Blanket CurMB y The features are sorted in descending order based on the class-specific conditional dependency strength between the features and the target variable Y, resulting in a sorted feature list.

[0045] 4.2 Filtering high-priority features: Set a percentage threshold k, k∈(0,100], and select the top k% of features from the sorted feature list to form a feature queue Q. y ;

[0046] 4.3 Missing Feature Recovery: For feature queue Q y Each feature Q in y [i], calculate the extended MarkovBlanketCurMB corresponding to this feature. y Q y [i] = ESMB(Q) y[i],F), where ESMB is the extended Markov Blanket search function used to search for the expression Q. y [i] There exists a feature set with conditional dependencies; if the target variable Y belongs to CurMB y (Q y If [i]), then the associated feature corresponding to that feature is added to the current Markov Blanket;

[0047] 4.4 Generating the Optimized Markov Blanket: After completing the missing feature recovery for all features, the optimized Markov Blanket CurMB for each category y is obtained. y .

[0048] Preferably, step 5 specifically includes:

[0049] 5.1 Feature Contribution Evaluation: For each optimized Markov BlanketCurMB y Calculate the in-class contribution C of feature f in the class. y (f) = dep(f, Y|MB) y {f}), that is, the conditional dependence of feature f on target variable Y within category y; at the same time, the global contribution of feature f is calculated. Where ω y The weight coefficients for category y are... N y The number of samples for category y;

[0050] 5.2 Redundant Feature Removal: For any two features f1 and f2, if there exists a category y such that... γ is the redundancy judgment threshold. If γ > 0 and the global contribution C(f1) ≤ C(f2), then feature f1 is removed.

[0051] 5.3 Final Feature Subset Generation: Features with global contribution C(f) ≥ δ, where δ is the feature selection threshold, and δ > 0 are retained to form the final feature subset F. * .

[0052] Preferably, the preset thresholds θ0 and θ1 are determined by the following method: based on the intra-class sample subset D y Sample size N y θ0 = 0.01 × log(N) y ) / N y θ1 = 0.05 × log(N) y ) / N y This ensures that the significance of the dependency assessment is appropriate for the sample size.

[0053] Preferably, the percentage threshold k ranges from 30% to 80%, and the optimal value is determined through cross-validation. During the cross-validation process, the k value that gives the classifier the highest accuracy on the validation set is selected.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] This invention employs a class-specific conditional independence test method, constructing a separate sample subset for each class and calculating the conditional dependency strength, effectively avoiding statistical bias caused by differences in data distribution between classes. By accurately capturing the correlation structure that exists only within a specific class, the conditional independence test results more closely match the true data structure, significantly improving resolution.

[0056] The class-specific feature selection mechanism of this invention independently evaluates feature contributions for each class, enabling the identification of features with strong discriminative power for a specific class while retaining common features that are effective across all classes. This approach fully utilizes class difference information in multi-class classification tasks, avoiding the loss of difference signals caused by uniform processing across classes.

[0057] This invention discovers and optimizes class-specific Markov Blankets, combined with a cross-integration strategy to eliminate redundant features, resulting in a final feature subset that is both discriminative and concise. In scenarios involving multi-class classification, imbalanced classes, high-dimensional data, and multimodal data, it effectively improves the generalization ability of various classifiers such as KNN and SVM, and reduces the risk of overfitting.

[0058] This invention can clearly define the contribution of each feature in different categories and clearly reflect the category-specific relationship between features and target variables. Attached Figure Description

[0059] Figure 1 This is a flowchart of the method of the present invention;

[0060] Figure 2 This is a flowchart of the algorithm for obtaining feature subsets in this invention;

[0061] Figure 3 This is a flowchart of the accuracy test for the algorithm of this invention;

[0062] Figure 4 These are the algorithm effect diagrams of this invention; the diagrams correspond to KNN, logistic regression, and SVM, respectively.

[0063] Figure 5 This is a schematic diagram comparing the algorithm performance of KNN under this invention;

[0064] Figure 6 This is a schematic diagram comparing the performance of the logistic regression algorithm of this invention;

[0065] Figure 7 This is a schematic diagram comparing the algorithm performance under SVM in this invention. Detailed Implementation

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0067] Please see Figure 1-7 This invention provides a multi-class feature selection method for class-specific conditional independent testing, comprising the following steps:

[0068] Step 1: Data preprocessing and classification. Obtain the original dataset containing the target variable Y and the feature set F, and classify the data according to the values ​​of the target variable Y {y1, y2, ..., y3}. n The original dataset is divided into Y in-class sample subsets D. y1 D y2 D yn , where each class has a subset of samples D y It only includes samples where the target variable takes the value y; specifically:

[0069] 1.1 Raw Data Acquisition: Collect datasets corresponding to multiple categories of tasks. The datasets include the target variable Y and the feature set F = {X1, X2, ..., X...} m}, where the target variable Y is a discrete categorical variable, and its value set is {y1, y2, ..., y}. n}, n≥2, the feature set F contains m candidate features, m≥1;

[0070] 1.2 Data cleaning: Remove missing value samples and outlier samples from the dataset. The outlier samples are identified by the 3σ criterion or box plot method. Numerical features are normalized for range, and discrete features are encoded using one-hot encoding.

[0071] 1.3 Classification: Based on the value of the target variable Y, the cleaned dataset is divided into n non-overlapping within-class sample subsets D. y1 D y2 D yn ,satisfy and i≠j, the sample subset D within each class y The target variable for all samples is y.

[0072] Step 2: Class-specific conditional independence test, for each class-specific subset D of samples. yFor any two features X in the feature set F i With X j Given a condition set S, calculate the class-specific conditional dependency strength, which is obtained through conditional mutual information and G. 2 This can be achieved through statistics or other measures of independence.

[0073] Based on G 2 The specific method for calculating the class-specific conditional dependency strength of a statistic is as follows: for a subset D of samples within a class... y Any two features X in i With X j Given condition sets S and G 2 The formula for calculating the statistic is:

[0074]

[0075] Where, n y (x i ,x j ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i Feature X j The value is x j The frequency of samples when the condition set S takes the value s; n y (s) represents the subset of samples D within the class. y In the given condition set S, the sample frequency is the value of s; n y (x i ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i The frequency of samples when the condition set S takes the value s; n y (x j ,s) represents the subset of samples D within the class. y In the middle, feature X j The value is x j The sample frequency when the condition set S takes the value s;

[0076] The larger the value of X, the higher the feature X is within class y. i With X j Given a set of conditions S, the less independent the conditions are, the stronger the conditional dependency is.

[0077] The smaller the value of X, the higher the feature X is within class y. i With X j Given a condition set S, the closer the condition is to independence, the lower the strength of the conditional dependency.

[0078] The specific method for calculating class-specific conditional dependency strength based on conditional mutual information is as follows:

[0079] For the in-class sample subset D y Any two features X in i With X j Given a condition set S and conditional mutual information I y (X i ,X j The formula for calculating |S) is:

[0080]

[0081] Among them, P y (x i ,x j ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i Feature X j The value is x j The joint probability when the condition set S takes the value s. N y For the in-class sample subset D y Total number of samples; P y (x i ,x j |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X i The value is x i And feature X j The value is x j The conditional probability of P; y (x i |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X i The value is x i The conditional probability of P; y (x j |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X j The value is x j The conditional probability;

[0082] Conditional Mutual Information I y (X i ,X j The larger the value of |S), the stronger the feature X within class y. i With X j Given a condition set S, the stronger the condition dependency, the higher the strength; when I y (X i ,Xj When |S)=0, it indicates that within class y, feature X i With X j They are independent given a condition set S;

[0083] Step 3: Class-Specific Markov Blanket Discovery. Based on the conditional dependency strength results from Step 2, an initial Markov Blanket is discovered for each class y through an iterative process of forward selection, candidate set pruning, and backward elimination. Specifically, this includes:

[0084] 3.1 Initialization parameters: For each category y, initialize the candidate feature set CanF = F.

[0085] 3.2 Forward Selection Stage: In the candidate feature set CanF, select the feature that makes the dependency dep(F) equal to the desired value. i Fbest is the largest feature of (Y|CurMB), and the dependency is obtained by the class-specific conditional dependency strength calculation method in step 2; if Fbest and Y have a significant conditional dependency given CurMB, and the dependency is greater than the preset threshold θ1, θ1∈(0,1), then Fbest is added to CurMB and removed from CanF.

[0086] 3.3 Candidate Set Pruning Stage: For each feature F in the candidate feature set CanF i If F i If F is independent of Y under given CurMB, and its dependency is less than or equal to a preset threshold θ0, where θ0∈(0,θ1), then F... i Remove from CanF;

[0087] 3.4 Reverse Elimination Phase: In the current CurMB, select the option that makes the dependency dep(F) equal to the value of F. j ,Y|CurMB{F j The smallest feature Fworst; if Fworst and Y are independent of each other given CurMB{Fworst} and their dependence is less than or equal to θ0, then Fworst is removed from CurMB;

[0088] 3.5 Iteration Termination: Repeat steps 3.2-3.4 until CurMB no longer changes, that is, the feature set of CurMB is completely consistent in two consecutive iterations. At this point, CurMB converges, and the initial Markov BlanketCurMB corresponding to each category y is obtained. y ;

[0089] The preset thresholds θ0 and θ1 are determined by the following method: based on the intra-class sample subset D ySample size N y θ0 = 0.01 × log(N) y ) / N y θ1 = 0.05 × log(N) y ) / N y To ensure that the significance of the dependency assessment is appropriate for the sample size;

[0090] The core pseudocode for this step is as follows:

[0091]

[0092]

[0093] Step 4: Markov Blanket Structure Optimization. Based on the initial Markov Blanket obtained in Step 3, the structure of the initial Markov Blanket is optimized using feature priority ranking and missing feature recovery strategies to obtain the optimized Markov Blanket for each category; specifically including:

[0094] 4.1 Feature Priority Ranking: For each category y, the initial Markov Blanket CurMB y Calculate the class-specific conditional dependency strength C between each feature f and the target variable Y. y (f) = dep(f, Y|MB) y {f}), and according to C y (f) Sort the features in descending order to obtain the sorted feature list L. y =[f1,f2,…,f k Among them, f1 has the strongest dependency strength, f k The dependence strength is the smallest

[0095] 4.2 Filtering high-priority features: Set a percentage threshold k, k∈(0,100], and select the top k% of features from the sorted feature list to form a feature queue Q. y The percentage threshold k ranges from 30% to 80%. The optimal value is determined through cross-validation. During cross-validation, the k value that gives the classifier the highest accuracy on the validation set is selected.

[0096] 4.3 Missing Feature Recovery: Define the extended Markov Blanket search function ESMB(q,F) to search for all feature sets that have conditional dependencies on feature q. The search process is based on the class-specific conditional dependency strength matrix M obtained in step 2. y For the feature queue Q y Each feature Q in y [i], calculate the extended MarkovBlanketCurMB corresponding to this feature.y Q y [i] = ESMB(Q) y [i],F), where ESMB is the extended Markov Blanket search function used to search for the expression Q. y [i] There exists a feature set with conditional dependencies; if the target variable Y belongs to CurMB y (Q y If [i]), then the associated feature corresponding to that feature is added to the current MarkovBlanket;

[0097] 4.4 Generating the Optimized Markov Blanket: After completing the missing feature recovery for all features, the optimized Markov Blanket CurMB for each category y is obtained. y ;

[0098] The core pseudocode for this step is as follows:

[0099]

[0100]

[0101] Step 5: Class-specific feature cross-integration. The feature contribution of the optimized Markov Blanket for all classes is evaluated. Redundant features are removed using a cross-integration strategy, retaining class-specific and common features to generate the final feature subset. Specifically, this includes:

[0102] 5.1 Feature Contribution Evaluation: For each optimized Markov BlanketCurMB y Calculate the in-class contribution C of feature f in the class. y (f) = dep(f, Y|MB) y {f}), that is, the conditional dependence of feature f on target variable Y within category y; at the same time, the global contribution of feature f is calculated. Where ω y The weight coefficients for category y are... N y The number of samples for category y reflects the comprehensive discriminative power of feature f across all categories;

[0103] 5.2 Redundant Feature Removal: For any two features f1 and f2, iterate through all categories y and calculate the class-specific conditional dependency strength. Set a redundancy detection threshold Where mean{·} represents the class-specific G of all feature pairs in the unconditional case. 2 Mean of the statistic;

[0104] If there exists a category y such that This indicates that f1 and f2 have significant redundancy in this category. γ is the redundancy judgment threshold. If γ > 0 and the global contribution C(f1) ≤ C(f2), then feature f1 is removed and feature f2 with higher global contribution is retained.

[0105] 5.3 Final Feature Subset Generation: Features with global contribution C(f) ≥ δ, where δ is the feature selection threshold, and δ > 0 are retained to form the final feature subset F. * ;

[0106] Step 6: Model Training and Validation

[0107] 6.1 Dataset partitioning: The original dataset is divided into training and test sets in a 7:3 ratio, while maintaining the same sample ratio for each category during the partitioning process (stratified sampling);

[0108] 6.2 Feature Subset Replacement: Replace the feature sets of the training and test sets with the final feature subset F obtained in step 5. * We obtain the training set T and the test set Te;

[0109] 6.3 Model Training: Select a classifier (such as KNN, SVM, logistic regression, etc.), train the model based on the training set T, and optimize the model parameters.

[0110] 6.4 Performance Validation: Input the test set Te into the trained model, calculate the model's classification accuracy, precision, recall, F1 score and other performance metrics to verify the effectiveness of the feature selection method.

[0111] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for selecting multi-class features under class-specific conditional independence testing, characterized in that: Includes the following steps: Step 1: Data preprocessing and classification. Obtain the original dataset containing the target variable Y and the feature set F, and classify the data according to the values ​​of the target variable Y {y1, y2, ..., y3}. n The original dataset is divided into Y in-class sample subsets D. y1 D y2 D yn , where each class has a subset of samples D y It only includes samples where the target variable has a value of y; Step 2: Class-specific conditional independence test, for each class-specific subset D of samples. y For any two features X in the feature set F i With X j Given a condition set S, calculate the class-specific conditional dependency strength, which is obtained through conditional mutual information and G. 2 This can be achieved through statistics or other measures of independence. Step 3: Class-specific Markov Blanket discovery. Based on the conditional dependency strength results from Step 2, an initial Markov Blanket is discovered for each class y through an iterative process of forward selection, candidate set pruning, and reverse elimination. Step 4: Markov Blanket structure optimization. Based on the initial Markov Blanket obtained in Step 3, the structure of the initial Markov Blanket is optimized through feature priority sorting and missing feature recovery strategies to obtain the optimized Markov Blanket for each category. Step 5: Cross-integration of class-specific features. The feature contribution of the optimized Markov Blanket for all classes is evaluated. Redundant features are removed by cross-integration strategy, and class-specific and common features are retained to generate the final feature subset. Step 6: Model training and validation. Input the final feature subset obtained in Step 5 into the classifier to train the model and validate its performance, thus completing the multi-class feature selection and classification task.

2. The multi-class feature selection method for class-specific conditional independence testing according to claim 1, characterized in that: Step 1 specifically includes: 1.1 Raw Data Acquisition: Collect datasets corresponding to multiple categories of tasks. The datasets include the target variable Y and the feature set F = {X1, X2, ..., X...} m }, where the target variable Y is a discrete categorical variable, and its value set is {y1, y2, ..., y3}. n }, n≥2, the feature set F contains m candidate features, m≥1; 1.2 Data cleaning: Remove missing value samples and outlier samples from the dataset. The outlier samples are identified by the 3σ criterion or box plot method. Numerical features are normalized for range, and discrete features are encoded using one-hot encoding. 1.3 Classification: Based on the value of the target variable Y, the cleaned dataset is divided into n non-overlapping within-class sample subsets D. y1 D y2 D yn ,satisfy and Each class's sample subset D y The target variable for all samples is y.

3. The multi-class feature selection method for class-specific conditional independence testing according to claim 1, characterized in that: Step 2 is based on G 2 The specific method for calculating the class-specific conditional dependency strength of a statistic is as follows: for a subset D of samples within a class... y Any two features X in i With X j Given condition sets S and G 2 The formula for calculating the statistic is: Where, n y (x i ,x j ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i Feature X j The value is x j The frequency of samples when the condition set S takes the value s; n y (s) represents the subset of samples D within the class. y In the given condition set S, the sample frequency is the value of s; n y (x i ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i The frequency of samples when the condition set S takes the value s; n y (x j ,s) represents the subset of samples D within the class. y In the middle, feature X j The value is x j The sample frequency when the condition set S takes the value s; The larger the value of X, the higher the feature X is within class y. i With X j Given a set of conditions S, the less independent the conditions are, the stronger the conditional dependency is. The smaller the value of X, the higher the feature X is within class y. i With X j Given a condition set S, the closer the condition is to independence, the lower the strength of the conditional dependency.

4. The multi-class feature selection method for class-specific conditional independence testing according to claim 1, characterized in that: The specific method for calculating class-specific conditional dependency strength based on conditional mutual information in step 2 is as follows: For the in-class sample subset D y Any two features X in i With X j Given a condition set S and conditional mutual information I y (X i ,X j The formula for calculating |S) is: Among them, P y (x i ,x j ,s) represents the subset of samples D within the class. y In the middle, feature X i The value is x i Feature X j The value is x j The joint probability when the condition set S takes the value s. N y For the in-class sample subset D y Total number of samples; P y (x i ,x j |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X i The value is x i And feature X j The value is x j The conditional probability of P; y (x i |s) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X i The value is x i The conditional probability of P; y (x j |S) represents the subset of samples D within the class. y In the given condition set S takes the value s, the feature X j The value is x j The conditional probability; Conditional Mutual Information I y (X i ,X j The larger the value of |S), the stronger the feature X within class y. i With X j Given a condition set S, the stronger the condition dependency, the higher the strength; when I y (X i ,X j When |S)=0, it indicates that within class y, feature X i With X j They are independent of each other given a set of conditions S.

5. The multi-class feature selection method for class-specific conditional independence testing according to claim 1, characterized in that: Step 3 specifically includes: 3.1 Initialization parameters: For each category y, initialize the candidate feature set CanF = F. 3.2 Forward Selection Stage: In the candidate feature set CanF, select the feature that makes the dependency dep(F) equal to the desired value. i Fbest is the largest feature of (Y|CurMB), and the dependency is obtained by the class-specific conditional dependency strength calculation method in step 2; if Fbest and Y have a significant conditional dependency given CurMB, and the dependency is greater than the preset threshold θ1, θ1∈(0,1), then Fbest is added to CurMB and removed from CanF. 3.3 Candidate Set Pruning Stage: For each feature F in the candidate feature set CanF i If F i If F is independent of Y under given CurMB, and its dependency is less than or equal to a preset threshold θ0, where θ0∈(0,θ1), then F... i Remove from CanF; 3.4 Reverse Elimination Phase: In the current CurMB, select the option that makes the dependency dep(F) equal to the value of F. j ,Y|CurMB{F j The smallest feature Fworst; if Fworst and Y are independent of each other given CurMB{Fworst} and their dependence is less than or equal to θ0, then Fworst is removed from CurMB; 3.5 Iteration Termination: Repeat steps 3.2-3.4 until CurMB no longer changes, obtaining the initial Markov BlanketCurMB for each category y. y .

6. The multi-class feature selection method for class-specific conditional independence testing according to claim 1, characterized in that: Step 4 specifically includes: 4.1 Feature Priority Ranking: For each category y, the initial Markov Blanket CurMB y The features are sorted in descending order based on the class-specific conditional dependency strength between the features and the target variable Y, resulting in a sorted feature list. 4.2 Filtering high-priority features: Set a percentage threshold k, k∈(0,100], and select the top k% of features from the sorted feature list to form a feature queue Q. y ; 4.3 Missing Feature Recovery: For feature queue Q y Each feature Q in y [i], calculate the extended MarkovBlanketCurMB corresponding to this feature. y Q y [i] = ESMB(Q) y [i],F), where ESMB is the extended Markov Blanket search function used to search for the expression Q. y [i] There exists a feature set with conditional dependencies; if the target variable Y belongs to CurMB y (Q y If [i]), then the associated feature corresponding to that feature is added to the current Markov Blanket; 4.4 Generating the Optimized Markov Blanket: After completing the missing feature recovery for all features, the optimized Markov Blanket CurMB for each category y is obtained. y .

7. The multi-class feature selection method for class-specific conditional independence testing according to claim 1, characterized in that: Step 5 specifically includes: 5.1 Feature Contribution Evaluation: For each optimized Markov BlanketCurMB y Calculate the in-class contribution C of feature f in the class. y (f) = dep(f, Y|MB) y {f} represents the conditional dependence of feature f on the target variable Y within category y; simultaneously, the global contribution of feature f is calculated. Where ω y The weight coefficients for category y are... N y The number of samples for category y; 5.2 Redundant Feature Removal: For any two features f1 and f2, if there exists a category y such that... γ is the redundancy judgment threshold. If γ > 0 and the global contribution C(f1) ≤ C(f2), then feature f1 is removed. 5.3 Final Feature Subset Generation: Features with global contribution C(f) ≥ δ, where δ is the feature selection threshold, and δ > 0 are retained to form the final feature subset F. * .

8. The multi-class feature selection method for class-specific conditional independent testing according to claim 5, characterized in that: The preset thresholds θ0 and θ1 are determined by the following method: based on the intra-class sample subset D y Sample size N y θ0 = 0.01 × log(N) y ) / N y θ1 = 0.05 × log(N) y ) / N y This ensures that the significance of the dependency assessment is appropriate for the sample size.

9. The multi-class feature selection method for class-specific conditional independence testing according to claim 6, characterized in that: The percentage threshold k ranges from 30% to 80%, and the optimal value is determined through cross-validation. During the cross-validation process, the k value that gives the classifier the highest accuracy on the validation set is selected.

Images