Explanatable classification method and system based on rule learning and neural network architecture search

Abstract

The invention discloses an interpretable classification method and system based on rule learning and neural network architecture search, and belongs to the field of artificial intelligence and machine learning. The method comprises the following steps of performing discretization and binarization processing on continuous features of original data; performing neural rule learning on the data of the input layer, and generating a rule set through conjunctive and disjunctive operations; and performing neural network architecture search on the model structure by using reinforcement learning to find an optimal scheme for model architecture processing. The system comprises a feature preprocessing module, a rule learning and generating module and a neural network architecture searching module which can realize the steps of the method. Experimental results show that the method has very strong competitiveness in the aspects of prediction accuracy and interpretability, can generate highly interpretable logic rules, and keeps excellent prediction performance at the same time.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and machine learning, specifically relating to an interpretable classification method based on rule learning and neural network architecture search. Technical Background

[0002] With the rapid development of artificial intelligence technology, deep learning, with its powerful feature representation capabilities, has been widely applied in the field of machine learning and has achieved breakthroughs in many complex tasks. However, in critical areas such as healthcare, financial risk control, and judicial decision-making, where the reliability and interpretability of model outputs are extremely important, the "black box" nature of deep neural networks severely limits their practical application. Therefore, how to enhance the interpretability and auditability of the decision-making process while ensuring model performance has become a core issue that urgently needs to be addressed in high-risk fields.

[0003] Existing interpretability research mainly falls into two categories: one is ex-post interpretation methods, which indirectly reveal the prediction mechanism by analyzing the output behavior of existing complex models; the other is endogenously interpretable models, such as decision trees and rule set models, which rely on the logical expression of their structure to achieve transparency of the reasoning process. The former has the advantage of strong versatility, applicable to almost any model architecture, and can be deployed without modifying the original model structure. However, due to the inability to deeply understand the internal mechanisms of the model, the resulting explanations often lack accuracy and consistency, and are difficult to reflect the model's true decision-making logic. In addition, most methods only have local interpretability and are difficult to provide a global transparent perspective. The latter enhances the transparency of the model by introducing explicit rules. In recent years, it has often been combined with strategies such as ensemble learning to improve performance. However, in complex tasks, due to the increased number of stacked layers in the model structure, the original interpretability is difficult to maintain, limiting its applicability.

[0004] To strike an effective balance between model performance and interpretability, researchers have proposed neural rule learning methods, aiming to integrate logical rule representations with the expressive power of deep models, enabling the model to automatically generate interpretable rule sets to guide decision-making. However, these methods still face bottlenecks such as high rule generation complexity, weak feature interaction modeling capabilities, and reliance on manual model structure design, making it difficult to fully realize their potential in practical applications.

[0005] In recent years, Neural Architecture Search (NAS) has become an important means to improve the efficiency of model structure design. NAS automatically explores network structure and hyperparameter combinations through mechanisms such as reinforcement learning, evolutionary algorithms, or gradient optimization. While reducing human intervention, it dynamically optimizes the network depth and width according to task requirements, thereby improving the model's adaptability and generalization ability.

[0006] In summary, there is currently a lack of classification methods that can simultaneously balance model performance, interpretability, and structural adaptability. To address this, we propose an interpretable classification method that integrates rule learning and NAS (Neural Network Architecture). This method automatically learns logical rules to represent structured data features and jointly searches for data-driven network structures that adapt to different tasks. It effectively balances classification accuracy and model transparency, reduces the cost of human intervention in model development, and possesses broad application prospects and significant research value. Summary of the Invention

[0007] This invention provides an interpretable classification method based on rule learning and neural network architecture search, aiming to solve the problem of balancing prediction performance and model interpretability in existing structured data classification methods. By introducing a fusion design of rule learning mechanism and neural network architecture search technology, this invention achieves simultaneous improvement in classification performance and model transparency, possessing automatic modeling, high interpretability, and good generalization ability. It is suitable for scenarios with high requirements for decision credibility, such as medical diagnosis, financial risk control, and judicial review.

[0008] The method of the present invention includes three core modules: a feature preprocessing module, a rule learning and generation module, and a neural network architecture search module.

[0009] In the feature preprocessing module, a discretization algorithm based on the Class-Attribute Contingency Coefficient (CACC) is introduced for continuous features in the input data. This algorithm first sorts all values ​​of each continuous feature in ascending order, calculates the midpoint between adjacent values, and combines the maximum and minimum values ​​of the feature to form a candidate splitting boundary set, ensuring comprehensive coverage of the original feature value range. After initializing the discretization scheme, the CACC value is calculated based on an evaluation function containing information such as the number of classes, the number of intervals, and the number of samples within each interval. An iterative optimization strategy is used to select the splitting result with the largest CACC value from the candidate schemes as the current optimal solution, until the model performance no longer improves or the preset splitting limit is reached. After discretization, all discrete features (including the original discrete features and the discretized continuous features) and label variables are one-hot encoded, transforming them into a unified binary feature vector, which serves as the input for subsequent model learning.

[0010] In the rule learning and generation module, a neural rule learning method combining Conceptual Rule Structure (CRS) and Multi-Layer Logical Perceptron (MLLP) is proposed. This method utilizes a CRS structure composed of alternating conjunction and disjunction layers to achieve a structured expression of classification rules, thereby improving the model's interpretability. However, since the discrete weights in CRS cannot be directly trained using gradient descent, this invention introduces a continuous surrogate model—MLLP. This model approximates logical operations using fully connected networks and continuous activation functions. By restricting the connection weights to the interval [0,1], it achieves a differentiable representation of the logical functions. Furthermore, a mean squared error loss function with L1 regularization is designed to optimize model parameters, improving modeling accuracy while ensuring clear and interpretable rules.

[0011] In the neural network architecture search module, an automated search framework based on reinforcement learning is employed to optimize the network structure of MLLP. The controller module is constructed using an RNN and generates hyperparameter combinations for candidate MLLP architectures, such as the number of network layers and the number of nodes per layer, based on the REINFORCE policy gradient algorithm. The controller uses the validation set accuracy as a reward signal to guide the search direction and enforces a structural specification that alternates between conjunctive and disjunctive layers, thereby ensuring the interpretability of the rule representation.

[0012] The classification method for the above systems includes the following steps:

[0013] S1. The continuous feature values ​​are discretized using the CACC-based continuous feature discretization method, and then one-hot encoding is used to convert the discretized continuous features and the original category features into binary feature vectors.

[0014] S2. Input layer data is processed using neural rule learning data, and an interpretable rule set is constructed through logical conjunction and disjunction operations;

[0015] S3. A neural architecture search framework based on reinforcement learning is adopted, with a constrained conjunctive-disjunctive alternating structure. RNN is used to generate candidate architectures, and the number of network layers and nodes is automatically optimized with the verification accuracy as the reward signal.

[0016] Further, in step S1, the continuous feature values ​​are discretized using a CACC-based continuous feature discretization method. Specifically, for each continuous feature... First, determine its minimum value. and maximum value Collect their unique values ​​and sort them to form a set. Calculate the midpoints of adjacent values ​​to generate a candidate boundary set. Initialization discrete scheme = .

[0017] The CACC value of the new scheme is calculated by iteratively inserting candidate boundaries:

[0018]

[0019]

[0020] in, For the number of categories, For interval numbers, For the first Class in The number of samples in the interval; Ns represents the number of samples; N p，+ N represents the number of samples belonging to class p. +，q This represents the number of samples where the value of attribute A falls within the q-th interval. Choose the option that maximizes CACC and update... .

[0021] Furthermore, in step S2, the neural rule learning model is constructed by using a multilayer logic perceptron (MLLP) as a proxy model for the concept rule set (CRS), and by using continuous activation functions to mimic logical conjunction and disjunction operations.

[0022]

[0023] Among them, let and These represent the corresponding CRS nodes in MLLP. and Neurons. In MLLP, the first The weight matrix of the layer, where ∈[0, 1]. When simulating disjunction and conjunction operations, the output must be restricted to the range [0, 1]. Therefore, using... Function shearing weights. Then, using mean squared error and... The loss function for training an MLLP model using regularization is:

[0024]

[0025] in It is the mean square error. express Regular terms, Represents the MLLP model. This represents all the weights of the model.

[0026] Furthermore, in step S3, a neural architecture search framework based on reinforcement learning is used to find the optimal configuration, including the number of network layers and the number of nodes per layer, so that the resulting MLLP can achieve the best classification performance while maintaining the interpretability of the model.

[0027] An RNN is used as the controller to generate the architecture description of candidate MLLP models. The controller parameters are updated by using the verification accuracy as the reward signal. In the search space, the architecture is constrained to follow the alternation pattern of conjunction-disjunction layers, the form of weights, and the network depth and width. Distributed parallel training, early stopping, and parameter sharing strategies are introduced to transform the optimal architecture into an interpretable rule-based classifier.

[0028] Compared with the prior art, the present invention has the following significant technical advantages:

[0029] (1) This invention innovatively integrates rule learning and neural network architecture search technology, overcoming the technical bottleneck of difficulty in achieving both prediction accuracy and model interpretability in traditional classification methods, and automatically generating high-performance models with transparent reasoning logic.

[0030] (2) The neural rule learning model proposed in this invention can explicitly extract decision rules while maintaining classification ability, which is convenient for human understanding, review and reuse, and enhances the regulatory compliance and applicability of the model.

[0031] (3) The neural architecture search module implemented by reinforcement learning can automatically explore the optimal model structure, reduce the cost of manual parameter tuning, and has good cross-dataset transfer ability and rule generalization ability.

[0032] (4) The generated rule set has a clear logical structure and feature constraints, which can be used to build a domain knowledge base and assist expert decision-making. It is especially suitable for key industry scenarios with high requirements for model controllability. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0034] Figure 1 is a schematic diagram of the interpretable classification method based on rule learning and neural network architecture search.

[0035] Figure 2 shows the NRL-NAS model structure. Detailed Implementation

[0036] To more clearly illustrate the purpose, technical solutions, and advantages of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0037] Example 1

[0038] refer to Figure 1 , Figure 1 This is a flowchart illustrating the rule-based learning and neural network architecture search method of the present invention. The provided method includes:

[0039] S1 uses a CACC-based continuous feature discretization method to discretize continuous feature values ​​and converts them into binary features using one-hot encoding.

[0040] The goal of the feature preprocessing module is to convert structured data into binary vectors suitable for learning logical rules, discretize continuous features, and process the continuous features based on the CACC algorithm. Collect all distinct values ​​and sort them in ascending order to form a set. Calculate the midpoint and minimum value of adjacent values. and maximum value Generate candidate boundary set Initialization Discrete Scheme = Then, the iterative process begins, calculating the CACC value of the current scheme:

[0041]

[0042]

[0043] in, For the number of categories, For interval numbers, For the first Class in The number of samples in the interval. Iteratively insert candidate boundaries, select the solution that maximizes CACC and update the optimal solution, until the global optimal solution cannot be improved or reaches the upper limit of the interval, and output the discretized solution. Then, perform one-hot encoding on the discretized continuous features and the original categorical features to generate binary feature vectors.

[0044] S2 uses neural rule learning data to process input layer data. It converts the trained MLLP into a discrete CRS structure through weight binarization and constructs an interpretable rule set by combining logical conjunction and disjunction operations.

[0045] The goal of the rule learning and generation module is to generate rules that possess both interpretability and classification capabilities. Inspired by MLLP and CRS, a neural rule learning model is proposed and constructed. CRS is... It consists of alternating "conjunctive layers" and "disjunctive layers," with node connections formed by an adjacency matrix. express:

[0046]

[0047] While CRS exhibits good expressiveness and transparency, its weights are discrete, making it impossible to train the model using gradient descent. Therefore, we employ MLLP as an alternative model to mimic the behavior of CRS. The number of layers and nodes per layer in MLLP correspond one-to-one with the data in CRS. The key difference is that MLLP is a fully connected neural network model.

[0048] To enable MLLP to mimic the logical disjunction and simultaneous operations in CRS, MLLP uses continuous weights to simulate logical operations, and its continuous logic activation function is:

[0049]

[0050] in, pass The function is used to trim the weights, ensuring the output is within the range [0,1]. When the weights of MLLP are the same as the corresponding weights of CRS, MLLP can completely replicate the behavior of CRS. The loss function for training MLLP is:

[0051]

[0052] in It is the mean square error. express Regular terms, Represents the MLLP model. This represents all the weights of the model.

[0053] S3 is a neural architecture search framework based on reinforcement learning. It has an alternating structure of constraint conjunction and disjunction layers, uses RNN to generate candidate architectures, and automatically optimizes the number of network layers and nodes with the verification accuracy as the reward signal.

[0054] The goal of the neural network architecture search module is to automatically discover high-performance, interpretable neural rule structures. A reinforcement learning-based neural architecture search framework is employed to find the optimal configuration, including the number of layers and nodes per layer, thereby enabling the generated MLLP to achieve the best classification performance while maintaining interpretability. An RNN is used as the learning controller to progressively generate structural descriptions of candidate MLLP models. The controller predicts structural hyperparameters, instantiates the model, and uses validation accuracy as the reward signal. The REINFORCE algorithm optimizes the controller parameters while constraining the search space to ensure an interpretable structure with alternating conjunctive and disjunctive layers. Binary or truncated continuous weights are enforced to ensure logical consistency, and network depth and width are limited to avoid excessive rule complexity. A distributed training scheme is used to address the high cost issue. Early stopping and parameter sharing strategies are employed to accelerate the convergence speed of the search phase. Top-K architectures are selected for further fine-tuning, extracting the best-performing architecture and interpreting it as a rule-based optimized classifier.

[0055] Example 2

[0056] This invention achieves a balance between prediction performance and interpretability in structured data classification through a closed-loop process of feature preprocessing → rule learning and generation → neural network architecture search, providing an effective solution for scenarios requiring high transparency.

[0057] The specific experimental procedure is as follows:

[0058] Step 1: Dataset selection and feature processing;

[0059] Twelve domain-representative benchmark datasets were selected from the UCI Machine Learning Repository, including datasets for adults, bank marketing, and banknotes. These datasets exhibit significant differences in sample size (100-67557), class distribution (2-26 classes), and feature space (4-42 dimensions, including digit, categorical, and mixed features). For continuous features, a Gaussian-assumption-based mean estimation method was used to handle missing values, automatically switching to median estimation when significant skewness was detected. For categorical variables, a maximum likelihood-based pattern estimation method was employed, using random sampling to select estimated values ​​when multiple patterns existed. All estimation operations were performed independently on the training set, strictly adhering to data isolation mechanisms. Continuous features were discretized using a CACC-based method. The CACC value of candidate boundaries was iteratively calculated, and the optimal boundary was selected to maximize classification performance. Subsequently, the discretized continuous features, the original discrete features, and the labels were encoded.

[0060] Step 2: Setting evaluation indicators;

[0061] The macro F1 score was used as the primary evaluation metric, and the unweighted average of the F1 scores across all classes was calculated to eliminate the influence of sample size differences. A stratified 5-fold cross-validation method was employed to ensure that the class distribution at each fold strictly matched the original dataset. In each round of validation, 80% of the data was used for model training, and 20% served as an independent validation set for hyperparameter tuning. The final experimental results are reported as the mean and standard deviation of the five-fold test set metrics, and statistical significance tests were conducted to ensure the reliability of the conclusions.

[0062] Step 3: Comparison method selection;

[0063] Nine baseline models covering different modeling paradigms were selected for systematic comparison, including CRS, decision trees (C4.5), classification and regression trees (CART), scalable Bayesian rule lists (SBRL), logistic regression (LR), fragmented linear neural networks (PLNN), support vector machines (SVM), gradient boosting decision trees (GBDT), and two variants of random forests (e=10 and e=100). These baseline models exhibit gradient characteristics on the interpretability spectrum, establishing a multidimensional reference framework for evaluation.

[0064] Step 4: Model training and optimization;

[0065] An RNN is used as the controller, and the expected reward is optimized using the REINFORCE algorithm. The controller parameters are updated using the verification accuracy as the reward signal. The controller sequentially generates structural descriptions of candidate MLLP models, including hyperparameters such as the number of layers and nodes per layer. Each sampled architecture is instantiated as an MLLP model and trained using a defined alternative loss function. During training, the search space is constrained to ensure that all generated architectures strictly adhere to the alternation pattern of connection and disjunction layers, while binary or truncated continuous weights are enforced to maintain the logical consistency of the model. To prevent the construction of overly complex rule structures, upper limits are set on the depth and width of the network, and a distributed training scheme is adopted to train multiple candidate MLLP models in parallel. Early stopping and parameter sharing strategies are also employed to accelerate the convergence speed during the search phase.

[0066] Step 5: Analysis of experimental results;

[0067] The proposed NRL-NAS model achieves an average macro F1 score of 87.23% on 12 UCI benchmark datasets (including Adult, Wine, etc.) and 100% classification accuracy on datasets such as Mushroom. The average length of the generated rule set is dynamically adjusted according to the data dimension. For example, the average rule length is 2.11 on the Magic04 dataset and 10.87 on Connect-4. The rules are concise for low-dimensional data and cover complex feature interactions for high-dimensional data, balancing accuracy and readability.

[0068] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A system for interpretable classification based on rule learning and neural network architecture search, characterized in that, It includes a feature preprocessing module, a rule learning and generation module, and a neural network architecture search module: The feature preprocessing module uses a CACC-based continuous feature discretization method to discretize continuous feature values ​​and then uses one-hot encoding to convert them into binary features. The rule learning and generation module uses neural rule learning data to process the input layer data. It converts the trained MLLP into a discrete CRS structure through weight binarization and constructs an interpretable rule set by combining logical conjunction and disjunction operations. The neural network architecture search module is based on a neural architecture search framework using reinforcement learning. It employs an alternating structure of constraint conjunction and disjunction layers, uses RNNs to generate candidate architectures, and automatically optimizes the number of network layers and nodes using verification accuracy as the reward signal.

2. The classification method of the system according to claim 1, characterized in that, Includes the following steps: S1. The continuous feature values ​​are discretized using the CACC-based continuous feature discretization method, and then one-hot encoding is used to convert the discretized continuous features and the original category features into binary feature vectors. S2. Input layer data is processed using neural rule learning data, and an interpretable rule set is constructed through logical conjunction and disjunction operations; S3. A neural architecture search framework based on reinforcement learning is adopted, with a constrained conjunctive-disjunctive alternating structure. RNN is used to generate candidate architectures, and the number of network layers and nodes is automatically optimized with the verification accuracy as the reward signal.

3. The classification method according to claim 2, characterized in that, In step S1, the continuous feature values ​​are discretized using a CACC-based continuous feature discretization method. Specifically, for each continuous feature... First, determine its minimum value. and maximum value Collect their unique values ​​and sort them to form a set. Calculate the midpoints of adjacent values ​​to generate a candidate boundary set. Initialization discrete scheme = ; The CACC value of the new scheme is calculated by iteratively inserting candidate boundaries: ； ； in, For the number of categories, For interval numbers, For the first Class in The number of samples in the interval; Ns represents the number of samples; N p，+ N represents the number of samples belonging to class p. +，q This represents the number of samples where the value of attribute A falls within the q-th interval; select the option that maximizes CACC and update. Then, the discretized continuous features and the original discrete features are converted into binary features using one-hot encoding.

4. The classification method according to claim 2, characterized in that, In step S2, the neural rule learning model is constructed using a multilayer logic perceptron (MLLP) as a continuous-valued version of the concept rule set (CRS), and activation functions are used to mimic logical conjunction and disjunction operations. ； Among them, let and These represent the corresponding CRS nodes in MLLP. and Neurons; In MLLP, the first The weight matrix of the layer, where ∈[0, 1]; when simulating disjunction and conjunction operations, the output must be restricted to the range [0, 1]; therefore, using Function weight clipping; then train the MLLP model: ； in It is the mean square error. express Regular terms, Represents the MLLP model. This represents all the weights of the model.

5. The classification method according to claim 2, characterized in that, In step S3, a neural architecture search framework based on reinforcement learning is used to find the optimal configuration, including the number of network layers and the number of nodes per layer, so that the resulting MLLP can achieve the best classification performance while maintaining the interpretability of the model. An RNN is used as the controller to generate the architecture description of candidate MLLP models. The controller parameters are updated by using the verification accuracy as the reward signal. In the search space, the architecture is constrained to follow the alternation pattern of conjunction-disjunction layers, the form of weights, and the network depth and width. Distributed parallel training, early stopping, and parameter sharing strategies are introduced to transform the optimal architecture into an interpretable rule-based classifier.

Images