Complex concept-based interpretable data representation learning method

Abstract

The invention discloses an interpretable data representation learning method based on a complex concept, and belongs to the crossing field of deep learning and a fuzzy system. According to the method, an axiom fuzzy set theory and a neural network are combined, and a model based on fuzzy rule representation learning (FRRL) is constructed. According to the method, firstly, through a fuzzification step, input data features are converted into fuzzy concepts; generating a complex rule for describing a nonlinear relation of the features by using a fuzzy logic layer; and carrying out classification decision through a linear classification layer. In addition, a gradient descent algorithm is introduced to optimize rules and parameters in a fuzzy system, so that the learning efficiency of the model is improved. While the classification accuracy is ensured, the method can explain the decision-making process of the model by extracting fuzzy rules. The method is particularly suitable for application scenes requiring high transparency and interpretability requirements, such as medical diagnosis and financial risk assessment.

Description

Technical Field

[0001] The present invention belongs to the intersection of deep learning and fuzzy systems, and in particular, proposes an interpretable data representation learning method by combining axiomatic fuzzy set theory (AFS) with neural networks. The present invention utilizes the fusion of fuzzy logic and deep learning models to solve the problem of insufficient interpretability in existing deep learning models, while improving classification performance, and is suitable for application scenarios such as medical diagnosis and financial evaluation that require high transparency and interpretability. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, deep learning models have made significant progress in handling complex tasks such as image recognition, speech processing, and natural language understanding. With its powerful data processing capabilities and highly accurate classification effects, deep learning has been widely used in many fields. However, one of the main limitations of such models is the lack of interpretability. They are often regarded as "black box models" that cannot clearly explain the logical relationship between input data and output results. This unexplainability has caused great concern in areas such as medical diagnosis and financial risk assessment that require high decision transparency. Users and regulators not only require models to provide accurate predictions, but also require an understanding of the basis for their decisions to ensure rationality and compliance.

[0003] Currently, deep learning models face the following technical challenges:

[0004] 1. Insufficient explainability. Although deep learning models can achieve high-precision classification results in complex tasks, it is difficult to explain the internal logic of their decision-making process. For many key areas, models must provide clear decision-making basis to help users understand their working mechanism and trust the prediction results.

[0005] 2. High-dimensional data optimization challenges. Traditional optimization methods usually perform poorly when dealing with high-dimensional data. As the data dimension increases, the complexity of the model increases greatly, resulting in longer training time, increased optimization difficulty, and possible overfitting problems, which affects the generalization ability of the model.

[0006] 3. Limitations of fuzzy systems: Fuzzy systems have certain advantages due to their ability to handle uncertainty and fuzzy data. Through fuzzy rules, they can clearly express the relationship between input features and output results. However, when faced with high-dimensional data or complex concepts, the number of rules of traditional fuzzy systems tends to increase dramatically, resulting in complex rule design and difficulty in system optimization, which in turn affects classification performance.

[0007] Axiomatic fuzzy set theory (AFS) provides an effective solution to the above problems. AFS can automatically generate fuzzy membership functions, reducing the complexity of manually designed rules, and the generated rules have strong interpretability. Although AFS can alleviate the complexity problem of fuzzy systems to a certain extent, it still has performance limitations when dealing with nonlinear and high-dimensional data. In addition, the existing AFS theory needs to be improved in terms of optimization efficiency, especially when dealing with complex parameters, the convergence speed of conventional optimization methods is slow.

[0008] In addition, in the data representation learning of complex concepts, existing methods often focus on improving classification accuracy while ignoring the interpretability and transparency of the model. In the process of automatic feature extraction, it is necessary to more intuitively show the relationship between features and results in order to generate interpretable feature representations.

[0009] Therefore, the current technical field urgently needs a model design that can balance efficient classification performance and decision transparency, especially in the following aspects:

[0010] 1. Explainability: The model must not only provide highly accurate prediction results, but also be able to clearly explain the rules and basis of its decision-making process to enhance users’ trust in the model.

[0011] 2. Complex parameter optimization capability: Through advanced optimization algorithms, complex parameter problems in high-dimensional data can be solved, optimization efficiency can be improved, and the model can converge quickly when processing complex data.

[0012] 3. Automatic generation and simplification of rules: A method is needed to automatically generate concise and explanatory rule sets to cope with the problem of the surge in the number of rules and improve the efficiency and operability of the system. Summary of the invention

[0013] In order to solve the above problems, the present invention proposes an interpretable data representation learning method based on complex concepts. This method combines the advantages of axiomatic fuzzy set theory (AFS) and neural networks, and designs a fuzzy rule representation learner (FRRL) model. The model can efficiently process complex nonlinear data relationships, automatically generate clear fuzzy rules, and optimize the membership functions and weight parameters in the system through the gradient descent optimization algorithm, thereby achieving efficient optimization of complex parameters. The FRRL model can not only generate efficient classification results, but also provide clear explanations for each classification decision, which is particularly suitable for application scenarios that require high transparency and interpretability.

[0014] The technical solution adopted by the present invention is: a method for learning interpretable data representation based on complex concepts, the method comprising the following steps:

[0015] S1. Fuzzification: The fuzzification layer processes data by converting the input feature vector into a fuzzy vector. For each feature in the data set, a set of simple concepts is defined to represent the fuzzy state of the feature. By dividing the value range of the feature, each feature is assigned multiple fuzzy concepts. The system generates a fuzzy representation by calculating the membership degree of the sample to the fuzzy concept.

[0016] Assume the data set is and feature set ,in and Represent the number of samples and the number of features respectively; for each feature , defines a set of simple concepts ,in is the number of concepts; through the weight function , calculate the degree of association between each sample and the simple concept, the formula is as follows:

[0017]

[0018] in, is an adjustable parameter, , is the partition spacing, indicating the division of the eigenvalue range;

[0019] Membership function Representation sample In the collection The membership degree in is:

[0020]

[0021] This formula measures the membership of a sample in a fuzzy set and represents the fuzzy association between the sample and a simple concept;

[0022] S2. Fuzzy logic rule generation: The fuzzy logic layer combines the axiomatic fuzzy system and the neural network structure for processing. The axiomatic fuzzy system uses multi-level "and" operations AND OR operation , automatically generates complex fuzzy rules to capture the nonlinear relationship between input features; by alternating between conjunctive normal form CNF and disjunctive normal form DNF, the interaction between complex features is expressed; the neural network part shares parameters with the axiomatic fuzzy system, and the neural network optimizes the weight parameters through back propagation and gradient descent algorithms; finally, the rule generation mechanism of the fuzzy logic layer effectively captures feature associations;

[0023] The fuzzy logic layer includes two types: axiomatic fuzzy system and neural network;

[0024] The multi-layer structure of axiomatic fuzzy system: the fuzzy logic layer consists of The axiomatic fuzzy system consists of layers, each layer contains fuzzy "Computational Layer and blur "Computational Layer , where odd-numbered layers perform " "operation, even layers perform " "Operation; Node Represents fuzzy rules, nodes Represents a set of rules, and the operations are calculated by fuzzy and operator product and Max fuzzy or operator respectively; the formula for learning feature representation layer by layer is as follows:

[0025]

[0026] Neural network structure: The neural network part corresponds to the nodes of the axiomatic fuzzy system. The continuous weights within the interval are optimized; all layers are fully connected layers, and the logical activation function is used to simulate the "∧" operation and the "∨" operation:

[0027]

[0028]

[0029] The gradient descent algorithm is used to update the weights, and the gradient is calculated by back propagation to update the weights of each layer. :

[0030]

[0031] in, is the learning rate, is the loss function; the loss function uses mean square error MSE combined with L2 regularization, and the formula is as follows:

[0032]

[0033] in, is the regularization coefficient, is the category label, is the predicted value of the model;

[0034] S3. Linear classification: The linear classification layer makes decisions based on the new features generated by the fuzzy logic layer. The fuzzification layer and the fuzzy logic layer act as feature learners, generating feature representations through fuzzy rules, and the linear layer performs classification based on the feature representations.

[0035] The linear layer of FRRL combines the fuzzification layer and the fuzzy logic layer as a feature learner and classifier; the linear layer identifies important features through weights and analyzes the corresponding fuzzy rules to explain the model decision; the linear layer uses KANS, and a learnable activation function is used at the edge of the network. Each weight is parameterized by a spline function; the model complexity is controlled by reducing the number of logic layers and nodes; the model complexity is limited by optimizing the loss function through L2 regularization, and the formula is as follows:

[0036]

[0037] is the regularization coefficient;

[0038] S4. Rule extraction and simplification: By converting the complex decision-making process of the fuzzy logic layer into "if-then" rules; the rules express the relationship between input features and classification results, providing a clear and explainable basis for each classification; in the model, each path from the fuzzification layer to the linear classifier is represented as a fuzzy rule;

[0039] For each input sample , and its corresponding rules are extracted as follows:

[0040]

[0041] is the eigenvalue of the sample, Represents the fuzzy set membership function corresponding to each feature, which evaluates the degree to which the feature belongs to a fuzzy set; the conclusion part of the rule is the classification result given by the model. Multiple conditions are connected by "AND", which means that when all conditions are met at the same time, the sample Belong to a certain class;

[0042] Simplify rules: merge rules with similar premises or replace some "AND" conditions with "OR". The simplified rules are:

[0043]

[0044] In this rule, some conditions are connected using "OR", which means that as long as any one of the conditions is met, it will be OK.

[0045] The beneficial effects of the present invention are as follows: the present invention solves the problem of insufficient transparency and interpretability in existing deep learning models. The method combines axiomatic fuzzy set theory with neural networks to construct a model based on fuzzy rule representation learning (FRRL). Specifically, the method first converts the input data features into fuzzy concepts through a fuzzification step; then, a fuzzy logic layer is used to generate complex rules for describing the nonlinear relationship of features; and then a linear classification layer is used to make classification decisions. In addition, the present invention introduces a gradient descent algorithm to optimize the rules and parameters in the fuzzy system, thereby improving the learning efficiency of the model. While ensuring the accuracy of classification, the method can also explain the decision-making process of the model by extracting fuzzy rules.

[0046] The innovative combination of axiomatic fuzzy set theory and neural network not only effectively processes complex nonlinear data, but also explains the basis of classification decisions through the generated fuzzy rules. This method is particularly suitable for application scenarios with high requirements for decision transparency, such as medical diagnosis, financial risk assessment, etc. Experimental results show that the present invention exhibits excellent classification performance and good interpretability on multiple standard data sets, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the model structure. DETAILED DESCRIPTION

[0048] This paper proposes an interpretable data representation learning method based on complex concepts, which solves the problems of insufficient transparency and interpretability of traditional deep learning models by combining axiomatic fuzzy set theory with neural network models.

[0049] A method for learning interpretable data representation based on complex concepts, the method comprising the following steps:

[0050] Fuzzification: In this step, the fuzzification layer processes the data by converting the input feature vector into a fuzzy vector. For each feature in the data set, a set of simple concepts are defined to represent the fuzzy state of the feature, such as "very small", "medium", or "very large". By dividing the value range of the feature, each feature is assigned multiple fuzzy concepts. The system generates a fuzzy representation by calculating the membership of the sample to these concepts. The membership reflects the degree of match between the sample and each concept. The closer the concept, the higher the membership. This process effectively captures the fuzziness of the feature and provides a basis for the subsequent fuzzy rule generation.

[0051] In the fuzzification process, the fuzzification layer converts the input feature vector into a fuzzy vector. Assume that the data set is and feature set ,in and Represent the number of samples and the number of features respectively. For each feature , defines a set of simple concepts ,in is the number of concepts. Through the weight function , calculate the degree of association between each sample and the simple concept, the formula is as follows:

[0052]

[0053] in, is an adjustable parameter, , is the partition interval, indicating the division of the eigenvalue range. Representation sample In the collection The membership degree in is:

[0054]

[0055] This formula measures the membership of a sample in a fuzzy set and reflects the fuzzy association between the sample and a simple concept.

[0056] Fuzzy logic rule generation: In this step, the fuzzy logic layer combines the axiomatic fuzzy system and the neural network structure for processing. The axiomatic fuzzy system automatically generates complex fuzzy rules through multi-level "and" and "or" operations to capture the nonlinear relationship between input features. By alternating between the conjunctive normal form (CNF) and the disjunctive normal form (DNF), the system can flexibly express the complex interactions between features. The neural network part of the structure is similar to the multi-layer perceptron and shares parameters with the axiomatic fuzzy system. Through back-propagation and gradient descent algorithms, the neural network optimizes the weight parameters and further improves the accuracy of rule generation. Finally, the rule generation mechanism of the fuzzy logic layer effectively captures feature associations and improves the accuracy and interpretability of classification decisions.

[0057] In order to enable the fuzzy logic layer to learn complex concepts and generate data representations, two versions of the logic operation layer are designed: discrete and continuous. The discrete version is used for model training, testing, and semantic interpretation, while the continuous version is only used for training. The two share model parameters. The fuzzy logic layer includes two versions: axiomatic fuzzy system and neural network.

[0058] The multi-layer structure of axiomatic fuzzy system: the fuzzy logic layer consists of The axiomatic fuzzy system consists of layers, each layer contains fuzzy "Computational Layer and blur "Computational Layer , where odd-numbered layers perform " "operation, even layers perform " "Operation. Node Represents fuzzy rules, nodes Represents a set of rules, and operations are performed using the fuzzy AND operator (product) and the fuzzy OR operator (Max). The formula for learning feature representation layer by layer is as follows:

[0059]

[0060] Neural network structure design and training: The neural network part corresponds to the nodes of the axiomatic fuzzy system. All layers are fully connected layers, and the logical activation function is used to simulate the "∧" and "∨" operations:

[0061] ,

[0062] Gradient descent algorithm: In order to optimize the model parameters, the gradient descent algorithm is used to update the weights. The gradient is calculated by back propagation to update the weights of each layer. :

[0063]

[0064] in, is the learning rate, is the loss function. The loss function uses mean square error (MSE) combined with L2 regularization, and the formula is as follows:

[0065]

[0066] in, is the regularization coefficient, is the category label, is the predicted value of the model. Through gradient descent optimization, the model is ensured to have high classification performance while maintaining simplicity.

[0067] Linear classification: In this step, the linear classification layer makes decisions based on the new features generated by the fuzzy logic layer. The fuzzification layer and the fuzzy logic layer act as feature learners, generating feature representations through fuzzy rules, and the linear layer classifies based on these features. Weights are used to identify key features and enhance interpretability through rule analysis. Inspired by Kolmogorov-Arnold Networks (KANS), FRRL uses learnable activation functions instead of traditional fixed activation functions, which improves the flexibility of the model. By adjusting the depth and width of the logic layer and using L2 regularization, the model can strike a balance between classification accuracy and complexity to meet the needs of different scenarios.

[0068] After the model is trained, the axiomatic fuzzy system can be used for testing and interpretation. The linear layer of FRRL combines the fuzzification layer and the fuzzy logic layer to act as a feature learner and classifier. The linear layer identifies important features through weights and analyzes the corresponding fuzzy rules to explain the model decisions.

[0069] The linear layer uses Kolmogorov-Arnold Networks (KANS), and uses a learnable activation function at the edge of the network to replace the fixed activation function in the traditional MLP. Each weight is parameterized by a spline function. The model complexity can be controlled by reducing the number of logical layers and nodes. The loss function is optimized by L2 regularization to limit the model complexity. The formula is as follows:

[0070]

[0071] is the regularization coefficient, which reduces overfitting and simplifies the model.

[0072] In terms of rule simplification and interpretation, by analyzing the linear layer weights, key features can be extracted and the fuzzy rule set can be simplified, making the model more interpretable.

[0073] Rule extraction and simplification: This step converts the complex decision-making process of the fuzzy logic layer into "if-then" rules. These rules clearly express the relationship between input features and classification results, providing a clear and explainable basis for each classification. This regularized output helps users understand the internal logic of the model, ensuring its transparency and explainability. Through this mechanism, the system can not only classify efficiently, but also clearly present the decision-making process, allowing users to trace and understand the basis behind each decision.

[0074] In the model, each path from the fuzzification layer to the linear classifier can be represented as a fuzzy rule. As the complexity of the task increases, the fuzzy logic layer may generate more rules, so the rules need to be extracted and simplified.

[0075] For each input sample , and its corresponding rules can be extracted as follows:

[0076]

[0077] is the eigenvalue of the sample, Represents the fuzzy set membership function corresponding to each feature, which evaluates the degree to which the feature belongs to a fuzzy set. The conclusion part of the rule is the classification result given by the model. Multiple conditions are connected by "AND", which means that when all conditions are met at the same time, the sample Belongs to a class.

[0078] To simplify the rules, you can merge rules with similar premises or replace some "AND" conditions with "OR" to reduce the complexity of the rules. For example, the simplified rule may be:

[0079]

[0080] Explanation: In this rule, some conditions are connected using "OR", which means that as long as any one of the conditions is met, it will be fine. This simplification can reduce unnecessary redundancy and improve the interpretability and computational efficiency of the model. This simplification method can effectively reduce the number of rules and maintain the classification effect.

[0081] The specific implementation steps of this method are described in detail below:

[0082] 1. Fuzzification. In this step, the input features are fuzzified through the fuzzification layer. Specifically, the system defines a set of simple concepts for each input feature, such as "very small", "medium", and "very large", and maps these features to fuzzy concepts through membership functions. The membership function calculates the degree of match between each sample and these fuzzy concepts, and the formula is as follows:

[0083]

[0084] in, is an adjustable parameter, Represents the central value of each fuzzy set. In this way, the fuzzified features can better reflect the uncertainty and ambiguity of the data.

[0085] 2. Fuzzy logic rule generation. After fuzzification, the fuzzy logic layer generates complex fuzzy rules based on the input features. This layer combines the axiomatic fuzzy system and performs multi-layer “AND” operations. AND OR operation To capture the nonlinear relationship between input features. The rule generation follows the following formula:

[0086]

[0087] By alternating between conjunctive normal form (CNF) and disjunctive normal form (DNF), the system can flexibly express the interaction between complex features. The neural network part shares parameters with the axiomatic fuzzy system, and optimizes the weights through backpropagation and gradient descent algorithms to further improve model performance. The gradient calculation formula for weight update is as follows:

[0088]

[0089] in, is the learning rate, is the loss function, and the mean square error (MSE) and L2 regularization are combined in the optimization process to ensure the simplicity and accuracy of the model:

[0090]

[0091] is the regularization coefficient to prevent overfitting.

[0092] 3. Linear classification. The generated fuzzy rules are passed through the linear classification layer for final decision making. The linear classification layer identifies key features and classifies them by linearly combining feature weights. Its structure is inspired by Kolmogorov-Arnold Networks (KANS), which optimizes the classification process through learnable activation functions and simplifies the expression of complex features. In this way, the model can not only maintain high classification accuracy, but also has high computational efficiency.

[0093] 4. Rule extraction. After the classification results are generated, the system uses the rule extraction mechanism to express the complex decision-making process of the fuzzy logic layer in an "if-then" form, providing a clear basis for each classification. The rule extraction mechanism ensures the interpretability of the classification process, and users can understand the logic behind the model's decisions by viewing the generated rules. By analyzing these rules, the system can not only achieve efficient classification, but also provide users with clear explanations to ensure model transparency.

[0094] In order to better demonstrate the workflow of our proposed classifier, we use the Iris dataset for illustration. The Iris dataset contains three classes, each with 50 samples, corresponding to a type of iris. and test set It is used for classification tasks. Describes sepal length, sepal width, petal length, and petal width. Simple concept set This corresponds to fuzzy classification of these features, such as "very small", "medium", and "very large".

[0095] Table 1: Simple concepts and their semantics defined for the Iris dataset

[0096]

[0097] The parameters are given in Table 1. and variance The Gaussian weight of each sample for different simple concepts can be calculated according to the following formula:

[0098] ,

[0099] Table 2: Generated semantic rules

[0100]

[0101] The rules in Table 2 are directly converted from the trained FRRL model and are used to describe the relationship between features and classification results.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the above embodiments describe the present invention in detail, it should be understood that a person skilled in the art can still modify the technical solutions or replace some of the technical features with equivalents. These modifications or replacements do not change the essence of the corresponding technical solutions, and still fall within the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A complex concept-based interpretable data representation learning method, characterized in that: The method comprises the following steps: S1. Fuzzification: The fuzzification layer processes data by converting the input feature vector into a fuzzy vector. For each feature in the data set, a set of simple concepts is defined to represent the fuzzy state of the feature. By dividing the value range of the feature, each feature is assigned multiple fuzzy concepts. The system generates a fuzzy representation by calculating the membership degree of the sample to the fuzzy concept. Assume the data set is and feature set ,in and Represent the number of samples and the number of features respectively; for each feature , defines a set of simple concepts ,in is the number of concepts; through the weight function , calculate the degree of association between each sample and the simple concept, the formula is as follows: ； in, is an adjustable parameter, , is the partition spacing, indicating the division of the eigenvalue range; Membership function Representation sample In the collection The membership degree in is: ； This formula measures the membership of a sample in a fuzzy set and represents the fuzzy association between the sample and a simple concept; S2. Fuzzy logic rule generation: The fuzzy logic layer combines the axiomatic fuzzy system and the neural network structure for processing. The axiomatic fuzzy system uses multi-level "AND" operations. AND or , automatically generates complex fuzzy rules to capture the nonlinear relationship between input features; by alternating between conjunctive normal form CNF and disjunctive normal form DNF, the interaction between complex features is expressed; the neural network part shares parameters with the axiomatic fuzzy system, and the neural network optimizes the weight parameters through back propagation and gradient descent algorithms; finally, the rule generation mechanism of the fuzzy logic layer effectively captures feature associations; The fuzzy logic layer includes two types: axiomatic fuzzy system and neural network; The multi-layer structure of axiomatic fuzzy system: the fuzzy logic layer consists of Layer axiomatic fuzzy system, each layer contains fuzzy" "Computational Layer and blur "Computational Layer , where odd-numbered layers execute " "Operation, even-numbered layers executed" "Operation; Node Represents fuzzy rules, nodes Represents a set of rules, and the operations are calculated by fuzzy and operator product and Max fuzzy or operator respectively; the formula for learning feature representation layer by layer is as follows: ； Neural network structure: The neural network part corresponds to the nodes of the axiomatic fuzzy system. The continuous weights within the interval are optimized; all layers are fully connected layers, and the logical activation function is used to simulate the "∧" operation and the "∨" operation: ； ； The gradient descent algorithm is used to update the weights, and the gradient is calculated by back propagation to update the weights of each layer. : ； in, is the learning rate, is the loss function; the loss function uses mean square error MSE combined with L2 regularization, and the formula is as follows: ； in, is the regularization coefficient, is the category label, is the predicted value of the model; S3. Linear classification: The linear classification layer makes decisions based on the new features generated by the fuzzy logic layer. The fuzzification layer and the fuzzy logic layer act as feature learners, generating feature representations through fuzzy rules, and the linear layer performs classification based on the feature representations. The linear layer of FRRL combines the fuzzification layer and the fuzzy logic layer as a feature learner and classifier; the linear layer identifies important features through weights and analyzes the corresponding fuzzy rules to explain the model decision; the linear layer uses KANS, and a learnable activation function is used at the edge of the network. Each weight is parameterized by a spline function; the model complexity is controlled by reducing the number of logic layers and nodes; the model complexity is limited by optimizing the loss function through L2 regularization, and the formula is as follows: ； is the regularization coefficient; S4. Rule extraction and simplification: By converting the complex decision-making process of the fuzzy logic layer into "if-then" rules; the rules express the relationship between input features and classification results, providing a clear and explainable basis for each classification; in the model, each path from the fuzzification layer to the linear classifier is represented as a fuzzy rule; For each input sample , and its corresponding rules are extracted as follows: ； is the characteristic value of the sample, Represents the fuzzy set membership function corresponding to each feature, which evaluates the degree to which the feature belongs to a fuzzy set; the conclusion part of the rule is the classification result given by the model. Multiple conditions are connected by "AND", which means that when all conditions are met at the same time, the sample Belong to a certain class; Simplify rules: merge rules with similar premises or replace some "AND" conditions with "OR". The simplified rules are: ； In this rule, some conditions are connected using "OR", which means that as long as any one of the conditions is met, it will be OK.

Images