Text classification model interpretable method based on input marginalization

Abstract

The invention discloses an input marginalization-based text classification model interpretable method, which comprises the following steps of: firstly, carrying out pre-processing operation on text data, training a to-be-interpreted model, constructing a word list, preliminarily screening candidate features by utilizing a local interpretable method irrelevant to the model and generating a to-be-marginalized sample; according to the method, samples are marginalized through a mask language model and model feedforward, the importance of features is predicted, the calculation cost is saved through threshold truncation in the process, and finally an interpretation result is output through weight combination. Meanwhile, a new interpretation stability index is provided as an interpretable algorithm evaluation index, the index not only considers feature level problems such as a feature order relation and interpretation flipping, but also pays attention to adjacent interpretation feature weight difference values, and multi-dimensional and refined evaluation can be carried out on an algorithm interpretation result. According to the method, the stability of an interpretable algorithm of the text classification model is remarkably improved, and the scientificity of an interpretation result is improved.

Description

Technical Field

[0001] The present invention relates to an interpretable method for a text classification model in the technical field of natural language processing, and in particular to an interpretable method for a text classification model based on input marginalization. Background Art

[0002] With the explosive growth of information and the rapid development of artificial intelligence technology, the use of deep models to achieve automatic text classification has gradually replaced the work of manual annotation and classification of data. Text classification models need to be deployed in multiple application scenarios such as recommendation systems, sentiment analysis, and literature review. However, due to the lack of interpretability of deep models, their inherent black box characteristics severely limit the transparency, fairness, and accountability of artificial intelligence systems that rely on these models, making them difficult to deploy and apply in high-risk fields such as law, regulation, medicine, and finance.

[0003] In order to meet this challenge, many researchers have conducted extensive research on how to improve model interpretability, and eXplainable AI (XAI) has emerged as a discipline. XAI focuses on understanding why the model makes corresponding predictions. As the complexity of black box models continues to develop and is applied in key areas, it is imperative to provide explanations for model predictions through XAI.

[0004] Among the many interpretable methods, the most popular one is the post-interpretable method. Among them, the local interpretable model-agnostic explanations (LIME) explanation method has become one of the most commonly used interpretable tools in the XAI framework, and is widely used in many fields such as recommendation systems, biomedicine, and industry. However, due to its random perturbation sample generation method, the method itself still has shortcomings in terms of stability and scientificity. Among them, stability means that two consecutive explanations of the same sample to be explained should obtain the same result, and LIME has the defect of poor stability due to the randomness of virtual sample generation, which will lead to the lack of credibility of the algorithm; scientificity means that the samples generated by its random deletion perturbation method are not in the distribution range of the original training data of the classification model, which makes the interpretable algorithm produce misleading explanations and hinders the identification of model errors and biases.

[0005] In addition, the evaluation of interpretable algorithms is also an important issue. When evaluating the stability of interpretable algorithms, indicators such as the feature stability index are often used to evaluate the stability of interpretable algorithms. Some studies have proposed improvement directions for the defects of existing indicators, and additionally considered the order relationship and feature flipping of features, but the feature weight difference that can quantitatively measure the stability of adjacent explanations is still not considered.

[0006] In order to solve the instability problem of local interpretability methods, Sharath et al. (Shankaranarayana SM, Runje D. ALIME: Autoencoder based approach for local interpretability [C] / / Intelligent Data Engineering and Automated Learning – IDEAL 2019: 20th International Conference, Manchester, UK, November 14–16, 2019, Proceedings, Part I 20. Springer International Publishing, 2019: 454-463) used an autoencoder to map the data from the original feature space to the latent space and then weighted it. Finally, they selected the neighborhood of the data to be explained according to the sample weight, thereby improving the local fidelity of LIME explanation. Zhou et al. (Zhou Z, Hooker G, Wang FS-lime: Stabilized-lime for model explanation [C] / / Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2021: 2429-2438.) proposed stability LIME, which uses a hypothesis testing framework based on the central limit theorem to determine the number of data points that need to be perturbed to ensure the stability of the results. Zafar et al. (Zafar MR, Khan N. Deterministic local interpretable model-agnostic explanations for stable explainability [J]. Machine Learning and Knowledge Extraction, 2021, 3 (3): 525-541.) proposed deterministic LIME, which first clusters the data using hierarchical clustering (HC), and then uses K-nearest neighbor (KNN) to select relevant clusters of the instances to be explained to replace the random perturbation sampling method of LIME, thereby improving the stability of LIME explanation.In the context of text data, Burger et al. (Burger C, Chen L, Le T. "Are Your Explanations Reliable?" Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attack [C] / / Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023: 12831-12844.) evaluated the inherent instability of LIME on text data, proposed a new algorithm XAIFOOLER to perturb text input and explanations, and introduced Rank Biased Overlap (RBO) as a key part to guide the optimization of XAIFOOLER to meet all requirements for explanation similarity measurement. In the face of the OOD problem caused by the zero-erasure method in the algorithm, Chang et al. (Chang CH, Creager E, Goldenberg A, et al. Explaining image classifiers by counterfactual generation [J]. arXiv preprint arXiv: 1807.08024, 2018.) proposed FIDO, which marginalizes the calculation of credible alternative inputs and significantly improves the existing methods. In the text field, Jin et al. (Jin X, Wei Z, Du J, et al. Towards hierarchical importance attribution: Explaining compositional semantics for neural sequence models [J]. arXiv preprint arXiv: 1911.06194, 2019.) proposed the SCD and SOC algorithms, which marginalize the context words and can generate information-rich hierarchical explanations, but because their core is still the zero-erasure scheme, they still cannot overcome the OOD problem.Kim et al. (Kim, S., Yi, J., Kim, E., & Yoon, S. Interpretation of NLP Models through Input Marginalization [C] / / Conference on Empirical Methods in Natural Language Processing (2020).) focused on the widely used zero-erasure scheme, which explains NLP models through input marginalization to alleviate the OOD problem and produce a more faithful explanation. However, this method requires marginalized prediction of all features of the sample, which is computationally inefficient.

[0007] In response to the above problems, this paper proposes an interpretable method for text classification models based on input marginalization, which significantly improves the stability of the algorithm and enhances the scientific nature of the explanation, making the explanation results more credible. First, LIME is used to quickly extract more important features, and samples to be marginalized are assembled according to the extracted features. The MLM module is used for prediction, and the instability and OOD problems of the model are solved by external component enhancement. At the same time, the threshold truncation is introduced to save computing costs and balance the experimental results with the computing cost. At the same time, in view of the many problems existing in the evaluation indicators of the existing interpretable methods, an explanation stability index is proposed, which can make a more multi-dimensional and accurate evaluation of the interpretable algorithm.

[0008] In summary, the interpretable method of text classification models based on input marginalization is an innovative research issue with important research significance. Summary of the invention

[0009] Model interpretability is a significant problem in text classification tasks and is also a hot topic in current artificial intelligence research. However, the most widely used model-independent local interpretable algorithms are seriously unstable, which makes the interpretation results obtained by this type of interpretable algorithm lack credibility, and the samples generated by its random deletion perturbation method are not in the distribution range of the original training data of the classification model, which makes the interpretable algorithm produce misleading interpretations. Therefore, this paper proposes an interpretable method for text classification models based on input marginalization. First, LIME is used to screen more important features, and the constructed marginalized samples are input into MLM to obtain the feature importance value, and then the combined weight calculation is performed to obtain the interpretation result. In the process, a threshold truncation is set to save computing cost. At the same time, in the evaluation stage of the interpretation result, a more comprehensive and accurate interpretation stability index is proposed.

[0010] The interpretable method of text classification model based on input marginalization includes:

[0011] (1) Data preprocessing module

[0012] Data preprocessing mainly involves cleaning the text dataset, removing stop words, and obtaining standardized samples and labels to train the text classification model as the model to be explained. At the same time, a vocabulary vocab is constructed for edge computing.

[0013] (2) Evidence feature extraction module

[0014] The LIME algorithm is used to perform preliminary evidence feature extraction, and the random deletion mechanism is used to quickly filter out features that have little impact on the model decision, forming a more important feature set, and then constructing a sample set to be marginalized based on the source text.

[0015] The specific process of the method (2) includes the following steps:

[0016] The S1 evidence feature set is extracted by LIME through the perturbation method of random deletion. In this paper, the text sequence T is taken as the sample to be explained. LIME performs interpretable analysis on the classification model based on random perturbation of T, and obtains the evidence feature set E L and the corresponding feature importance weight set W L The list of interpretation results [E L ,W L ].

[0017] S2 replaces the k copies of the source text in sequence based on the position of the obtained feature set in the source text, and replaces the token at the corresponding position with [MASK] to obtain k samples to be marginalized.

[0018] (3) Marginalization module and threshold truncation

[0019] The constructed sample sets to be marginalized are input into the MLM module for prediction. Each important feature is replaced by a predefined vocabulary and feed-forward predicted, and the marginalization results are obtained through the evidence weight algorithm.

[0020] S1 inputs the samples to be marginalized into the MLM module, replaces the [MASK] positions one by one based on the predefined vocabulary vocab and performs mask prediction.

[0021] S2 introduces a threshold truncation function to truncate some tokens that have little impact on the total weight of evidence. These features and corresponding samples will not enter the subsequent calculation process.

[0022] After obtaining the truncated MLM prediction results and the corresponding sample set T, S3 inputs T into the target model for feedforward to obtain the model prediction results, and inputs the prediction results and the model feedforward results into the WoE (Weight of Evidence) evidence weight algorithm for calculation.

[0023] S4 combines the features and importance weights of the marginalized results with the corresponding items of the LIME explanation results through weighted averaging, reorders and intercepts them to obtain the final explanation result [E, W].

[0024] (4) Algorithm evaluation indicators

[0025] The comparative experiment in the algorithm evaluation section considers the algorithm stability and algorithm effectiveness. The stability experiment compares the FSI, FSSI and ESI indicators proposed in this paper, and the effectiveness is compared through the AUC indicator. In addition, in order to verify the efficiency of the algorithm, experiments are designed to consider the degree to which different modules improve the algorithm's computational efficiency, and the hyperparameters used in threshold truncation are experimentally analyzed. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Diagram of the interpretable algorithm structure of the text classification model based on input marginalization

[0027] Figure 2 Comparison chart of the stability experiment results of interpretable methods

[0028] Figure 3 Comparison chart of the effectiveness of the interpretable method experiment

[0029] Figure 4 Figure 1. Setting up the efficiency experiment for interpretable methods.

[0030] Figure 5 Comparison chart of the experimental results of the high efficiency of the interpretable method

[0031] Figure 6 The experimental results of average score ratio and marginalization number under different thresholds are shown in Figure 2. DETAILED DESCRIPTION

[0032] In order to make the technical solution of the present invention clearer, the specific implementation methods in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0033] This paper proposes an interpretable method for text classification model based on input marginalization. The specific process is as follows: Figure 1 The method is characterized in that given a segment of samples to be interpreted T = [t1, t2, ..., t n ], where t n represents the nth token in the text segment, f is a text classification model, which may include conventional text classification models such as convolutional neural network models and long short-term memory network models. The sample to be interpreted is input into the LIME algorithm to obtain a list of the more important top-k tokens [t L1 ,t L2 ,…,t Lkand the corresponding importance weight list [w L1 , w L2 , …, w Lk as the result of the LIME part. Then, based on the top-k token list [t L1 , t L2 , …, t Lk , replace the tokens at the corresponding positions in the source text with [MASK] respectively to obtain the sample set to be marginalized consisting of k samples Replace through the Vocab vocabulary to Obtain the MLM prediction result through masked prediction And construct based on the predefined vocabulary Input it into the model f for feedforward to obtain the model prediction result

[0034] After obtaining the masked language prediction And the confidence of the model feedforward prediction , input the above results into the evidence weight algorithm to obtain the marginalized result, and perform the above operations on the top-k features respectively to obtain the marginalized evidence feature weight list [W′1, W′2, …, W′ K . Finally, calculate the combined weight of LIME and the marginalized result, reorder and intercept the top p (p < k) important features to obtain a set of evidence features E = [E1, E2, …, E p and the corresponding feature importance weights W = [W1, W2, …, W p . The algorithm result is presented as an explanation result L = [E, W] consisting of two sets of explanation lists, where E is a set of evidence features with importance decreasing from high to low, and W is the corresponding set of feature importance weights

[0035] Based on the input marginalized text classification model interpretable method, specifically including:

[0036] (1) Data preprocessing module

[0037] Data preprocessing mainly cleans and removes stop words from the text data set containing n pieces of data to obtain the standardized sample x i and label y i to train the text classification model and use it as the sample to be explained for subsequent explanations

[0038] (2) Evidence feature extraction module

[0039] The model-independent locally interpretable LIME algorithm is used to extract evidence features from the samples to be explained. For the i-th text sequence with an input length of n, the more important top-k features and the corresponding feature weight sets are quickly screened out through random deletion perturbation. The combined output is a list of explanation results that serve as the basis for the target model to map the text sequence to the target label.

[0040] Specifically, when LIME is applied to the explanation task of the text classification model, for the i-th text sequence T with an input length of n i =[t i1 ,t i2 ,…,t in ], select the top-k features E through feature selection L =[t L1 ,t L2 ,…,t Lk ] and the corresponding feature weight set W L =[w L1 ,w L2 ,…,w LK ] and output the LIME explanation results in list form L = [[t L1 ,t L2 ,…,t Lk ],[w L1 ,w L2 ,…,w Lk ]], as a classification model, maps text sequences to the set of marginalized explanations of corresponding labels.

[0041] (3) Input marginalization module

[0042] In order to weigh the contribution of all possible candidate tokens and consider their likelihood, the probability of a token appearing at that position is inferred through MLM represented by BERT's bidirectional encoder and the predefined vocabulary vocab. MLM can accurately obtain the probability of candidate tokens, thereby alleviating the OOD problem caused by the random deletion perturbation sample generation method through marginalization. In the specific implementation, it is necessary to perform marginalization calculations on all possible candidate tokens through BERT's MLM in the part of the evidence weight algorithm that measures the attribution score of the token.

[0043] The traditional method deletes the token observation model and does not include the i-th token (i.e., t i ) of samples to be explained T -i Prediction is y c The probability of class p(y c |T -i ) to examine the importance of tokens for model prediction, y cIndicates the category predicted by the model. However, since deleting the samples generated by the token will lead to OOD problems, this paper introduces MLM for marginalization operation, and the formula for examining the importance of tokens is expressed as follows:

[0044]

[0045] Further expansion:

[0046]

[0047] Among them, V is the predefined candidate word list vocab, and v is the word in the word list that replaces t. i The first part of the above formula represents the prediction of the word appearing in t through MLM. i The second part represents the possibility of the word replacing t i The resulting sample is predicted by the classification model as y c Specifically, t i Replace it with the [MASK] tag and predict it through the mask to get the first part of the above formula. Replace the [MASK] tag position with v as the sample input to the classification model to get y c The classification confidence of the class can be obtained by the second part of the above formula, and t is obtained by looping and calculating all candidate words v in V. i The attribution score.

[0048] However, since the input marginalization process requires marginalization of the entire vocabulary, it will take up a lot of computing time, making the model output results abnormally slow, so it is necessary to balance the algorithm effect and the computational cost. Unlike marginalizing all features in the sample, the proposed algorithm extracts the top-k features through LIME to obtain the candidate feature set before the input marginalization step, which preliminarily controls the computational cost. In the input marginalization part, the threshold σ is introduced to truncate tokens that do not contribute much to the sum of evidence weights, and only tokens with a probability greater than the threshold are marginalized and normalized, so it is further rewritten:

[0049]

[0050] in, To filter out the remaining candidate words whose mask prediction value is greater than the threshold σ. For different data sets, the optimal threshold selection may vary accordingly. Usually, σ is 10 -4 ~10 -5 By selecting a suitable threshold according to the proportion of partial marginalization scores and the average number of marginalization times under different threshold settings, the computational complexity of the marginalization process can be greatly reduced while ensuring that the algorithm results are not much different from the complete marginalization effect.

[0051] This paper adopts the widely used WoE algorithm in the weight of evidence algorithm:

[0052] WoE i (y c |T)=log2(odds(y c |T))-log2(odds(y c |T -i )) (4)

[0053] Formula (4) indicates that when the i-th token in the sample to be explained T is included or not included in T, the classification model predicts the sample as y c The logarithmic probability difference of the class probability. The first term of this formula is the original predicted probability, which can be obtained by feeding forward the original model, while the second term is obtained using the marginalization calculation method mentioned above after threshold truncation. Among them:

[0054]

[0055] Formula (5) is defined as predicting the sample to be explained T as y c The ratio of the probability of a class event to the probability of the event being denied is the probability. c |T -i ):

[0056]

[0057] (4) Algorithm evaluation indicators

[0058] S1 In terms of stability, traditional evaluation indicators mainly evaluate the stability of interpretable explanations based on the returned explanation list. For two consecutive explanations of a sample, the following indicators are used to consider the stability of the explanations:

[0059] The Jaccard coefficient represents the similarity between two sets of explanations, expressed as the difference between the two sets of explanation feature sets:

[0060]

[0061] Among them, e i and e j The points represent the list of evidence features in the adjacent interpretation results.

[0062] The FSI calculates the overall average value of the Jaccard coefficient at the sample set level:

[0063] FSI = Average {J(e i ,e j )} (8)

[0064] Some studies have improved the above indicators, taking into account the order relationship between features and explanation flipping. Only when the corresponding feature items in the explanation result sequence are the same and the signs of the feature weights are the same, the adjacent explanation results of the instance in the experiment are considered stable. The larger the FSSI value, the higher the stability of the explanation:

[0065]

[0066] in, and represents the i-th evidence feature in the adjacent explanation result, and represents the weight of the i-th evidence feature in the adjacent explanation result, and They are the adjacent explanation results of the interpretable algorithm for the same sample.

[0067] The explanation stability index (ESI) proposed in this paper adds a feature weight difference penalty term to the FSSI stability index. In order to distinguish the difference in weights of neighboring explanation features. In the experiment, it was observed that the difference in weights of neighboring explanation features is usually distributed in the interval [0, 0.1]. When the difference penalty coefficient λ is selected in the interval [30, 50], it can effectively reflect the stability of the neighboring explanation results:

[0068]

[0069] In terms of effectiveness, S2 takes the original instance input, gradually deletes features according to the explanation list, and observes the probability curve. A more accurate explanation should lead to a faster decrease in the confidence score of the curve, that is, a lower area under the output probability curve (AUC). The calculation method for AUC in this article is as follows: multiple samples are obtained by deleting a single sample multiple times, and the classification confidence interval of the original sample is mapped to the [0,1] interval. The number of features that are gradually deleted is also mapped to the [0,1] interval. The integral of the confidence-feature number curve is calculated to obtain AUC. The following is a simplified expression of AUC:

[0070]

[0071] Among them, AUC represents the area under the curve, and S(t) represents the confidence score when t tokens are deleted.

[0072] Since the confidence score changes are not continuous but discrete, the actual integral cannot be calculated, so the AUC needs to be calculated at discrete points through numerical integration. Using the trapezoidal rule, the AUC can be approximated as the sum of the trapezoidal areas, and the calculation formula is:

[0073]

[0074] Among them, n represents the total number of features, S i Indicates that the confidence score at the i tokens has been deleted.

[0075] This paper mainly uses the two indicators FSI and FSSI commonly used in explanation algorithms and the ESI proposed in this paper to evaluate stability, and compares the effectiveness of the explanation algorithms through the AUC indicator.

[0076] The data set used in the present invention comes from two standard text classification data sets: Stanford Sentiment Treebank binary classification corpus SST-2 (Stanford Sentiment Treebank) and IMDB Movie Reviews data set.

[0077] The SST-2 dataset contains movie reviews from the movie review website Rotten Tomatoes, where the regression score of each sentence-level example is binarized to form a binary classification task, labeled as positive sentiment (positive, sample label corresponding to 1) and negative sentiment (negative, sample label corresponding to 0). The dataset is split into 67,350 training samples and 871 test samples.

[0078] The IMDB dataset is a standard binary classification dataset for sentiment analysis. It contains 50,000 reviews from the Internet Movie Database (IMDB), which are divided into positive or negative reviews based on a 10-point rating system. Negative reviews are rated between 0 and 4 points, while positive reviews are rated between 7 and 10 points. In the experiment, the dataset is split into 25,000 training samples and 2,500 test samples.

[0079] like Figure 2As shown in the figure, in order to verify the effectiveness of the proposed interpretable method for text classification model based on input marginalization in improving the stability compared with the original method LIME, Random Forest, Logistic Regression and Adaboost models were selected as classification models, the final interpretation length p of the algorithm was 6, the number of random forest base evaluators was 300, and the number of Adaboost base evaluators was 500. The maximum number of features k of LIME was set to 10, the LIME sampling rate was set to 5000, and the sample weighted distance metric used cosine. FSI, FSSI and the proposed ESI were selected as stability experimental evaluation indicators for comparison. Through comparative experiments, it can be seen that under multiple indicators, for different text classification models and data sets, our method has achieved the best results, and the stability of the algorithm has been greatly improved. And under the premise of ensuring that the interpretation feature set is basically consistent, the change of the feature weight difference can be effectively controlled.

[0080] like Figure 3 As shown in the figure, in order to verify whether the proposed interpretable method for text classification model based on input marginalization is more scientific than the original method LIME, the effectiveness experiment of interpretable algorithm is also conducted on the above classification models and datasets. The experimental results show that the AUC value of the proposed method is the lowest in the average AUC value of 2500 IMDB sentences and 871 SST-2 sentences, which means that this method can capture the importance of tokens more accurately than LIME, further improving the credibility of the interpretation results.

[0081] like Figure 4 As shown in the figure, in order to verify the efficiency of the proposed method, an ablation experiment is set up from the perspective of time cost. LIME-MLM without the LIME module for selecting top-k features and threshold truncation (named E1), E1 with threshold truncation (named E2), LIME-MLM model without threshold truncation (named E3), and the LIME-MLM proposed in this paper (named E4) are compared on the IMDB dataset through three classification models.

[0082] The interpretation results are compared and analyzed by the average time taken for a single interpretation by the interpretation algorithm. Figure 5It can be seen that under the same experimental conditions, E1 takes the longest time, the threshold truncation module improves E1 and E3 by an average of 92.6% and 84.5% respectively, and LIME as a feature screening module improves E1 and E2 by an average of 94.8% and 85.7% respectively. In summary, in terms of time cost, LIME as a fast evidence feature extraction module and the threshold truncation part as a process to save computing resources are necessary, so that the algorithm can generate stable and effective explanation results while also having high operating efficiency, providing strong support for the practical application of interpretable algorithms.

[0083] In order to verify whether the selection of the introduced hyperparameter threshold σ is scientific and reasonable, by setting multiple different thresholds σ, observe the number of marginalizations that the algorithm needs to calculate after the fixed threshold is cut off and the corresponding proportion of the score of partial marginalization to complete marginalization. Randomly extract samples from the data set for the experiment and observe the experimental results.

[0084] like Figure 6 As shown in Figure 2, the proportion of partial marginalization scores and the average marginalization times under different threshold settings. It can be seen that when σ=10 -4 and 10 -5 When σ = 10, the results are very similar to those of complete marginalization, but the average number of marginalizations is reduced to 3.4% and 27.8% of complete marginalization, respectively. The number of marginalizations required for complete marginalization is 30522. -4 As the optimal threshold selection, the addition of this threshold enables the algorithm to obtain results very similar to complete marginalization with a smaller number of marginalizations. Threshold truncation can greatly reduce the computational complexity of the algorithm and make the algorithm have higher operating efficiency.

[0085] Aiming at the unstable interpretation and OOD problems in the interpretable tasks of text classification models, this paper proposes a new model-independent local interpretable algorithm. The stability of the algorithm is improved by external component enhancement, and the scientific nature of the interpretation results is further increased. At the same time, thanks to the setting of threshold truncation, the computational cost consumed by the algorithm is greatly reduced, providing an effective solution for the engineering application of the algorithm in the interpretable text classification model.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for interpreting a text classification model based on input marginalization, characterized by: Given a segment of samples to be explained T = [t1, t2, ..., t n ], where t n represents the nth token in the text segment, f is the text classification model, and the sample to be interpreted is input into the LIME algorithm to obtain a list of the more important top-k tokens [t L1 , t L2 , ..., t Lk ] and the corresponding importance weight list [w L1 , w L2 , ..., w Lk ] as the result of the LIME part; then based on the top-k token list [t L1 , t L2 , ..., t Lk ]Replace the corresponding positions of tokens in the source text with [MASK] to obtain a sample set to be marginalized consisting of k samples Will After the Vocab vocabulary is replaced by MLM prediction results are obtained after mask prediction And build on predefined vocabulary Input it into the model f and feed forward to get the model prediction result Getting the masked language prediction Feedforward prediction confidence with model Finally, the above results are input into the evidence weight algorithm to obtain the marginalized results. Perform the above operations separately to obtain the marginalized evidence feature weight list [W′1, W′2, ..., W′ K ]; Finally, LIME and marginalized results are combined and weighted, and the first p important features are reordered and intercepted to obtain a set of evidence features E = [E1, E2, ..., E p ] and the corresponding feature importance weights W = [W1, W2, ..., W p ], where p<k; the algorithm result is expressed as an explanation result L=[E,W] consisting of two sets of explanation lists, where E is a set of evidence features with decreasing importance, and W is a set of corresponding feature importance weights; The interpretable method of text classification model based on input marginalization includes: (1) Data preprocessing module Data preprocessing mainly involves converting a text dataset containing n pieces of data Perform data cleaning, remove stop words, and obtain a standardized sample x i and label y i To train text classification models and use them as samples to be explained for subsequent explanation; (2) Evidence feature extraction module When LIME is applied to the explanation task of a text classification model, for the i-th text sequence T of input length n i =[t i1 , t i2 , ..., t in ], select the top-k features E through feature selection L =[t L1 , t L2 , ..., t Lk ] and the corresponding feature weight set W L =[w L1 , w L2 , ..., w Lk ] and output the LIME explanation results in list form L = [[t L1 , t L2 , ..., t Lk ],[w L1 , w L2 , ..., w Lk ]], as a classification model, maps text sequences to the set of marginalized explanations of corresponding labels; (3) Input marginalization module Introducing MLM for marginalization operation, the formula for examining the importance of tokens is expressed as follows: Further expansion: Among them, V is the predefined candidate word list vocab, and v is the word in the word list that replaces t. i The first part of the above formula represents the prediction of the word appearing in t through MLM i The second part represents the possibility of the word replacing t i The resulting sample is predicted by the classification model as y c class; specifically, t i Replace it with the [MASK] tag and predict it through the mask to get the first part of the above formula. Replace the [MASK] tag position with v as the sample input to the classification model to get y c The classification confidence of the class can be obtained by the second part of the above formula, and t is obtained by looping and calculating all candidate words v in V. i 's attribution score; In the input marginalization part, a threshold σ is introduced to cut off the tokens that do not contribute much to the sum of the evidence weights, and only tokens with a probability greater than the threshold are marginalized and normalized: in, To filter out the remaining candidate words whose mask prediction value is greater than the threshold σ; σ is 10 -4 ~10 -5 between; The WoE algorithm is used in the weight of evidence algorithm: WoE i (y c |T)=log2(odds(y c |T))-log2(odds(y c |T -i )) (4) Formula (4) indicates that when the i-th token in the sample to be explained T is included or not included in T, the classification model predicts the sample as y c The logarithmic probability difference of the class probability; the first term of this formula is the original predicted probability, which can be obtained by feeding forward the original model, while the second term is obtained using the marginalization calculation method mentioned above after threshold truncation; where: Formula (5) is defined as predicting the sample to be explained T as y c The ratio of the probability of the event of the class to the probability of the negation of the event is the probability; and we get odds(y c |T -i ):

Images