A Convolutional Neural Network Hidden Layer Interpretation Method

Abstract

The present invention relates to a convolutional neural network hidden layer interpretation method, the method comprising the following steps: obtaining a hidden layer activation value set of a convolutional neural network model; clustering the activation value set once to obtain M different clusters; performing secondary clustering on each cluster, obtaining a number of factors for each cluster, assigning an ID to each factor, and using the ID as a label; constructing M feature conversion classifiers, using the original sample feature as input and the label as output for each feature conversion classifier, training the feature conversion classifier; inputting the original sample into the trained M feature conversion classifiers to obtain M-dimensional features; using the M-dimensional new features as input and the original sample labels as output, and using a C4.5 decision tree to build a classification model. The method corresponds the neural network classification process to the human cognitive process, and improves the readability of the interpretation information.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a convolutional neural network hidden layer interpretation method. Background Art

[0002] At present, there are mainly hidden layer representation interpretation methods, interpretable convolutional neural network (CNN) construction methods and model imitation interpretation methods to study the interpretability of neural networks. In the research using the model imitation interpretation method, most of them use interpretable models to replace certain parts of the neural network or learn a new alternative model from the neural network to obtain some classification rules from the neural network. However, these rules are often obtained by simulating the input and output of CNN by the model, and do not reflect the changes inside the neural network. They are also different from human cognitive methods and are not intuitive enough. Summary of the invention

[0003] In view of this, the main purpose of this application is to provide a convolutional neural network hidden layer interpretation method, the method comprising the following steps:

[0004] S100, obtaining a set of hidden layer activation values ​​of a convolutional neural network model;

[0005] S200, clustering the activation value set once to obtain M different clusters;

[0006] S300, performing secondary clustering on each cluster, obtaining a number of factors for each cluster, assigning an ID to each factor, and using the ID as a label;

[0007] S400, constructing M feature conversion classifiers, and for each feature conversion classifier, using the original sample feature as input and the label as output, training the feature conversion classifier;

[0008] S500, inputting the original sample into the trained M feature conversion classifiers to obtain M-dimensional features;

[0009] S600, taking the M-dimensional features as input and the original sample labels as output, and using the C4.5 decision tree to build a classification model;

[0010] M is an integer greater than 0.

[0011] Preferably, in the method, the activation value set is obtained by the following steps:

[0012] S101, obtaining a trained convolutional neural network model and using it as an original model;

[0013] S102, constructing a network with the original model input as input and the selected hidden layer as output, and constructing an interception model;

[0014] S103: Based on the truncation model, sample data is input to obtain an activation value set.

[0015] Preferably, in the method, S102 uses the Model function provided by Keras to construct the interception model.

[0016] Preferably, in the method, the primary clustering and the secondary clustering adopt a density-based spatial clustering algorithm in noisy applications.

[0017] Preferably, in the method, S400 adopts a random forest model to obtain a classifier.

[0018] Preferably, in the method, the random forest model performs feature transformation interpretation through the following steps:

[0019] S401, traverse T trees in the random forest model to obtain all classification paths of each tree;

[0020] S402, after the sample data is input, the variable discrimination criteria of each node classification corresponding to each classification path are combined into a variable rule;

[0021] S403, extracting T variable rules from the T classification paths, merging the rules of the same variables, and arranging them in order to obtain a sorted classification rule set, and then obtaining the maximum compatible rule;

[0022] S404: Reconstruct samples based on the maximum compatibility rule.

[0023] Preferably, in the method, the maximum compatibility rule is obtained by the following steps:

[0024] S4031. Create a blank list;

[0025] S4032. For each variable rule, determine whether it already exists in the list; if not, execute S4033; otherwise, execute S4034;

[0026] S4033, taking the union of the variables involved and adding them into the list;

[0027] S4034. Take the intersection of the variables involved to cover all the rules and add them into the list.

[0028] Preferably, in the method, the sample reconstruction comprises the following steps:

[0029] S4041. Construct a zero matrix with the same size as the original sample;

[0030] S4042, taking corresponding special values ​​of the variables involved in the rule, and constructing a special instance that satisfies the maximum compatibility rule;

[0031] The special value includes taking the median value, the maximum value or using a random value.

[0032] Compared with existing technologies:

[0033] The present invention matches the neural network classification process with the human cognitive process to improve the readability of the explained information. The model classification accuracy is improved by optimizing the feature conversion and feature classification methods. Furthermore, the random forest classifier in the feature conversion process is subjected to rule extraction and sample reconstruction to improve the model interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 , a schematic diagram of the process of the present invention;

[0036] Figure 2 , Schematic diagram of human decision-making model;

[0037] Figure 3 , intercept the model schematic diagram;

[0038] Figure 4 ,Schematic diagram of the DTOHL model establishment process;

[0039] Figure 5 , schematic diagram of random forest classification path;

[0040] Figure 6 , schematic diagram of the reconstruction effect of pneumonia X-ray image samples;

[0041] Figure 7 ,for Figure 6 Schematic diagram of pneumonia X-ray images;

[0042] Figure 8 , schematic diagram of the reconstruction effect of normal X-ray image samples;

[0043] Fig. 9 ,for Figure 8 Schematic diagram of a normal X-ray image. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0045] In one embodiment, hidden layer interpretation is performed for convolutional neural network model classification.

[0046] First, the image set involved in the study is cleaned and screened to remove images with unclear or unreadable sample features. Depending on the situation, the image set is standardized by means of completion, cropping, or stretching, that is, the images are scaled to the same size. After the standardized images are converted into matrices, the original training set, validation set, and test set D0 required for model establishment are obtained.

[0047] Secondly, with the help of the mature deep learning framework Keras, through the sequential model Sequential serialization method, use the add() function to add network elements to build a convolutional neural network model. Use the data set D0 for model training, and use Grid-search to tune the parameters until the model accuracy reaches the highest value, and obtain the convolutional neural network model to be studied.

[0048] Next, the following method steps are used to establish a classification model, which is recorded as DTOHL (Decision Tree of Hidden Layer), to study how the specific hidden layer in the Convolutional Neural Network (CNN) obtained above classifies samples. The method includes the following steps:

[0049] S100, obtaining a set of hidden layer activation values ​​of a convolutional neural network model;

[0050] S200, clustering the activation value set once to obtain M different clusters; M is an integer greater than 0;

[0051] S300, performing secondary clustering on each cluster, obtaining a number of factors for each cluster, assigning an ID to each factor, and using the ID as a label;

[0052] S400, constructing M feature conversion classifiers, and for each feature conversion classifier, using the original sample feature as input and the label as output, training the feature conversion classifier;

[0053] S500, inputting the original sample into the trained M feature conversion classifiers to obtain M-dimensional features;

[0054] S600, taking the M-dimensional features as input and the original sample labels as output, and using the C4.5 decision tree to build a classification model.

[0055] The above method transforms the feature space of the data and converts the original sample features into a simplified effective information combination. This process is the same as the human decision-making process. When humans make decisions, they will transform external information into a self-cognitive framework and then make judgments. Figure 2 As shown in Figure 2, for the model, it is also the effective information of the sample that helps in classification.

[0056] In the method, in order to obtain the activation value set conveniently, the trained convolutional neural network model is used as the original model, and a network model with the original model input as input and the selected hidden layer as output is constructed, and this model is used as the intercepted model. Specifically, taking a fully connected layer in the convolutional neural network as an example, using the Model function provided by Keras to construct a network with the original model input as input and the selected hidden layer as output is equivalent to intercepting all structures before the hidden layer in the original model, such as Figure 3 As shown. By passing all samples through the intercepted model, we can get the activation value set.

[0057] Preferably, in the method, the primary clustering and the secondary clustering use a density-based spatial clustering algorithm in noisy applications (DBSCAN for short). Specifically, the activation value set is clustered using the DBSCAN algorithm and defined as different clusters F, the number of which is recorded as M. The DBSCAN algorithm is used to perform secondary clustering within each cluster to generate a number of factors, and an ID is assigned to each factor.

[0058] Preferably, the S400 uses a random forest model to obtain a classifier. Specifically, the original sample features in D0 are used as input, the clustering result ID is used as a label, and M feature conversion classifiers are trained using a random forest model in M ​​clusters. After all samples pass through the trained feature conversion classifier, M-dimensional features are obtained.

[0059] With the M-dimensional features as input and the original sample labels as output, a classification model is built using the C4.5 decision tree to obtain a DTOHL (Decision Tree of Hidden Layer) model, which can fit the process of humans converting sample data into effective information and then making decisions.

[0060] If a random forest classifier is used, the random forest model needs to be further interpreted by feature transformation, including the following steps:

[0061] S401, traverse T trees in the random forest model to obtain all classification paths of each tree;

[0062] S402, after the sample data is input, the variable discrimination criteria of each node classification corresponding to each classification path are combined into a variable rule;

[0063] S403, extracting T variable rules from the T classification paths, merging the rules of the same variables, and arranging them in order to obtain a sorted classification rule set, and then obtaining the maximum compatible rule;

[0064] S404: Reconstruct samples based on the maximum compatibility rule.

[0065] In one embodiment, the above method is used to study the interpretability of classifiers obtained by applying a random forest model. For a given random forest model with T trees, a depth-first traversal is performed on the branches of each tree in the random forest model, and all classification paths of a tree are obtained by traversing from the root node to the leaf node in the order of traversing the left subtree first and then the right subtree.

[0066] Next, after the sample data is input into the random forest model, the corresponding sample features are compared one by one against the classification path to obtain the decision path through each tree. This path corresponds to the variable discrimination criteria for each node classification, that is, it corresponds to a whole symbol rule. For one sample, T classification paths can be obtained, such as Figure 4 shown.

[0067] Then extract T variable rules from T classification paths, for example, the following rules are obtained:

[0068]

[0069]

[0070]

[0071] It is rather confusing to directly obtain the variable rules from the model. There may be multiple nodes containing rules for one variable or branches that do not contain classification rules for some variables. Therefore, we first organize the rules, merge the rules involving the same variables, and arrange the variables in order. The organized classification rule set is as follows:

[0072]

[0073] RULE2: (x1≥0.5)∧(x2<2.5)∧(x3≥4)

[0074] RULE T (x1<2)∧(x2<2.8)∧(x4≥3)

[0075] The same sample and the same variable can meet different classification rules in T trees, which proves that although the scope of the rules is different, the classification paths of each tree have intersections that can satisfy the sample. Therefore, a blank list L is set based on the T group rule set, and the T group rules are traversed. For the variables that have not appeared in L, the union is taken and added, and the variables that already exist in L are taken to cover all rules. The maximum compatibility rules (Maximum Compatibility Rules, MCR) of the sample in the random forest model can be obtained, as shown below:

[0076] RULE: (0.5≤x1<2)∧(2≤x2<2.5)∧(x3≥4)∧(x4≥3)

[0077] The classification model DTOHL model based on the maximum compatible rule to explain the random forest model classifier is as follows Figure 5 As shown in the figure. Experiments have shown that the accuracy of classification using the new features of the DTOHL model is higher than that of directly using the original sample features of the C4.5 decision tree model. This shows that the feature transformation process enhances the effectiveness of information. Therefore, it is meaningful for this method to imitate the classification process constructed by the human decision-making process, and the exact relationship between the new features and the original features needs further study.

[0078] When the values ​​of each variable of the sample meet the value ranges of the maximum compatible rule, the decision path in each tree is consistent with the T initial paths extracted, and as long as the coverage of any variable is expanded, the classification path of the sample in a certain tree will change, affecting the final result.

[0079] Based on the maximum compatibility rule, each variable takes a special value within the rule range, such as the median, maximum value, or a random value. Construct a zero matrix with the same size as the original sample, take the corresponding special values ​​for the variables involved in the rule, and construct a special instance that meets the maximum compatibility rule. Sample reconstruction can be completed by using the function in matplotlib to restore the matrix to an image, and a visualization effect of the variables involved in the classification process can be obtained. In an embodiment of pneumonia X-ray image recognition, when classifying normal samples and pneumonia samples, sample variables that affect the classification of X-ray images can be obtained through rule extraction and sample reconstruction, such as Figure 6-9 shown. Figure 6 This is a schematic diagram of the reconstruction effect of pneumonia X-ray image samples; Figure 7 for Figure 6 Schematic diagram of pneumonia X-ray images; Figure 8 This is a schematic diagram of the reconstruction effect of a normal X-ray image sample; Fig. 9 for Figure 8Schematic diagram of a normal X-ray image. It can be seen from the restored image that after the sample reconstruction based on the maximum compatible rule, the variables obtained are a valid variable set for image classification.

[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method of the present disclosure can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present disclosure, software program implementation is a better implementation method in most cases.

[0081] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.

Claims

1. A convolutional neural network hidden layer interpretation method, characterized in that: The method constructs a classification model based on the maximum compatible rule to explain the random forest model classifier, which is used to explain the classification of image samples by a specific hidden layer in a convolutional neural network. The construction of the classification model includes the following steps: S100, obtaining a set of hidden layer activation values ​​of a convolutional neural network model, wherein the set of activation values ​​is obtained by the following steps: S101, using the image data set to train the convolutional neural network model, and when the parameters are tuned until the model accuracy reaches the highest value, a trained convolutional neural network model is obtained and used as the original model; S102, constructing a network with the original model input as input and the selected hidden layer as output, and constructing an interception model; S103, based on the interception model, inputting the image sample data to obtain an activation value set; S200, clustering the activation value set once using a density-based spatial clustering algorithm in noisy applications to obtain M different clusters, where M is an integer greater than 0; S300, using a density-based spatial clustering algorithm in noisy applications to perform secondary clustering on each cluster, obtaining a number of factors for each cluster, assigning an ID to each factor, and using the ID as a label; S400, construct and obtain M feature conversion classifiers using a random forest model, and for each feature conversion classifier, use the original image sample feature as input and the label as output to train the feature conversion classifier. The random forest model can perform feature conversion interpretation through the following steps: S401, traverse T trees in the random forest model to obtain all classification paths of each tree; S402, after the image sample data is input, the variable discrimination criteria of each node classification corresponding to each classification path are combined into a variable rule; S403, extracting T variable rules from the T classification paths, merging the rules of the same variables, and arranging them in order to obtain a sorted classification rule set, and then obtaining the maximum compatible rule; S404, reconstructing image samples based on the maximum compatibility rule; S500, inputting the original image sample into the trained M feature conversion classifiers to obtain M-dimensional image features; S600, using C4.5 decision tree to build a classification model, taking M-dimensional image features as input and the corresponding labels of the original image samples as output. The classification accuracy of the new features after feature transformation is higher than that of directly using the original image sample features. It is determined that the feature transformation process enhances the effectiveness of information.

2. The method according to claim 1, characterized in that: The S102 uses the Model function provided by Keras to construct the interception model.

3. The method according to claim 1, characterized in that The maximum compatibility rule is obtained by the following steps: S4031. Create a blank list; S4032. For each variable rule, determine whether it already exists in the list; If it does not exist, execute S4033; Otherwise, execute S4034; S4033, taking the union of the variables involved and adding them into the list; S4034. Take the intersection of the variables involved to cover all the rules and add them into the list.

4. The method according to claim 1, characterized in that The sample reconstruction comprises the following steps: S4041. Construct a zero matrix with the same size as the original sample; S4042, taking corresponding special values ​​of the variables involved in the rule, and constructing a special instance that satisfies the maximum compatibility rule; The special value includes taking the median value, the maximum value or using a random value.

Images