Bernoulli sampling-based interpretable CNN (Convolutional Neural Network) training method and device and medium

Abstract

The invention relates to an interpretable CNN (Convolutional Neural Network) training method and device based on Bernoulli sampling and a medium, and the method comprises the following steps: inputting a picture into a CNN, and obtaining a response feature map of a filter; performing Bernoulli sampling on the response feature map to obtain a binary distribution matrix; calculating a filter average weight matrix of each picture category according to the binarization distribution matrix, and calculating the sum of pairwise differences; calculating the Hadamard product of the distribution vector of the binary distribution matrix and the response feature map to obtain a mask feature map; respectively inputting the response feature map and the mask feature map into a CNN full connection layer to respectively obtain classification prediction probability vectors, and respectively calculating cross entropy loss with a real label; and according to the sum of the pairwise differences and the cross entropy loss, using a stochastic gradient descent method to realize network training, and obtaining an interpretable CNN for image classification. Compared with the prior art, the method has the advantages of high adaptability, high interpretability and the like.

Description

Technical Field

[0001] This invention relates to the field of deep training technology, and in particular to an interpretable CNN training method, apparatus and medium based on Bernoulli sampling. Background Technology

[0002] Convolutional Neural Networks (CNNs) possess exceptional discriminative power and can be applied to numerous fields in real-world production and daily life. However, in safety-critical areas such as autonomous driving and healthcare, the interpretability of neural networks is a crucial attribute that must be considered. Interpretable neural networks not only aid in understanding and validating the model's decision-making process but also provide diagnostic information when problems arise, ensuring the safety and reliability of deep learning systems.

[0003] In recent years, research on the interpretability of CNNs has received increasing attention. Early ex-post interpretation methods focused on generating offline explanations for trained neural networks, with a representative paper being Zhou, B., et al. (2016), Learning deep features for discriminative localization. However, ex-post methods cannot improve the intrinsic interpretability of neural networks because they are independent of the neural network's training process and cannot guide the learning of neural network parameters. Recently, researchers have begun to focus on training interpretable neural networks. For example, Zhang, Q., et al. (2018), Interpretable convolutional neural networks. This paper encourages specifying that filters in convolutional layers respond only to specific parts of an image object. However, this method still relies on ex-post interpretation to match the semantics learned by the filters from a predefined set of semantic concepts, and cannot directly reveal the interpretable information of the filters. This means that further research is needed on how to learn interpretable filters during training to improve the intrinsic interpretability of neural networks, thereby better applying them to high-security domains and enhancing user trust in deep learning systems. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of insufficient interpretability in the prior art by providing an interpretable CNN training method, apparatus and medium based on Bernoulli sampling.

[0005] The objective of this invention can be achieved through the following technical solutions: An interpretable CNN training method based on Bernoulli sampling includes the following steps: S1: Move the image The weights are obtained by inputting into the CNN. of Response characteristic diagram of each filter ,in Indicates the first The trainable weight matrix of each filter. Indicates the first Response characteristic maps of each filter; S2: Using a trainable filter-class correspondence matrix Using each element as a parameter, Bernoulli sampling is performed to obtain a binarized allocation matrix. ,in Indicates the first The filter is dedicated to the first one. Image categories, This represents the total number of image categories in the dataset; S3: Based on the binarized allocation matrix Calculate the average weight matrix of the filters for each image category. And calculate the sum of pairwise differences between each average weight matrix. ; S4: Allocation vector based on binary allocation matrix With response feature map Hadamaji mask image Non-class-specific filters are used to obtain mask feature maps. ,in Represents the allocation matrix The OK, Image The true category label; S5: Response Feature Map and mask feature map The classification prediction probability vectors are obtained by inputting each of the fully connected layers of the CNN. and and calculate respectively With real category labels Cross-entropy loss between ; S6: Calculate the total training loss ,in These are weight coefficients, updated based on the total training loss using stochastic gradient descent, along with the CNN parameters and filter-class correspondence matrix. To achieve network training; S7: Use the trained network to classify images and obtain interpretable image classification results.

[0006] Furthermore, the CNN uses any specified... The image is divided into a feature extractor and a classifier, with each filter's convolutional layer as a boundary. The input to the feature extractor is the original image data, and the output is the specified convolutional layer. The response feature maps of each filter; the input of the classifier is a specified... The output is the image classification probability score, which is the complete or masked feature map of each filter.

[0007] Furthermore, during network training, the response matrix is ​​used... Perform a Bernoulli trial to sample the binarized filter-image correspondence for the parameters, quantize the differences between filters, and mask the non-class-specific filter feature map of the input image.

[0008] Furthermore, in step S2, the matrix is ​​allocated. The calculation expression is: In the formula, Representation matrix The Line number Column elements, Representation matrix The Line number Column elements, It is a constant. The calculation expression is: In the formula, It is a random variable that follows a uniform distribution between 0 and 1.

[0009] Furthermore, step S3 includes the following steps: S31: The Average filter weights for each image category The calculation expression is: In the formula, Hadamard product, elements First, it is broadcast into a matrix. The two matrices have the same shape, and then the product is calculated element by element; S32: Calculate the average weight difference of class-specific filters for different classes. Average weight difference The calculation expression is: In the formula, The average weight difference of class-specific filters for different classes.

[0010] Further, step S4 includes the following steps: S41: Input image Classification prediction probability score of complete feature map Sampling pseudo-labels for multi-class distribution parameters The calculation expression for the pseudo-label is: In the formula, vector A zero element was inserted at the beginning. Representing vectors The One element, Representing vectors The One element, It is a random variable that follows a uniform distribution between 0 and 1; S42: Based on pseudo-labels Index allocation matrix The The row yields a binary allocation vector. ; S43: By calculating the allocation vector With feature map The Hadamard product is used to filter non-class-specific filters to obtain the mask feature map. : In the formula, For Hadama accumulation, Represents the allocation vector The One element, Indicates the first Response characteristic map of each filter.

[0011] Further, step S5 includes the following steps: S51: Transfer feature map and mask feature map The classification prediction probability vectors are obtained by inputting each of the fully connected layers of the CNN. and ; S52: Calculate the prediction results respectively and With pictures Real category labels The cross-entropy loss between them is calculated using the following expression: In the formula, One-hot tag vector The One element, For the prediction result vector The One element, The Each element.

[0012] Further, step S6 includes the following steps: S61: The expression for calculating the total loss function for training a neural network is: In the formula, the weighting parameter is fixed as follows: ; S62: Calculate the total loss function About neural network parameters First-order partial derivative and about the response matrix The first-order partial derivative is calculated as follows: Update using stochastic gradient descent and The updated calculation expression is: In the formula, is the training rate for stochastic gradient descent.

[0013] In a second aspect, an interpretable CNN training apparatus based on Bernoulli sampling is provided, comprising a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement an interpretable CNN training method based on Bernoulli sampling as described above.

[0014] A third aspect of the invention is a storage medium having a program stored thereon, which, when executed, implements an interpretable CNN training method based on Bernoulli sampling as described above.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention proposes a learning filter-class correspondence matrix, using a reparameterization technique based on Bernoulli distribution sampling, which supports end-to-end training of Bernoulli probability parameters, learns interpretable class-specific filters, transforms standard CNNs into more interpretable CNNs, has strong adaptability, and can be flexibly combined with various CNN architectures.

[0016] 2) The CNN obtained by the interpretable CNN learning method of the present invention shows a significant improvement over the standard CNN in multiple interpretability metrics, while maintaining the original discrimination ability, thus verifying the effectiveness of the method. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the learning process of the interpretable CNN of the present invention.

[0018] Figure 2 This is a diagram of the network structure of the interpretable CNN of the present invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0020] Example 1 This invention relates to an interpretable CNN training method, apparatus, and medium based on Bernoulli sampling, such as... Figure 1 As shown, the method includes the following steps: S1: Move the image The weights are obtained by inputting into the CNN. of Response characteristic diagram of each filter ,in Indicates the first The trainable weight matrix of each filter. Indicates the first Response characteristic maps of each filter; S2: Using a trainable filter-class correspondence matrix Using each element as a parameter, Bernoulli sampling is performed to obtain a binarized allocation matrix. ,in Indicates the first The filter is dedicated to the first one. Image categories, This represents the total number of image categories in the dataset; Allocation matrix The calculation expression is: In the formula, Representation matrix The Line number Column elements, Representation matrix The Line number Column elements, It is a constant. The calculation expression is: In the formula, It is a random variable that follows a uniform distribution between zero and one.

[0021] S3: Based on the binarized allocation matrix Calculate the average weight matrix of the filters for each image category. And calculate the sum of pairwise differences between each average weight matrix. ; S31: The Average filter weights for each image category The calculation expression is: In the formula, Hadamard product, elements First, it is broadcast into a matrix. The two matrices have the same shape, and then the product is calculated element by element; S32: Calculate the average weight difference of class-specific filters for different classes. Average weight difference The calculation expression is: In the formula, The average weight difference of class-specific filters for different classes.

[0022] S4: Allocation vector based on binary allocation matrix With response feature map Hadamaji mask image Non-class-specific filters are used to obtain mask feature maps. ,in Represents the allocation matrix The OK, Image The true category label; S41: Input image Classification prediction probability score of complete feature map Sampling pseudo-labels for multi-class distribution parameters The calculation expression for the pseudo-label is: In the formula, vector A zero element was inserted at the beginning. Representing vectors The One element, Representing vectors The One element, It is a random variable that follows a uniform distribution between 0 and 1; S42: Based on pseudo-labels Index allocation matrix The The row yields a binary allocation vector. ; S43: By calculating the allocation vector With feature map The Hadamard product is used to filter non-class-specific filters to obtain the mask feature map. : In the formula, For Hadama accumulation, Represents the allocation vector The One element, Indicates the first Response characteristic map of each filter.

[0023] S5: Response Feature Map and mask feature map The classification prediction probability vectors are obtained by inputting each of the fully connected layers of the CNN. and and calculate respectively With real category labels Cross-entropy loss between ; S51: Transfer feature map and mask feature map The classification prediction probability vectors are obtained by inputting each of the fully connected layers of the CNN. and ; S52: Calculate the prediction results respectively and With pictures Real category labels The cross-entropy loss between them is calculated using the following expression: In the formula, One-hot tag vector The One element, For the prediction result vector The One element, The Each element.

[0024] S6: Calculate the total training loss ,in These are the weight coefficients, and the parameters and filter-class correspondence matrix of the CNN are updated using stochastic gradient descent. This enables network training, resulting in an interpretable CNN. S61: The expression for calculating the total loss function for training a neural network is: In the formula, the weighting parameter is fixed as follows: ; S62: Calculate the total loss function About neural network parameters First-order partial derivative and about the response matrix The first-order partial derivative is calculated as follows: Update using stochastic gradient descent and The updated calculation expression is: In the formula, is the training rate for stochastic gradient descent.

[0025] S7: Use the trained network to classify images and obtain interpretable image classification results.

[0026] Example 2 Based on Embodiment 1 of the present invention, the algorithm structure diagram for image classification of the present invention is as follows: Figure 2 As shown, the original image is input into the feature extractor to generate a complete feature map. On one hand, the complete feature map is input into the classifier to obtain the first classification prediction result. On the other hand, the complete feature map and the first classification prediction result are input into the sampling result of the filter-image category correspondence matrix to perform Hadamard product to generate a mask feature map. Subsequently, the mask feature map is input into the classifier again to generate the second classification prediction result, which is the final classification result.

[0027] With any specified having The CNN is divided into a feature extractor and a classifier, with each convolutional layer containing a specified number of filters (or kernels) as the boundary. The feature extractor takes the original image data as input and outputs the data from the specified convolutional layers. The response feature map of each filter; the input of the classifier is this The output is the image classification probability score, which is the complete or masked feature map of each filter.

[0028] Introducing a filter - image category response matrix To learn a specified convolutional layer Each filter and dataset The correspondence between image categories. During network training, the response matrix is ​​used... A Bernoulli trial is performed to sample the binarized filter-image assignment relation for the parameters. Based on this assignment relation, the differences between class-specific filters of different classes are quantified, and the non-class-specific filter feature maps of the input image are masked.

[0029] The learning objective of interpretable CNNs is to maximize the differences between class-specific filters of different classes and enable the network to accurately classify based on full or masked feature maps.

[0030] Standard CNN (STD) and internationally advanced interpretable CNN learning methods, including Class-Specific Gate (CSG) Neural Networks and Interpretable Combinatorial CNN (ICCNN), were selected. ResNet-18 was used as the backbone network, and validation was performed on the CIFAR-10 and STL-10 datasets. Discriminant metric ACC1 and three interpretability metrics, MIS, ACC2, and ACC3, were used as evaluation metrics. ACC1 represents the classification accuracy of the complete feature map, ACC2 represents the classification accuracy of the class-specific filter feature map, and ACC3 represents the classification accuracy of the non-class-specific filter feature map. MIS represents the mutual information between the filter and its corresponding class label. The corresponding class is represented by the learned filter-image class correspondence matrix. Read from, number The filter and the first Image correspondence. The MIS calculation expression is: In the formula, It is the first Feature maps of each filter These are the actual category labels for the images. This represents mutual information. All evaluation indicators range from 0 to 1. For ACC1, ACC2, and MIS, higher values ​​indicate better performance; for ACC3, lower values ​​indicate better performance.

[0031] Table 1 shows the performance of CNNs learned by different methods on two benchmark datasets. As can be seen from Table 1, the interpretable CNN of the present invention has a significant improvement in interpretability compared with standard neural networks and neural networks learned by other methods, while maintaining the high discriminative ability of standard neural networks.

[0032] Table 2 Performance of different backbone networks on the CIFAR-10 dataset As can be seen from Table 2, the interpretable CNN learning method of the present invention is compatible with a variety of mainstream backbone networks, which indicates that the learning method of the present invention has broad applicability and can provide flexibility for model selection and deployment in practical applications.

[0033] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0034] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CDROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0035] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An interpretable CNN training method based on Bernoulli sampling, characterized in that, Includes the following steps: S1: Move the image The weights are obtained by inputting into the CNN. of Response characteristic diagram of each filter ,in Indicates the first The trainable weight matrix of each filter. Indicates the first Response characteristic maps of each filter; S2: Using a trainable filter-class correspondence matrix Using each element as a parameter, Bernoulli sampling is performed to obtain a binarized allocation matrix. ,in Indicates the first The filter is dedicated to the first one. Image categories, This represents the total number of image categories in the dataset; S3: Based on the binarized allocation matrix Calculate the average weight matrix of the filters for each image category. And calculate the sum of pairwise differences between each average weight matrix. ; S4: Allocation vector based on binary allocation matrix With response feature map Hadamaji mask image Non-class-specific filters are used to obtain mask feature maps. ,in Represents the allocation matrix The OK, Image The true category label; S5: Response Feature Map and mask feature map The classification prediction probability vectors are obtained by inputting each of the fully connected layers of the CNN. and and calculate respectively With real category labels Cross-entropy loss between ; S6: Calculate the total training loss ,in These are weight coefficients, updated based on the total training loss using stochastic gradient descent, along with the CNN parameters and filter-class correspondence matrix. Thus, a trained network is obtained; S7: Use the trained network to classify images and obtain interpretable image classification results.

2. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, The CNN uses any specified... The image is divided into a feature extractor and a classifier, with each filter's convolutional layer as a boundary. The input to the feature extractor is the original image data, and the output is the specified convolutional layer. The response feature maps of each filter; the input of the classifier is a specified... The output is the image classification probability score, which is the complete or masked feature map of each filter.

3. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, During network training, the response matrix is ​​used Perform a Bernoulli trial to sample the binarized filter-image correspondence for the parameters, quantize the differences between filters, and mask the non-class-specific filter feature map of the input image.

4. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, Assigning the matrix in step S2 The calculation expression is: In the formula, Representation matrix The Line number Column elements, Representation matrix The Line number Column elements, It is a constant. The calculation expression is: In the formula, It is a random variable that follows a uniform distribution between 0 and 1.

5. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, Step S3 includes the following steps: S31: The Average filter weights for each image category The calculation expression is: In the formula, Hadamard product, elements First, it is broadcast into a matrix. The two matrices have the same shape, and then the product is calculated element by element. S32: Calculate the average weight difference of class-specific filters for different classes. Average weight difference The calculation expression is: In the formula, The average weight difference of class-specific filters for different classes.

6. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, Step S4 includes the following steps: S41: Input image Classification prediction probability score of complete feature map Sampling pseudo-labels for multi-class distribution parameters The calculation expression for the pseudo-label is: In the formula, vector A zero element was inserted at the beginning. Representing vectors The One element, Representing vectors The One element, It is a random variable that follows a uniform distribution between 0 and 1; S42: Based on pseudo-labels Index allocation matrix The The row obtains a binary assignment vector. ; S43: By calculating the allocation vector With feature map The Hadamard product is used to filter non-class-specific filters to obtain the mask feature map. : In the formula, For Hadama accumulation, Represents the allocation vector The One element, Indicates the first Response characteristic map of each filter.

7. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, Step S5 includes the following steps: S51: Feature map and mask feature map The classification prediction probability vectors are obtained by inputting each of the fully connected layers of the CNN. and ; S52: Calculate the prediction results respectively and With pictures Real category labels The cross-entropy loss between them is calculated using the following expression: In the formula, One-hot tag vector The One element, For the prediction result vector The One element, The Each element.

8. The interpretable CNN training method based on Bernoulli sampling according to claim 1, characterized in that, Step S6 includes the following steps: S61: The expression for calculating the total loss function for training a neural network is: In the formula, the weighting parameter is fixed as follows: ; S62: Calculate the total loss function About neural network parameters First-order partial derivative and about the response matrix The first-order partial derivative is calculated as follows: Update using stochastic gradient descent and The updated calculation expression is: In the formula, is the training rate for stochastic gradient descent.

9. An interpretable CNN training apparatus based on Bernoulli sampling, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements an interpretable CNN training method based on Bernoulli sampling as described in any one of claims 1-8.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements an interpretable CNN training method based on Bernoulli sampling as described in any one of claims 1-8.

Images