Defect detection method based on deep learning interpretable knowledge guidance
By combining guided learning classification networks with attribution saliency heatmaps, the problem of unstable performance of deep learning algorithms in surface defect detection is solved, achieving high accuracy and robustness in defect detection, which is suitable for high-reliability scenarios such as industrial quality inspection.
Patent Information
- Application Number
- CN202610109358.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-27
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional deep learning-based artificial intelligence algorithms are unstable in surface defect detection, especially in high-risk scenarios where the accuracy of defect classification is poor, making it difficult to meet the reliability requirements of industrial quality inspection.
A defect detection method based on deep learning interpretability knowledge guidance is adopted. By combining a pre-trained basic classification network and a guided learning classification network with an attribution saliency heatmap, feature extraction and classification prediction are performed. By using multi-task architecture design and loss function optimization, the accuracy of defect classification is improved.
It improves the accuracy and stability of defect classification, especially with a small sample size. It enables high-performance defect detection through image-level labels that do not require pixel-level annotation, thus enhancing the robustness and transparency of the model.
Smart Images
Figure CN122049480A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision, specifically to the field of deep learning and surface defect detection. Background Technology
[0002] In industrial production processes, surface defect detection technology is crucial for product quality control. However, relying on manual quality inspection is time-consuming, labor-intensive, inefficient, and prone to false positives. To improve the automation level of production, machine vision methods are often used for inspection. However, traditional machine vision methods are difficult to adapt to complex production environments, have poor robustness, and require different solutions for different problems. With the development of artificial intelligence technology, deep learning technology has been gradually applied to the field of surface defect detection, demonstrating superior performance.
[0003] However, the detection performance of deep learning-based artificial intelligence algorithms is unstable, especially in high-risk scenarios, where the reliability of deep learning methods cannot be guaranteed, resulting in poor defect classification accuracy. Therefore, these problems urgently need to be addressed. Summary of the Invention
[0004] To address the problem of unstable detection performance and poor defect classification accuracy caused by traditional deep learning-based artificial intelligence algorithms, this invention provides a defect detection method guided by interpretable knowledge from deep learning.
[0005] A defect detection method based on deep learning interpretability knowledge guidance, comprising:
[0006] Knowledge generation stage:
[0007] The pre-trained basic classification network is used to predict the category of the image to be detected. At the same time, the classification reasoning basis of the pre-trained basic classification network is extracted using deep learning interpretable methods to obtain an attribution saliency heatmap.
[0008] Guided learning phase:
[0009] By using a pre-trained guided learning classification network combined with an attribution saliency heatmap, the image to be detected is classified and predicted to obtain the defect classification result.
[0010] The guided learning classification network includes a backbone network, splicing units, a segmentation head, a classification head, a first pooling unit, a second pooling unit, and a comprehensive feature analysis unit;
[0011] The backbone network is used to extract features from the received image to be detected, and obtain multi-channel feature maps;
[0012] The splicing unit is used to splice the multi-channel feature map and the attribution saliency heatmap to obtain the spliced feature map;
[0013] The segmentation head is used to extract and segment features from the stitched feature map to obtain a segmented feature map.
[0014] The classification head is used to extract features from the concatenated feature map to obtain a classification feature map;
[0015] The first pooling unit is used to perform global max pooling and global average pooling operations on the segmented feature map respectively, and obtain the first feature vector and the second feature vector accordingly.
[0016] The second pooling unit is used to perform global max pooling and global average pooling operations on the classification feature map respectively, and obtain the third feature vector and the fourth feature vector accordingly.
[0017] The comprehensive feature analysis unit is used to analyze and process the first to fourth feature vectors to obtain the defect classification results.
[0018] Preferably, the comprehensive feature analysis unit analyzes and processes the first to fourth feature vectors to obtain the defect classification results in the following manner:
[0019] The first to fourth feature vectors are concatenated, and the concatenated feature vectors are used to predict the classification probability through a softmax classifier. The category corresponding to the highest classification probability is then output as the defect classification result.
[0020] Preferably, the specific process of obtaining the pre-trained guided learning classification network includes:
[0021] S1. Constructing a sample set: The sample set includes multiple training samples, each of which includes an image and the category label of that image;
[0022] S2. Train the guided learning classification network using each training sample to obtain the pre-trained guided learning classification network. The specific process includes:
[0023] The images in the current training samples are input into the pre-trained basic classification network for category prediction. At the same time, the classification reasoning basis of the pre-trained basic classification network is extracted using deep learning interpretable methods to obtain the attribution saliency heatmap. The attribution saliency heatmap is then binarized using a binarization method to obtain a binarized image.
[0024] The image in the current training sample and the category label corresponding to the image are used as the output and output of the guided learning classification network, respectively. The guided learning classification network is trained by combining the attribution saliency heatmap to obtain the pre-trained guided learning classification network.
[0025] During training, the segmentation loss is constructed using the segmentation feature map and binarized map output by the classification head of the guided learning classification network. The classification loss is constructed by combining the defective classification results of the guided learning classification network with the class labels in the current training samples. ,according to and Total construction loss Utilizing total losses Update the model parameters of the guided learning classification network.
[0026] Preferably, ;in, is a coefficient, with a value between 0 and 1.
[0027] Preferably, the segmentation loss The expression is: ;
[0028] in, This represents the total number of pixels in the segmentation feature map or binarized map. The total number of pixels in the segmentation feature map and the binarized map are the same. For pixel index, The first binarized image The pixel value, which is also used as the first pixel value in the binarized image. A pseudo-label of one pixel, It is the first segmentation feature map Each pixel value.
[0029] Preferably, classification loss The expression is: ;
[0030] In this case, the total number of pixels in the segmentation feature map and the binarized map are the same. For pixel index, The first binarized image The pixel value, which is also used as the first pixel value in the binarized image. A pseudo-label of one pixel, It is the first segmentation feature map Each pixel value.
[0031] Preferably, the basic classification network is implemented using a convolutional neural network.
[0032] Preferably, the deep learning interpretable method is GradCAM, LayerCAM, or FullGrad.
[0033] Preferably, the pre-trained base classification network and the pre-trained guided learning classification network are trained using the same batch of sample sets.
[0034] The beneficial effects of this invention are:
[0035] In the knowledge generation stage (first stage), this invention utilizes interpretable techniques to extract semantic features (i.e., attribution saliency heatmaps). These features serve as structured prior information to guide subsequent guided learning stages. In the guided learning stage (second stage), a specially designed knowledge fusion mechanism integrates previously acquired prior knowledge into the guided learning classification network. This allows the network to fuse earlier knowledge with current features to enhance its representational capabilities, thereby improving defect classification accuracy.
[0036] This invention addresses the defect classification problem with a small sample size by automatically mining more information, resulting in more stable training and stronger performance. This invention eliminates the need for pixel-level annotation; it only requires image-level labels (category labels) to obtain pseudo-pixel labels from "network knowledge." Through a multi-task architecture design, it guides the network's learning.
[0037] During the training of the guided learning classification network, the attribution saliency heatmap is binarized. This prior knowledge can also be transformed into a soft segmentation supervision signal, which helps the guided learning classification network to achieve more accurate optimization and further improve the accuracy of defect classification. Attached Figure Description
[0038] Figure 1 This is a schematic diagram illustrating the principle of reasoning using the defect detection method guided by deep learning interpretability knowledge as described in this invention.
[0039] Figure 2 This is a schematic diagram illustrating the principle of training guided learning classification networks. Detailed Implementation
[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0041] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.
[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.
[0043] Specific Implementation Method 1: Combination Figure 1 As shown in this embodiment, the defect detection method based on deep learning interpretability knowledge guidance includes:
[0044] Knowledge generation stage:
[0045] The pre-trained basic classification network is used to predict the category of the image to be detected. At the same time, the classification reasoning basis of the pre-trained basic classification network is extracted using deep learning interpretable methods to obtain an attribution saliency heatmap.
[0046] Guided learning phase:
[0047] By using a pre-trained guided learning classification network combined with an attribution saliency heatmap, the image to be detected is classified and predicted to obtain the defect classification result.
[0048] The guided learning classification network includes a backbone network, splicing units, a segmentation head, a classification head, a first pooling unit, a second pooling unit, and a comprehensive feature analysis unit;
[0049] The backbone network is used to extract features from the received image to be detected, and obtain multi-channel feature maps;
[0050] The splicing unit is used to splice the multi-channel feature map and the attribution saliency heatmap to obtain the spliced feature map;
[0051] The segmentation head is used to extract and segment features from the stitched feature map to obtain a segmented feature map.
[0052] The classification head is used to extract features from the concatenated feature map to obtain a classification feature map;
[0053] The first pooling unit is used to perform global max pooling and global average pooling operations on the segmented feature map respectively, and obtain the first feature vector and the second feature vector accordingly.
[0054] The second pooling unit is used to perform global max pooling and global average pooling operations on the classification feature map respectively, and obtain the third feature vector and the fourth feature vector accordingly.
[0055] The comprehensive feature analysis unit is used to analyze and process the first to fourth feature vectors to obtain the defect classification results.
[0056] This invention achieves a dual leap in detection performance and model transparency through interpretable knowledge guidance. Attribution heatmaps, serving as visual prior knowledge, precisely anchor the network's attention to semantically critical regions, enabling the model to effectively suppress background interference and focus on the essential features of defects during training. Injecting general visual prior knowledge into the guided learning classification network for specialized defect detection tasks allows the network to possess both human-understandable reasoning logic and generalization reliability surpassing traditional methods, providing an innovative solution for high-reliability scenarios such as industrial quality inspection.
[0057] Specifically, deep learning interpretable methods are not limited to GradCAM, LayerCAM, or FullGrad. Furthermore, the comprehensive feature analysis unit analyzes and processes the first to fourth feature vectors to obtain the defect classification results, implemented as follows:
[0058] The first to fourth feature vectors are concatenated, and the concatenated feature vectors are used to predict the classification probability through a softmax classifier. The category corresponding to the highest classification probability is then output as the defect classification result.
[0059] When applied, the pre-trained base classification network and the pre-trained guided learning classification network use the same batch of samples. For cases where the number of defective samples is small, more information can be obtained for effective training.
[0060] The basic classification network is implemented using ordinary convolutional neural networks, such as VGG and ResNet.
[0061] The pre-trained base classification network is as follows:
[0062] Constructing a sample set: The sample set includes multiple training samples, each of which includes an image and the class label of that image;
[0063] The images and category labels of each training sample are used as the input and output data of the network, respectively, to train the basic classification network and obtain the trained basic classification network.
[0064] Further, see Figure 2 The specific process of obtaining a pre-trained guided learning classification network includes:
[0065] S1. Constructing a sample set: The sample set includes multiple training samples, each of which includes an image and the category label of that image;
[0066] S2. Train the guided learning classification network using each training sample to obtain the pre-trained guided learning classification network. The specific process includes:
[0067] The images in the current training samples are input into the pre-trained basic classification network for category prediction. At the same time, the classification reasoning basis of the pre-trained basic classification network is extracted using deep learning interpretable methods to obtain the attribution saliency heatmap. The attribution saliency heatmap is then binarized using a binarization method to obtain a binarized image.
[0068] The image in the current training sample and the category label corresponding to the image are used as the output and output of the guided learning classification network, respectively. The guided learning classification network is trained by combining the attribution saliency heatmap to obtain the pre-trained guided learning classification network.
[0069] During training, the segmentation loss is constructed using the segmentation feature map and binarized map output by the classification head of the guided learning classification network. The classification loss is constructed by combining the defective classification results of the guided learning classification network with the class labels in the current training samples. ,according to and Total construction loss Utilizing total losses Update the model parameters of the guided learning classification network;
[0070] ;
[0071] in, is a coefficient, with a value between 0 and 1, used to balance the segmentation loss and the classification loss.
[0072] Segmentation loss The expression is:
[0073] ;
[0074] Classification loss The expression is:
[0075] ;
[0076] in, This represents the total number of pixels in the segmentation feature map or binarized map. The total number of pixels in the segmentation feature map and the binarized map are the same. For pixel index, The first binarized image The pixel value, which is also used as the first pixel value in the binarized image. A pseudo-label of one pixel, It is the first segmentation feature map Each pixel value.
[0077] In this preferred embodiment, a multi-task architecture is adopted. The attribution saliency map and binarized map obtained in the first stage (i.e., the knowledge generation stage) are used as prior information to constrain the network learning in the classification loss and segmentation loss outputs, respectively. The effect is that the prior information promotes the model to learn more effective classification feature representations, while the classification information can constrain the erroneous background noise in the segmentation loss. Furthermore, pixel-level annotation is not required; only image-level labels (category labels) are needed to obtain pseudo-pixel labels from "network knowledge," achieving high-performance defect classification.
[0078] By employing guided learning training, the network can efficiently learn guidance for segmentation and classification tasks. At the same time, it can use more accurate classification information to correct background noise from semantic pseudo-labels, making model training more efficient.
[0079] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.
Claims
1. A defect detection method guided by deep learning interpretability knowledge, characterized in that, The method includes: Knowledge generation stage: The pre-trained basic classification network is used to predict the category of the image to be detected. At the same time, the classification reasoning basis of the pre-trained basic classification network is extracted using deep learning interpretable methods to obtain an attribution saliency heatmap. Guided learning phase: By using a pre-trained guided learning classification network combined with an attribution saliency heatmap, the image to be detected is classified and predicted to obtain the defect classification result. The guided learning classification network includes a backbone network, splicing units, a segmentation head, a classification head, a first pooling unit, a second pooling unit, and a comprehensive feature analysis unit; The backbone network is used to extract features from the received image to be detected, and obtain multi-channel feature maps; The splicing unit is used to splice the multi-channel feature map and the attribution saliency heatmap to obtain the spliced feature map; The segmentation head is used to extract and segment features from the stitched feature map to obtain a segmented feature map. The classification head is used to extract features from the concatenated feature map to obtain a classification feature map; The first pooling unit is used to perform global max pooling and global average pooling operations on the segmented feature map respectively, and obtain the first feature vector and the second feature vector accordingly. The second pooling unit is used to perform global max pooling and global average pooling operations on the classification feature map respectively, and obtain the third feature vector and the fourth feature vector accordingly. The comprehensive feature analysis unit is used to analyze and process the first to fourth feature vectors to obtain the defect classification results.
2. The defect detection method based on deep learning interpretability knowledge guidance according to claim 1, characterized in that, The comprehensive feature analysis unit analyzes and processes the first to fourth feature vectors to obtain the defect classification results as follows: The first to fourth feature vectors are concatenated, and the concatenated feature vectors are used to predict the classification probability through a softmax classifier. The category corresponding to the highest classification probability is then output as the defect classification result.
3. The defect detection method based on deep learning interpretability knowledge guidance according to claim 1, characterized in that, The specific process of obtaining a pre-trained guided learning classification network includes: S1. Constructing a sample set: The sample set includes multiple training samples, each of which includes an image and the category label of that image; S2. Train the guided learning classification network using each training sample to obtain the pre-trained guided learning classification network. The specific process includes: The images in the current training samples are input into the pre-trained basic classification network for category prediction. At the same time, the classification reasoning basis of the pre-trained basic classification network is extracted using deep learning interpretable methods to obtain the attribution saliency heatmap. The attribution saliency heatmap is then binarized using a binarization method to obtain a binarized image. The image in the current training sample and the category label corresponding to the image are used as the output and output of the guided learning classification network, respectively. The guided learning classification network is trained by combining the attribution saliency heatmap to obtain the pre-trained guided learning classification network. During training, the segmentation loss is constructed using the segmentation feature map and binarized map output by the classification head of the guided learning classification network. The classification loss is constructed by combining the defective classification results of the guided learning classification network with the class labels in the current training samples. ,according to and Total construction loss Utilizing total losses Update the model parameters of the guided learning classification network.
4. The defect detection method based on deep learning interpretability knowledge guidance according to claim 3, characterized in that, ;in, is a coefficient, with a value between 0 and 1.
5. The defect detection method based on deep learning interpretability knowledge guidance according to claim 3, characterized in that, Segmentation loss The expression is: ; in, This represents the total number of pixels in the segmentation feature map or binarized map. The total number of pixels in the segmentation feature map and the binarized map are the same. For pixel index, The first binarized image The pixel value, which is also used as the first pixel value in the binarized image. A pseudo-label of one pixel, It is the first segmentation feature map Each pixel value.
6. The defect detection method based on deep learning interpretability knowledge guidance according to claim 3, characterized in that, Classification loss The expression is: ; In this case, the total number of pixels in the segmentation feature map and the binarized map are the same. For pixel index, The first binarized image The pixel value, which is also used as the first pixel value in the binarized image. A pseudo-label of one pixel, It is the first segmentation feature map Each pixel value.
7. The defect detection method based on deep learning interpretability knowledge guidance according to claim 1, characterized in that, The basic classification network is implemented using a convolutional neural network.
8. The defect detection method based on deep learning interpretability knowledge guidance according to claim 1, characterized in that, Interpretable deep learning methods include GradCAM, LayerCAM, or FullGrad.
9. The defect detection method based on deep learning interpretability knowledge guidance according to claim 1, characterized in that, The pre-trained base classification network and the pre-trained guided learning classification network are trained using the same batch of sample sets.