An explainable powder residue visual inspection method and apparatus
By constructing an interpretable visual detection model for powder residue suitable for small samples, and combining transfer learning and category activation graph techniques, the problems of high annotation cost and poor interpretability are solved, achieving efficient and interpretable powder residue detection, which is suitable for industrial scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING VOCATIONAL UNIV OF IND TECH
- Filing Date
- 2023-06-12
- Publication Date
- 2026-05-19
AI Technical Summary
Existing visual detection models for residues are costly to annotate, have poor interpretability, and are not very practical, making it difficult to meet the real-time and interpretability requirements of industrial scenarios.
We employ transfer learning, weakly supervised learning, and category activation graph techniques to construct an interpretable visual detection model for powder residues suitable for small samples. This model includes a data augmentation module, a feature extraction module, a joint attention adjustment module, and a dual-head output module. Feature extraction and classification are performed using a ResNet18 network to generate interpretable heatmaps.
It achieves efficient and interpretable powder residue detection with small sample training, reduces annotation costs, improves model adaptability and detection efficiency, and meets the real-time and interpretability requirements of industrial applications.
Smart Images
Figure CN116740433B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of visual inspection, and specifically relates to an interpretable visual inspection method and device for powder residue. Background Technology
[0002] Residue detection is an essential part of industrial production. For example, in automated filling production line systems, the cleanliness of containers or conveying pipelines is required to prevent impurities from contaminating the product. Especially in pharmaceutical filling processes, it is strictly forbidden for other residues to be mixed into the medicine. During injection molding, product residues are often found in the molds of injection molding machines. These residues not only affect product quality but can also damage the molds. Therefore, it is necessary to inspect and clean these residues. Currently, residue detection in industrial production processes is usually done manually. With increasing working hours, the accuracy of manual detection cannot be guaranteed, and workers are prone to fatigue due to repetitive operations, further affecting the accuracy of the test results.
[0003] Traditional automated visual inspection methods based on image processing and machine learning [1] can improve detection efficiency and accuracy. However, these techniques are limited in that handcrafted features are often selected based on the characteristics of specific datasets, resulting in poor robustness, adaptability, and versatility. Especially in complex industrial environments, the overall performance of conventional visual inspection methods is poor, which limits their implementation and widespread application in the industrial field.
[0004] In recent years, with the rapid development of artificial intelligence technology, deep learning-based visual detection algorithms have been studied due to their powerful data abstraction and expression capabilities. Deep learning-based visual detection algorithms generally extract effective image features through a designed feature extraction network, and then further analyze and process the image features according to different detection tasks [2]. Currently, typical deep learning visual detection algorithms require a large number of labeled samples, especially for pixel-level visual segmentation tasks, where the accuracy requirements for sample labeling are very high, which greatly increases the labeling cost. Moreover, in order to meet the visual detection accuracy, the network model is continuously deepened and broadened, which reduces the detection efficiency and often fails to meet the real-time requirements of industrial scenarios. In addition, most deep learning-based visual detection models have poor interpretability and are difficult to meet the requirements of industrial practice for the interpretability of detection results.
[0005] Therefore, it is urgent to develop an efficient automatic visual inspection system for powder residue.
[0006] The following documents are technical background information related to this invention:
[0007] [1] Q. Luo, Y. Sun, P. Li, O. Simpson, L. Tian, and H. Yigang, "Generalized completed local binary patterns for time-efficient steel surfacedefect classification," IEEE Trans. Instrum. Meas., vol. 68, no. 3, pp. 667-679, Mar. 2019.
[0008] [2] Y. He, K. Song, Q. Meng and Y. Yan, "An End-to-End steel surfacedefect detection approach via fusing multiple hierarchical features," IEEETrans. Instrum. Meas., vol. 69, no. 4, pp. 1493-1504, Apr. 2020. Summary of the Invention
[0009] To address the aforementioned problems in existing technologies, namely the high annotation cost, poor interpretability, and limited practicality of existing visual detection models for residues, this invention provides an interpretable visual detection method for powder residues, the visual detection method comprising:
[0010] Step S10: Classify the collected powder container image set, and adjust the format and image size to obtain a set of images with and without residue, and a set of images with residue, with a set of formats and sizes.
[0011] Step S20: Construct a visual detection model for powder residue based on transfer learning. The visual detection model for powder residue includes a data augmentation module, a feature extraction module, a joint attention weight adjustment module, and a dual-head output module.
[0012] Step S30: Train the visual detection model for powder residue using the image sample set with and without residue.
[0013] Step S40: Detect powder residue in the container using the trained visual detection model for powder residue, and obtain the classification results and the corresponding interpretable heatmap.
[0014] In some preferred embodiments, the data augmentation module is used to expand the residual image sample set and the residual image sample set, including:
[0015] Randomly flip, randomly scale, and / or color-jitter each image sample in the image sample set without residue and the image sample set with residue, respectively, to obtain the image augmentation sample set without residue and the image augmentation sample set with residue.
[0016] In some preferred embodiments, the feature extraction module includes a feature extraction backbone network and a cascaded aggregation interaction module;
[0017] The feature extraction backbone network is a ResNet18 network with the last average pooling layer and fully connected layer removed;
[0018] The cascaded aggregation interaction module performs cascaded aggregation interaction on the residual blocks output by the feature extraction backbone network to obtain multi-scale feature maps corresponding to the input image samples.
[0019] In some preferred embodiments, the residual module unit of the feature extraction backbone network ResNet18 is denoted as Residual module unit Previous online notes ,Will The subsequent feature mapping is denoted as The four parts of the cascaded aggregation interaction module are denoted as follows: ;
[0020] The cascading aggregation interaction process includes:
[0021] Step S21, map the features After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. ;
[0022] Step S22, feature mapping After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. ;
[0023] Step S23, feature mapping After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. ;
[0024] Step S24, feature mapping After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. The fusion feature This is the multi-scale feature map corresponding to the input image sample.
[0025] In some preferred embodiments, the joint attention weight adjustment module includes a channel attention adjustment network and a spatial attention adjustment network;
[0026] The channel attention adjustment network includes a globally average pooling layer, two fully connected layers, and a non-linear activation function Sigmoid, all connected in sequence.
[0027] The spatial attention adjustment network includes an average pooling layer along the channel dimension, a max pooling layer along the channel dimension, and a 1×1 convolutional layer. The average pooling layer and the max pooling layer along the channel dimension are parallel, and the two parallel pooling layers are sequentially connected to the 1×1 convolutional layer.
[0028] In some preferred embodiments, the dual-head output module includes a main classification head and an auxiliary interpretation head;
[0029] The main classification head includes an average pooling layer, a flattening layer, and a fully connected layer connected in sequence, used to obtain the classification results of container powder residue detection;
[0030] The auxiliary interpretation head includes a weighted summation feature map multiplication layer, a ReLU function processing layer, and an upsampling layer, used to obtain an interpretability heatmap corresponding to the classification results of container powder residue detection.
[0031] In some preferred embodiments, the loss function used in training the powder residue visual detection model is:
[0032]
[0033] in, Let be the loss function of the visual detection model for powder residue. The number of categories in the detection results. For classification loss value, The classification loss threshold, This is the activation function.
[0034] In some preferred embodiments, the classification loss value is calculated as follows:
[0035]
[0036] in, For the true category labels of the image samples, The predicted class label for the image samples output by the model.
[0037] In another aspect, the present invention provides an electronic device comprising:
[0038] At least one processor;
[0039] and a memory communicatively connected to at least one of the processors;
[0040] The memory stores instructions that can be executed by the processor to implement the aforementioned interpretable visual detection method for powder residues.
[0041] In a third aspect, the present invention provides a computer-readable storage medium storing computer instructions for execution by the computer to implement the above-described interpretable visual detection method for powder residues.
[0042] The beneficial effects of this invention are:
[0043] (1) This invention provides an interpretable visual detection method for powder residues, addressing typical problems in industrial visual detection: small training samples, poor interpretability of deep learning models, and high sample annotation costs. By combining techniques such as transfer learning, weakly supervised learning, and class activation maps, an interpretable visual detection model for powder residues suitable for small samples is built. The model only requires a simple binary classification labeled dataset for training. The model can not only predict the classification results but also provide an interpretable heatmap of the prediction results. Thus, it lays a solid theoretical foundation for the practical application of deep learning-based visual detection.
[0044] (2) This invention provides an interpretable visual detection method for powder residue. For industrial scenarios, an interpretable, lightweight powder residue detection model is established. This model consists of a feature extraction module, a joint attention adjustment module, and a dual-head output module. It can not only detect the presence of powder residue but also provide an interpretable heatmap of the classification results without requiring additional training parameters. This improves the interpretability of deep learning models in industrial applications.
[0045] (3) The present invention provides an interpretable visual detection method for powder residues. In the feature extraction module, a pre-trained ResNet18 based on the ImageNet dataset is selected as the basis for the feature extraction network. Based on this, a cascaded aggregation interaction module is designed to fuse the features of residual blocks that are more abstract and have rich scale information. This improves the expressive power of the features and improves the model training efficiency by using a pre-trained model.
[0046] (4) The present invention provides an interpretable visual detection method for powder residues. It designs a joint attention adjustment module consisting of channel attention adjustment and spatial attention adjustment components, thereby enhancing useful salient features and weakening ineffective redundant features. In the channel attention adjustment stage, the correlation between different channels is utilized to improve the model's channel adaptability. In the spatial attention adjustment stage, prior knowledge is introduced to enable the model to adaptively select useful spatial information. Attached Figure Description
[0047] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0048] Figure 1 This is a schematic flowchart of an interpretable visual detection method for powder residue according to the present invention;
[0049] Figure 2 This is a schematic diagram of the structure of a powder residue visual detection model according to an embodiment of an interpretable powder residue visual detection method of the present invention;
[0050] Figure 3 This is a schematic diagram of the feature extraction module of an embodiment of the interpretable visual detection method for powder residues according to the present invention;
[0051] Figure 4 This is a schematic diagram of the channel attention adjustment network of the joint attention weight adjustment module in one embodiment of the interpretable visual detection method for powder residue of the present invention;
[0052] Figure 5 This is a schematic diagram of the spatial attention adjustment network of the joint attention weight adjustment module in one embodiment of the interpretable visual detection method for powder residues of the present invention;
[0053] Figure 6 This is an example of an interpretable visual detection method for powder residues according to the present invention, which includes input image samples and a classification interpretation heatmap. Detailed Implementation
[0054] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.
[0055] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0056] This invention provides an interpretable visual detection method for powder residue, addressing typical problems in industrial visual detection: small training samples, poor interpretability of deep learning models, and high sample annotation costs. By combining techniques such as transfer learning, weakly supervised learning, and class activation maps, an interpretable visual detection model for powder residue suitable for small samples is built. This model requires only a simple binary classification labeled dataset for training. The model can not only predict classification results but also provide interpretable heatmaps of the prediction results. This lays a solid theoretical foundation for the practical application of deep learning-based visual detection.
[0057] The present invention provides an interpretable visual detection method for powder residue, the visual detection method comprising:
[0058] Step S10: Classify the collected powder container image set, and adjust the format and image size to obtain a set of images with and without residue, and a set of images with residue, with a set of formats and sizes.
[0059] Step S20: Construct a visual detection model for powder residue based on transfer learning. The visual detection model for powder residue includes a data augmentation module, a feature extraction module, a joint attention weight adjustment module, and a dual-head output module.
[0060] Step S30: Train the visual detection model for powder residue using the image sample set with and without residue.
[0061] Step S40: Detect powder residue in the container using the trained visual detection model for powder residue, and obtain the classification results and the corresponding interpretable heatmap.
[0062] To more clearly illustrate the interpretable visual detection method for powder residue of the present invention, the following is in conjunction with... Figure 1 The steps in the embodiments of the present invention will be described in detail below.
[0063] A first embodiment of the present invention provides an interpretable visual detection method for powder residue, comprising steps S10-S40, each step of which is described in detail below:
[0064] Step S10: Classify the collected powder container image set, and adjust the format and image size to obtain a set of image samples with and without residue, with the set format and size.
[0065] In one embodiment of the present invention, the hardware is first installed: a camera is vertically mounted on the powder residue production line, with the powder container placed vertically below it, ensuring the camera can fully capture images of the container; then, a dataset is collected: the camera acquires image samples of powder residue, and these samples are classified and saved as residue-free image samples and residue-containing image samples. All images are in JPG format with a resolution of 990×990 pixels. These image samples are used for training and validation of the powder residue visual detection model.
[0066] The dataset collected in this invention contains image samples that have the following three characteristics:
[0067] The residual powder has an inconsistent form, including powder, granules, and lumps, and the residual powder size varies from 0.5 mm to 5 mm.
[0068] The distribution of residual powder varies, including curved, scattered, flake, and blocky shapes.
[0069] The inner wall of the container holding the powder has scratches, cracks, wear, stains, etc., and these disturbances are similar to those of powder residue.
[0070] The complexity of powder residue poses a significant challenge to the construction of visual inspection models.
[0071] Step S20: Construct a visual detection model for powder residue based on transfer learning, such as... Figure 2 The diagram shown is a schematic representation of the structure of a powder residue visual detection model according to an embodiment of an interpretable powder residue visual detection method of the present invention. The powder residue visual detection model includes a data augmentation module, a feature extraction module, a joint attention weight adjustment module, and a dual-head output module.
[0072] The data augmentation module is used to expand the residual image sample set and the residual image sample set, including:
[0073] Based on the characteristics of the image samples, each image sample in the image sample set without residue and the image sample set with residue is randomly flipped, randomly scaled, and / or color jittered to obtain the image augmentation sample set without residue and the image augmentation sample set with residue.
[0074] like Figure 3The diagram shown is a schematic of the feature extraction module of an embodiment of the interpretable visual detection method for powder residues according to the present invention. The feature extraction module includes a feature extraction backbone network and a cascaded aggregation interaction module: the feature extraction backbone network is a ResNet18 network with the last average pooling layer and fully connected layer removed; the cascaded aggregation interaction module performs cascaded aggregation interaction on the residue blocks output by the feature extraction backbone network to obtain multi-scale feature maps corresponding to the input image samples.
[0075] The feature extraction backbone network of this invention is built on the ResNet18 network. The last average pooling layer and fully connected layer of the ResNet18 network are removed. At the same time, the ResNet18 network is pre-trained using the ImageNet2012 dataset, and the trained ResNet18 network is used as the initialization reference for the feature extraction backbone network of this invention, thereby enabling the model of this invention to converge faster during training.
[0076] The residual module unit of the feature extraction backbone network ResNet18 is denoted as Residual module unit Previous online notes ,Will The subsequent feature mapping is denoted as The four parts of the cascaded aggregation interaction module are denoted as follows: .
[0077] The cascading aggregation interaction process includes:
[0078] Step S21, map the features After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. ;
[0079] Step S22, feature mapping After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. ;
[0080] Step S23, feature mapping After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. ;
[0081] Step S24, feature mapping After the residual module unit Processing yields feature maps. and through Feature mapping and feature mapping The corresponding elements are added together and fused to obtain the fusion feature. The fusion feature This is the multi-scale feature map corresponding to the input image sample.
[0082] Given a 416×416 original image as input, the 13×13 feature vectors obtained after ResNet8 feature extraction through fusion, cascade, aggregation, and interactive processing are obtained. (That is, the multi-scale feature map corresponding to the original image with an input of 416×416), which is used for subsequent classification and generation of interpretable heatmaps.
[0083] The adjustment network consists of 2×2 average pooling layers, 1×1 convolutional layers, batch normalization (BN) layers, and nonlinear LeakyReLU activation function layers.
[0084] It is worth noting that the cascaded aggregation interaction module designed in this invention introduces a small number of parameters in the 1×1 convolution operation, thus achieving high efficiency in computation and generalization. Furthermore, compared to the original feature extraction backbone network in ResNet18, the cascaded aggregation interaction module in the feature extraction backbone network designed in this invention integrates multi-scale features, achieving comprehensive perception of residual features.
[0085] The joint attention weight adjustment module includes a channel attention adjustment network and a spatial attention adjustment network:
[0086] like Figure 4 The diagram shown is a schematic of the channel attention adjustment network of the joint attention weight adjustment module in an embodiment of the interpretable visual detection method for powder residue of the present invention. The channel attention adjustment network includes a globally average pooling layer, two fully connected layers and a nonlinear activation function Sigmoid connected in sequence.
[0087] Channel attention adjustment enhances feature maps relevant to the visual task by adjusting the channel weights of each feature map. The channel attention adjustment network performs global average pooling (GAP) on the features to generate feature channel descriptors, then obtains the dependencies of each feature channel through two fully connected layers (FC), and reconstructs the number of channels to reduce computational complexity. Furthermore, a non-linear activation function ReLU is introduced after FC to improve generalization ability, and a non-linear activation function Sigmoid is used to normalize the channel attention weights to [0,1]. Finally, by applying the channel attention weights to the features, feature channel adjustment is achieved.
[0088] like Figure 5 The diagram shown is a schematic of the spatial attention adjustment network of the joint attention weight adjustment module in an embodiment of the interpretable visual detection method for powder residue of the present invention. The spatial attention adjustment network includes an average pooling layer along the channel dimension, a max pooling layer along the channel dimension, and a 1×1 convolutional layer. The average pooling layer along the channel dimension and the max pooling layer along the channel dimension are parallel, and the two parallel pooling layers are sequentially connected to the 1×1 convolutional layer.
[0089] Spatial attention adjustment enhances the discriminative features in the spatial domain by readjusting spatial weights. During the spatial attention adjustment stage, average pooling and max pooling along the channel dimension are performed on the features to generate two feature space descriptors. These two feature space descriptors are then concatenated along the channel dimension to obtain a joint feature descriptor. A 1×1 convolutional layer is then applied to the joint feature descriptor.
[0090] The dual-head output module includes a main classification header and an auxiliary interpretation header:
[0091] The main classification head consists of a sequentially connected average pooling layer, a flattening layer, and a fully connected layer, used to obtain the classification results for container powder residue detection.
[0092] In the main classification head, a small classification network is designed to output the model's classification results. The main classification network structure includes an average pooling layer, a flattening layer, and a fully connected layer. Specifically, since the average pooling layer can maintain significant localization ability, the 512×13×13 feature map is average pooled along the spatial dimension to obtain a 512×1×1 feature map. Then, a flattening operation is performed to adjust the size of the feature map to 512. Finally, a linear layer is used to adjust the 512-dimensional feature map into a 2-dimensional output class.
[0093] The auxiliary interpretation head includes a weighted summation feature map multiplication layer, a ReLU function processing layer, and an upsampling layer, which are used to obtain the interpretability heatmap corresponding to the classification results of container powder residue detection.
[0094] In the auxiliary interpreter head, an interpretability heatmap of the corresponding model's classification output is obtained by multiplying the classification training weights of the main classification head with the features adjusted by the attention weights. The auxiliary interpreter head network structure specifically includes a weight and feature map multiplication layer, a ReLU function processing layer, and an upsampling layer. The weight and feature map multiplication layer multiplies the 512-dimensional weights of the main classification network with the attention-processed feature maps, then sums them to obtain a 13×13 feature heatmap. The ReLU function processing layer truncates the processed feature maps, thereby improving the contrast of the feature heatmap. The upsampling layer upsamples the 13×13 feature heatmap using bilinear interpolation to obtain a 416×416 feature heatmap.
[0095] Step S30: Train the visual detection model for powder residue using the image sample set with and without residue.
[0096] During model training, the raw parameters of the ResNet18 backbone network were pre-trained using ImageNet for classification. The pre-trained ResNet18 contained 1000 output classes, while the powder residue detection in this invention includes two classes: containing powder residue and not containing powder residue; therefore, the model's data was adjusted accordingly. Simultaneously, the batch size and number of training iterations were set to 64 and 30, respectively.
[0097] The loss function used in the training of the powder residue visual detection model of the present invention is shown in equation (1):
[0098]
[0099] in, Let be the loss function of the visual detection model for powder residue. The number of categories in the detection results. For classification loss value, The classification loss threshold, This is the activation function.
[0100] The optimizer used during training is Adam, with a learning rate of 1×10⁻⁵. The input image is resized to 416×416 pixels. Furthermore, the threshold for the classification loss in this invention... Set it to 0.01.
[0101] This invention comprises two main categories: those containing powder residue and those without powder residue. The loss function reflects the similarity between the true and predicted values. Generally, when the loss is less than a certain threshold, the prediction can be considered consistent with the true value, and there is no need to introduce a small loss during backpropagation. Therefore, to improve training efficiency and avoid overfitting, small losses below the designed threshold are eliminated during backpropagation to update the parameters.
[0102] The classification loss value is calculated as shown in equation (2):
[0103]
[0104] in, For the true category labels of the image samples, The predicted class label for the image samples output by the model.
[0105] Furthermore, the experimental environment for the visual detection model of this invention uses AI Studio, an online artificial intelligence computing environment maintained and provided by Baidu. Within AI Studio, the hardware and software details are: Intel(R) Xeon(R) Gold 6271C CPU, 16GB Nvidia V100 card, PaddlePaddle 2.4 with CUDA, and Ubuntu 18.04 operating system.
[0106] After the detection model is trained, the model parameters are saved in .pt format for subsequent model validation and testing.
[0107] Step S40: Detect powder residue in the container using the trained visual detection model for powder residue, and obtain the classification results and the corresponding interpretable heatmap.
[0108] like Figure 6 The image shown illustrates the input image sample and classification interpretation heatmap of an embodiment of the interpretable visual powder residue detection method of the present invention. The image to be detected is input into a trained powder residue detection model, and the model outputs the image's category information and a visualized category activation map. The visualized category activation map primarily provides a visual interpretation of the model's reasoning in the prediction results.
[0109] Although the steps in the above embodiments are described in the above order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not need to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple variations are all within the protection scope of this invention.
[0110] A second embodiment of the present invention provides an interpretable visual inspection system for powder residue, the visual inspection system comprising:
[0111] The image acquisition and preprocessing module is configured to classify the acquired powder container image set and adjust its format and image size to obtain a set of image samples with and without residue, with a set of image samples with residue and a set of image samples with a set format and size.
[0112] The model building module is configured to build a visual detection model for powder residue based on transfer learning. The visual detection model for powder residue includes a data augmentation module, a feature extraction module, a joint attention weight adjustment module, and a dual-head output module.
[0113] The model training module is configured to train a visual detection model for powder residue using the set of images with and without residue.
[0114] The detection and classification module is configured to detect powder residue in containers using a trained visual detection model for powder residue, and obtain classification results and corresponding interpretable heatmaps.
[0115] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0116] It should be noted that the interpretable powder residue visual inspection system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.
[0117] An electronic device according to a third embodiment of the present invention includes:
[0118] At least one processor;
[0119] and a memory communicatively connected to at least one of the processors;
[0120] The memory stores instructions that can be executed by the processor to implement the aforementioned interpretable visual detection method for powder residues.
[0121] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions for execution by the computer to implement the above-described interpretable visual detection method for powder residues.
[0122] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the storage device and processing device described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0123] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.
[0124] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.
[0125] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.
[0126] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. An interpretable visual inspection method for powder residue, characterized in that, The visual detection method includes: Step S10: Classify the collected powder container image set, and adjust the format and image size to obtain a set of images with and without residue, and a set of images with residue, with a set of formats and sizes. Step S20: Construct a visual detection model for powder residue based on transfer learning. The visual detection model for powder residue includes a data augmentation module, a feature extraction module, a joint attention weight adjustment module, and a dual-head output module. The feature extraction module includes a feature extraction backbone network, which is a ResNet18 network with the last average pooling layer and fully connected layer removed. The feature extraction module obtains multi-scale feature maps corresponding to the input image samples through cascaded aggregation interaction, specifically including: The residual module unit of the ResNet18 network is denoted as Residual module unit Previous online notes ,Will The subsequent feature mapping is denoted as ; Features go through Processing yields features and features and characteristics The corresponding elements are added and merged to obtain the features. ; Features go through Processing yields features and features After processing with Adj_2 and features The corresponding elements are added and merged to obtain the features. ; Features go through Processing yields features and features After processing with Adj_3 and features The corresponding elements are added and merged to obtain the features. ; Features go through Processing yields features and features After processing with Adj_4 and features The corresponding elements are added and merged to obtain the features. The features This refers to the multi-scale feature map corresponding to the input image sample; Among them, the structures of Adj_2, Adj_3 and Adj_4 are all 1 2×2 average pooling layer + 1 1×1 convolutional layer + 1 batch normalization layer + 1 LeakyReLU function; Step S30: Train the visual detection model for powder residue using the image sample set with and without residue. Step S40: Detect powder residue in the container using the trained visual detection model for powder residue, and obtain the classification results and the corresponding interpretable heatmap.
2. The interpretable visual detection method for powder residue according to claim 1, characterized in that, The data augmentation module is used to expand the residual image sample set and the residual image sample set, including: Randomly flip, randomly scale, and / or color-jitter each image sample in the image sample set without residue and the image sample set with residue, respectively, to obtain the image augmentation sample set without residue and the image augmentation sample set with residue.
3. The interpretable visual detection method for powder residue according to claim 1, characterized in that, The joint attention weight adjustment module includes a channel attention adjustment network and a spatial attention adjustment network; The channel attention adjustment network includes a globally average pooling layer, two fully connected layers, and a non-linear activation function Sigmoid, all connected in sequence. The spatial attention adjustment network includes an average pooling layer along the channel dimension, a max pooling layer along the channel dimension, and a 1×1 convolutional layer. The average pooling layer and the max pooling layer along the channel dimension are parallel, and the two parallel pooling layers are sequentially connected to the 1×1 convolutional layer.
4. The interpretable visual detection method for powder residue according to claim 3, characterized in that, The dual-head output module includes a main classification head and an auxiliary interpretation head; The main classification head includes an average pooling layer, a flattening layer, and a fully connected layer connected in sequence, used to obtain the classification results of container powder residue detection; The auxiliary interpretation head includes a weighted feature map multiplication layer, a ReLU function processing layer, and an upsampling layer, used to obtain an interpretable heatmap corresponding to the classification results of container powder residue detection, wherein the weights are the weights of the fully connected layer of the main classification head.
5. The interpretable visual detection method for powder residue according to claim 1, characterized in that, The loss function used in training the aforementioned visual detection model for powder residue is: ； in, Let be the loss function of the visual detection model for powder residue. The number of categories in the detection results. For classification loss value, The classification loss threshold, This is the activation function.
6. The interpretable visual detection method for powder residue according to claim 5, characterized in that, The classification loss value is calculated as follows: ； in, For the true category labels of the image samples, The predicted class label for the image samples output by the model.
7. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement an interpretable visual detection method for powder residues as described in any one of claims 1-6.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by the computer to implement an interpretable visual inspection method for powder residues as described in any one of claims 1-6.