Capsule defect detection method, device, electronic equipment and storage medium

Abstract

The present invention provides a capsule defect detection method, device, electronic device and storage medium, which belongs to the field of computer vision, including obtaining multiple historical capsule images, selecting a pre-trained network as a teacher network, and constructing a student network with the same structure as the teacher network, and training the two networks through historical capsule images; inputting the capsule image to be tested into the teacher network and the trained student network respectively, taking the difference between the output results of the two and taking the square, and finding the mean of the square along the network channel, and obtaining the output feature difference map of the teacher network and the student network; using bilinear interpolation to restore the feature difference map to the original image size, taking the maximum pixel value in the feature difference map as the abnormal score of the entire capsule, and using the abnormal score to classify the capsule into good and defective products, thereby realizing the detection of defective products; taking the pixel value of each area in the feature difference map as the defect score of the pixel, and classifying the defects at the pixel level according to the defect score, thereby realizing the location of the defects. The method can effectively detect defective capsules and locate defects of various sizes.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and specifically relates to a capsule defect detection method, device, electronic equipment and storage medium, specifically a capsule defect detection and defect positioning method. Background Art

[0002] With the continuous development of deep learning, the application of deep learning models in the field of computer vision has gradually become a reality. In the field of industrial anomaly detection, computer vision algorithms have attracted much attention and have made good research progress in many different tasks. As a task with great application value in industrial anomaly detection, capsule defective detection and defect localization have received extensive attention from relevant researchers. At present, capsule defective detection and defect localization tasks generally use methods based on deep learning, which can be mainly divided into two categories, one is based on supervised learning methods, and the other is based on unsupervised learning methods.

[0003] In the capsule defective detection method based on supervised learning, an essential step is image acquisition and manual labeling. However, since the defects of defective capsules are often small and difficult to detect with the naked eye, it is difficult to fully label all defects manually. In addition, some companies have low technical levels, poor production environments, and are prone to change, resulting in uneven imaging levels and unstable quality of collected images. Sometimes a small change on the assembly line can cause various irregular anomalies and easily produce unknown defects. Therefore, due to the subjectivity of manual labeling, the types of defects are difficult to be accurately distinguished, which in turn leads to confusion and missing labels. Even if companies spend a lot of money to label image datasets, there is a serious class imbalance problem in the dataset due to the low defective rate in the capsule production process.

[0004] The above problems make traditional supervised learning methods difficult to apply and cannot meet the needs of enterprises. Therefore, researchers have gradually paid attention to unsupervised learning methods. Existing unsupervised learning methods are based on image similarity, feature similarity, and knowledge distillation, but they each have some problems. The image similarity-based method is to input a normal capsule image to train the network so that it can reconstruct the image. In the detection stage, by comparing the reconstructed image with the input image, an abnormality score is given according to the reconstruction effect to determine whether the capsule is defective. Since the neural network has a certain generalization, sometimes defects can also be reconstructed, which makes it difficult for this type of method to determine whether the capsule is defective. The feature similarity-based method is to use a pre-trained network model to extract the features of the normal capsule image and store them in the database. In the detection stage, since the defective capsule features extracted by the network do not appear in the database, it is used to determine whether the capsule is defective and determine the defect location of the defective capsule. However, this type of method consumes a lot of memory during the detection process and has certain deployment difficulties in actual production scenarios. The knowledge distillation-based method is to use normal samples to train the teacher-student network so that the student network has the ability to learn the normal sample features of the teacher network. The core assumption of this type of method is that since no defective capsule samples were used during training, the output difference between the teacher network and the student network for defective capsules is too large during detection, so defects can be detected and located. However, due to the generalization of neural networks, the output difference between the teacher and student networks for defective capsules may be very small. Summary of the invention

[0005] In order to solve the problem that the output difference between the teacher and student networks for defective capsules is very small due to the generalization of the neural network, the present invention provides a capsule defect detection method. The present invention can use capsule images taken from the production line to construct a data set and train a model to perform defective inspection and defect location on the capsule, determine whether the capsule is defective, and determine the defect location of the defective capsule. The present invention plays an important role in the field of capsule quality inspection and can be extended to industrial product quality inspection in other fields.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A capsule defect detection method comprises the following steps:

[0008] Obtain multiple historical capsule images and construct a training dataset;

[0009] Select a pre-trained network as the teacher network, and build a student network with the same structure as the teacher network; input each capsule image in the training data set into the teacher network and the student network in turn, obtain the output features of the two networks, and take the average of the squares after subtracting the output features of the two networks to obtain the regression loss; at the same time, mask multiple capsule images to obtain multiple mask images, input multiple mask images into the student network to obtain multiple student features, and use the average of the squares of multiple student features as the mask loss. Use the gradient descent method to iteratively train the student network, and save the student network with the smallest sum of regression loss and mask loss;

[0010] The capsule image to be tested is input into the teacher network and the trained student network respectively, the output results of the two are subtracted and squared, and the mean of the squares is calculated along the network channel. The output feature difference map of the teacher network and the student network is obtained based on the mean of the squares;

[0011] Use bilinear interpolation to restore the feature difference map to the original size, take the maximum pixel value in the feature difference map as the anomaly score of the entire capsule, and use the anomaly score to classify the capsule into good and defective products;

[0012] The pixel values ​​of each area in the feature difference map are used as the defect scores of the pixels, and the defects are classified at the pixel level according to the defect scores, and the defects are located according to the defect classification results.

[0013] Preferably, before training the teacher network and the student network through the historical capsule images, the historical capsule images are further normalized, and the normalization of the historical capsule images is specifically as follows:

[0014] Capture capsule images from the company's production line to build a training data set The number of image channels is 3, h is the image height, w is the image width, there are N images in total, and the capsule images in the data set are standardized. The calculation formula is as follows:

[0015]

[0016] Among them, μ is the mean of the images in the teacher network pre-training dataset on the three channels, and δ is the standard deviation on the three channels.

[0017] Preferably, the teacher network structure is as follows:

[0018] The teacher network structure is a convolutional layer with a size of 4×4 and 128 channels, an average pooling layer with a size of 2×2 and a stride of 2, a convolutional layer with a size of 4×4 and 256 channels, an average pooling layer with a size of 2×2 and a stride of 2, a convolutional layer with a size of 3×3 and 256 channels, and a convolutional layer with a size of 4×4 and 384 channels.

[0019] Preferably, the teacher network and the student network are trained as follows:

[0020] Keeping the parameters of the teacher network unchanged, each image in the training data set is input into the teacher network and the student network respectively, and two output features T(I i )∈R C×H×W and S(I i )∈R C×H×W , where C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map. The average of the square losses of the two is the regression loss L reg , the calculation formula is as follows:

[0021]

[0022] The size of the mask block is randomly selected as k∈{2, 4, 8, 16}, and the image is divided into several blocks according to the size of the mask block. The number of blocks is N, and the calculation formula is as follows:

[0023]

[0024] Where N must be an integer, the image size is an integer multiple of the mask block size, and the length and width of the input image are the same size after cropping;

[0025] Generate an N 2 A sequence of sizes starting from 0, randomly select one-third of them and set them to 0, and set the rest to 1; arrange this sequence into a matrix of size N×N, copy the matrix into three, and splice it along the channel direction to form a tensor of size N×N×3, and then copy each element into a tensor of size k×k to obtain a mask M of size (Nk)×(Nk)×3. Since Nk=H=W, the size of the mask is also H×W×3. Multiply the original image and the mask to obtain the mask image I′ i , and its calculation formula is as follows:

[0026] I′ i =I i ⊙M

[0027] Input the mask image into the student network to obtain the output feature S(I i ′), take its square and average it as the mask loss L mask , and its calculation formula is as follows:

[0028]

[0029] The total loss is denoted as L total , and its calculation formula is as follows:

[0030] L total =L reg +Lmask

[0031] The student network is trained using the gradient descent method, and the student model parameter θ with the minimum total loss is saved. The model training is now complete.

[0032] Preferably, the specific steps of obtaining the output feature difference map of the teacher network and the student network are:

[0033] For the capsule to be detected, take its image p and standardize it. The calculation formula is as follows:

[0034]

[0035] Among them, μ is the mean of the images in the teacher network pre-training dataset on the three channels, and δ is the standard deviation on the three channels;

[0036] The capsule image to be tested is input into the trained teacher network and student network respectively, and two output features T(p)∈R are obtained. C×H×W and S(p)∈R C×H×W , the output features of the two are squared after difference, and averaged along the channel dimension to obtain the output feature difference map lossmap∈R of the teacher network and the student network H×W , the calculation formula is as follows:

[0037]

[0038] Preferably, the feature difference map is restored to the original size using bilinear interpolation, the maximum pixel value in the feature difference map is used as the abnormality score of the entire capsule, and the capsule is classified into good and defective products using the abnormality score, specifically as follows:

[0039] Use bilinear interpolation to upsample the output feature difference map and restore it to the original image size lossmap′∈R h×w ;

[0040] lossmap′=bilinear_interpolation(lossmap)

[0041] The maximum pixel value in the output feature difference map is used as the abnormality score of the entire capsule score = max(lossmap'). Using the abnormality score, an appropriate threshold threshold1 is selected to classify the capsule into good and defective products. The classification result result∈{0, 1} is obtained, where 0 represents good products and 1 represents defective products, thus realizing the detection of defective products.

[0042]

[0043] Preferably, the pixel values ​​of each area in the feature difference map are used as the defect scores of the pixels, the defects are classified at the pixel level according to the defect scores, and the defects are located according to the defect classification results, specifically:

[0044] The pixel values ​​in the output feature difference map are taken as the defect scores of the points, and the appropriate threshold threshold2 is selected. The defect classification at the pixel level is performed according to the defect scores, and the defect classification map resultmap∈{0,1} is obtained. h×w In the figure, the area with a pixel value of 0 is a normal area, and the area with a pixel value of 1 is a defect area, which realizes the location of the defect;

[0045]

[0046] The present invention also provides a capsule defect detection device, comprising:

[0047] The dataset construction module is used to obtain multiple historical capsule images and construct a training dataset;

[0048] The network training module is used to select a pre-trained network as the teacher network and build a student network with the same structure as the teacher network; each capsule image in the training data set is input into the teacher network and the student network in turn to obtain the output features of the two networks, and the regression loss is obtained by taking the average of the squares after subtracting the output features of the two networks; at the same time, multiple capsule images are masked to obtain multiple mask images, and the multiple mask images are input into the student network to obtain multiple student features, and the square average of multiple student features is used as the mask loss, and the student network is iteratively trained using the gradient descent method, and the student network with the smallest sum of regression loss and mask loss is saved;

[0049] The difference map acquisition module is used to input the capsule image to be tested into the teacher network and the trained student network respectively, take the square of the difference between the output results of the two, and calculate the mean of the square along the network channel. The output feature difference map of the teacher network and the student network is obtained according to the mean of the square;

[0050] The classification module is used to restore the feature difference map to the original image size using bilinear interpolation, take the maximum pixel value in the feature difference map as the anomaly score of the entire capsule, and use the anomaly score to classify the capsule into good and defective products;

[0051] The positioning module is used to use the pixel value of each area in the feature difference map as the defect score of the pixel, classify the defects at the pixel level according to the defect score, and locate the defects according to the defect classification result.

[0052] The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any one of the steps in the capsule defect detection method.

[0053] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is loaded by a processor, the computer program can execute any one of the steps in the capsule defect detection method.

[0054] The capsule defect detection method provided by the present invention has the following beneficial effects:

[0055] Compared with the existing anomaly detection method based on knowledge distillation, the present invention uses masks to simulate the defects of defective capsules, and adjusts the loss function to train the teacher network and the student network so that the outputs of the two for normal capsules are similar, and the outputs for abnormal capsules are different. At the same time, it overcomes the problem of no defective samples during training, reduces the difficulty of constructing data sets, and improves the utilization rate of data sets. In addition, by adding masks of different sizes to the input image, defects of different sizes are simulated, which conforms to the real production scene, can effectively detect defective capsules, and locate defects of various sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 This is a network structure diagram of a capsule defect detection method according to Embodiment 1 of the present invention;

[0058] Figure 2 It is the structural diagram of the capsule defect detection device. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0060] Example 1

[0061] The present invention provides a method for detecting capsule defects, specifically Figure 1 As shown, the following steps are included:

[0062] S1. Take historical capsule images from the production line of the enterprise, build a training data set, and standardize the historical capsule images in the data set. The purpose of standardization is to make the data set more consistent with the data distribution of the teacher network pre-training data set and improve the detection results of the model.

[0063] S11. Take historical capsule images from the company’s production line to build a training dataset The number of image channels is 3, h is the image height, w is the image width, there are N images in total, and the capsule images in the data set are standardized. The calculation formula is as follows:

[0064]

[0065] Among them, μ is the mean of the images in the teacher network pre-training dataset on the three channels, and δ is the standard deviation on the three channels.

[0066] S2. Select a lightweight network that has been pre-trained on a large dataset as the teacher network, analyze its structure and design a student network with the same structure as the teacher network.

[0067] In this embodiment, the teacher network is a trained network, the parameters of which have reached the optimal state and can extract the features of the image well. The student network is an untrained network, the parameters of which are randomly initialized, and the extracted features cannot represent the information of the image. The goal of the present invention is to train the student network so that the features extracted by the student network on the capsule data are as similar as possible to those of the teacher network, so that the student network has the ability of the teacher network, and therefore is called the teacher network and the student network.

[0068] S21. Select a lightweight network that has been pre-trained on a large data set as the teacher network T, analyze its structure and design a student network S with the same structure as the teacher network. The teacher network structure selected by the present invention is a convolutional layer with a size of 4×4 and 128 channels, an average pooling layer with a size of 2×2 and a stride of 2, a convolutional layer with a size of 4×4 and 256 channels, an average pooling layer with a size of 2×2 and a stride of 2, a convolutional layer with a size of 3×3 and 256 channels, and a convolutional layer with a size of 4×4 and 384 channels.

[0069] S3. Keep the parameters of the teacher network unchanged, input each image in the training data set into the teacher network and the student network respectively, and make it regress the output of the teacher network to obtain the regression loss. At the same time, mask the image, input the masked image into the network to obtain the mask loss, use the gradient descent method to train the student network, and save the student model with the minimum total loss. Here, iterative training is performed for several rounds, and the loss of each round will change. Overall, the loss will become smaller and smaller, and it will decrease in a fluctuating manner. The model with the minimum loss is saved.

[0070] S31, keep the parameters of the teacher network unchanged, input each image in the training data set into the teacher network and the student network respectively, and obtain two output features T(I i )∈R C×H×W and S(I i )∈R c×H×W , where C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map. The average of the square losses of the two is the regression loss L reg , the calculation formula is as follows:

[0071]

[0072] S32. Randomly select the size of the mask block as k∈{2, 4, 8, 16}. This step is to simulate anomalies of different sizes and increase the robustness of the model. The image is divided into several blocks according to the size of the mask block. The number of blocks is N. The calculation formula is as follows:

[0073]

[0074] N must be an integer, because the input image size is an integer multiple of the mask block size, and the length and width of the input image are the same size after cropping.

[0075] Generate an N 2 A sequence of size starting from 0, randomly select one third of them and set them to 0, and set the rest to 1. Arrange this sequence into a matrix of size N×N, copy the matrix into three, and splice it along the channel direction to form a tensor of size N×N×3, and then copy each element into a tensor of size k×k to obtain a mask M of size (Nk)×(Nk)×3. Since Nk=H=W, the size of the mask is also H×W×3. Multiply the original image with the mask to obtain the mask image I′ i , and its calculation formula is as follows:

[0076] I′ i =I i ⊙M

[0077] S33, input the mask image into the student network to obtain the output feature S(I i′), take its square and average it as the mask loss L mask , and its calculation formula is as follows:

[0078]

[0079] The total loss is denoted as L total , and its calculation formula is as follows:

[0080] L total =L reg +L mask

[0081] The student network is trained using the gradient descent method, and the student model parameter θ with the minimum total loss is saved. The model training is now complete and can be used for detection.

[0082] S4. For the capsule to be tested, take its image and standardize it. The capsule image to be tested is input into the trained teacher network and student network respectively, and the output results of the two are subtracted to obtain the output feature difference map of the teacher network and the student network.

[0083] S41. For the capsule to be detected, take its image p and standardize it. The calculation formula is as follows:

[0084]

[0085] Among them, μ is the mean of the images in the teacher network pre-training dataset on the three channels, and δ is the standard deviation on the three channels.

[0086] S42, input the capsule image to be tested into the trained teacher network and student network respectively, and obtain two output features T(p)∈R C×H×W and S(p)∈R C×H×W , the output features of the two are subtracted and averaged along the channel dimension to obtain the output feature difference map lossmap∈R between the teacher network and the student network H×W , the calculation formula is as follows:

[0087]

[0088] S5. Use bilinear interpolation to upsample the feature difference map to restore it to the original image size. Use the maximum pixel value in the feature difference map as the abnormality score of the entire capsule, and use it to classify the capsule into good and defective products, thereby realizing the detection of defective products.

[0089] S51, use bilinear interpolation to upsample the feature difference map to restore it to the original image size lossmap′∈R h×w :

[0090] lossmap′=bilinear_interpolation(lossmap)

[0091] The maximum pixel value in the feature difference map is taken as the abnormality score of the entire capsule score = max(lossmap'). Using the abnormality score, an appropriate threshold threshold1 is selected to classify the capsule into good and defective products. The classification result result∈{0, 1} is obtained, where 0 represents good products and 1 represents defective products, thus realizing the detection of defective products.

[0092]

[0093] S6. The pixel values ​​of each area in the feature difference map are used as the defect scores of the pixels, and the defects are classified at the pixel level according to the defect scores, thereby realizing the location of the defects.

[0094] Specifically, the pixel value in the feature difference map is used as the defect score of the point, and a suitable threshold threshold2 is selected. According to the defect score, the defect classification at the pixel level is performed to obtain the defect classification map resultmap∈{0,1} h×w In the figure, the area with a pixel value of 0 is the normal area, and the area with a pixel value of 1 is the defect area, which realizes the positioning of the defect.

[0095]

[0096] Based on the same inventive concept, the present invention also provides a capsule defect detection device, such as Figure 2 As shown, it includes a dataset construction module, a network training module, a difference map acquisition module, a classification module and a positioning module.

[0097] Specifically, the dataset construction module is used to obtain multiple historical capsule images, construct a training dataset, and standardize the historical capsule images in the dataset; the network training module is used to select a pre-trained network as a teacher network, and construct a student network with the same structure as the teacher network; each capsule image in the training dataset is input into the teacher network and the student network in turn to obtain the output features of the two networks, and the regression loss is obtained by taking the average of the squares of the output features of the two networks after subtracting them; at the same time, multiple capsule images are masked to obtain multiple mask images, and the multiple mask images are input into the student network to obtain multiple student features, and the average of the squares of multiple student features is used as the mask loss, and the student network is iteratively trained using the gradient descent method to save the student network with the smallest sum of the regression loss and the mask loss.

[0098] The difference map acquisition module is used to input the capsule image to be tested into the teacher network and the trained student network respectively, take the square of the difference between the two output results, and find the mean of the square along the network channel. The output feature difference map of the teacher network and the student network is obtained according to the mean of the square; the classification module is used to restore the feature difference map to the original image size using bilinear interpolation, and take the largest pixel value in the feature difference map as the abnormality score of the entire capsule, and use it to classify the capsule into good and defective products. The positioning module is used to take the pixel value of each area in the feature difference map as the defect score of the pixel, classify the defects at the pixel level according to the defect score, and locate the defects according to the defect classification result.

[0099] Each module in the above-mentioned capsule defect detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0100] The present invention also provides an electronic device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the capsule defect detection method embodiment. The specific implementation method can be found in the method embodiment, which will not be described in detail here.

[0101] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the instructions can be executed by a processor of a computer device to complete the above method. For example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the capsule defect detection method embodiment can be implemented. The specific implementation method can be referred to the method embodiment, which will not be repeated here.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] It should be noted that the above-described specific implementation methods can enable those skilled in the art to more fully understand the invention, but do not limit the invention in any way. Therefore, although the present specification and embodiments have described the invention in detail, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

Claims

1. A capsule defect detection method, characterized in that: The following steps are involved: Obtain multiple historical capsule images and construct a training dataset; Select a pre-trained network as the teacher network, and build a student network with the same structure as the teacher network; Training the student network includes: inputting each capsule image in the training data set into the teacher network and the student network in turn, obtaining output feature maps of the two networks, taking the average of the squares after subtracting the output feature maps of the two networks to obtain regression loss, and at the same time, masking multiple capsule images to obtain multiple mask images, inputting the multiple mask images into the student network to obtain multiple output features, taking the average of the squares of the multiple output features as the mask loss, iteratively training the student network using the gradient descent method, and saving the student network with the smallest sum of the regression loss and the mask loss; The capsule image to be tested is input into the teacher network and the trained student network respectively, the output results of the two are squared after subtracting them, and the mean of the squares is calculated along the channel dimension. The output feature difference map of the teacher network and the student network is obtained based on the mean of the squares; Use bilinear interpolation to restore the output feature difference map to the size of the capsule image to be tested, take the maximum pixel value in the restored output feature difference map as the anomaly score of the entire capsule, and use the anomaly score to classify the capsule into good and defective products; The pixel values ​​of each area in the restored output feature difference map are used as the defect scores of the pixels, and the defects are classified at the pixel level according to the defect scores, and the defects are located according to the defect classification results; The training of the student network is specifically as follows: Keeping the parameters of the teacher network unchanged, each capsule image in the training data set is input into the teacher network and the student network respectively, and two output feature maps T(I i )∈R C×H×W and S(I i )∈R C×H×W , where I i is the capsule image, C is the number of channels of the output feature map, H is the height of the output feature map, W is the width of the output feature map, and the regression loss L is obtained by taking the square average of the difference between the output feature maps of the two networks. reg , the calculation formula is as follows: The size of the mask block is randomly selected as k∈{2, 4, 8, 16}, and the capsule images in the training data set are divided into several blocks according to the size of the mask block. The number of blocks is N, and the calculation formula is as follows: Where h is the image height, w is the image width, N must be an integer, the capsule image size is an integer multiple of the mask block size, and the length and width of the capsule images in the training dataset are the same size after cropping, where the number of image channels of the capsule images in the training dataset is 3; Generate an N 2 A sequence of sizes starting from 0, randomly select one-third of them and set them to 0, and set the rest to 1; arrange this sequence into a matrix of size N×N, copy the matrix into three, and splice it along the channel direction to form a tensor of size N×N×3, and then copy each element into a tensor of size k×k to obtain a mask M of size (Nk)×(Nk)×3. Since Nk=h=w, the size of the mask is also h×w×3. Multiply the capsule image in the training data set with the mask to obtain the mask image I′ i , and its calculation formula is as follows: I′ i =I i ⊙M Input the mask image into the student network to obtain the output feature S(I i ′), take its square and average it as the mask loss L mask , and its calculation formula is as follows: The total loss is denoted as L total , and its calculation formula is as follows: L total =L reg +L mask The student network is trained using the gradient descent method, and the student model parameter θ with the minimum total loss is saved. The model training is now complete.

2. A capsule defect detection method according to claim 1, characterized in that: The teacher network structure is as follows: The teacher network structure includes a convolutional layer of 4×4 size and 128 channels, an average pooling layer of 2×2 size and stride 2, a convolutional layer of 4×4 size and 256 channels, an average pooling layer of 2×2 size and stride 2, a convolutional layer of 3×3 size and 256 channels, and a convolutional layer of 4×4 size and 384 channels.

3. A capsule defect detection method according to claim 1, characterized in that: The output feature difference map is restored to the size of the capsule image to be tested using bilinear interpolation, the maximum pixel value in the restored output feature difference map is used as the abnormality score of the entire capsule, and the capsule is classified into good and defective products using the abnormality score, specifically: The output feature difference map is upsampled by bilinear interpolation, and the output feature difference map is restored to the size of the capsule image to be tested, and the restored output feature difference map lossmap′∈R is obtained. h×w ; The maximum pixel value in the restored output feature difference map is taken as the abnormal score of the entire capsule. The abnormal score is used to select the threshold threshold1 to classify the capsule into good and defective products. The classification result result∈{0,1} is obtained, where 0 represents good products and 1 represents defective products, thus realizing the detection of defective products.

4. A capsule defect detection method according to claim 3, characterized in that: The pixel values ​​of each region in the restored output feature difference map are used as the defect scores of the pixels, the defects are classified at the pixel level according to the defect scores, and the defects are located according to the defect classification results, specifically: The pixel values ​​in the restored output feature difference map are taken as the defect scores of the points, and the appropriate threshold threshold2 is selected. The defect classification at the pixel level is performed according to the defect scores, and the defect classification map resultmap∈{0,1} is obtained. h×w In the figure, the area with a pixel value of 0 is a normal area, and the area with a pixel value of 1 is a defect area, which realizes the location of the defect; 5. A detection device for executing the capsule defect detection method according to claim 1, characterized in that: include: The dataset construction module is used to obtain multiple historical capsule images and construct a training dataset; The network training module is used to select a pre-trained network as the teacher network and build a student network with the same structure as the teacher network; Training the student network includes: inputting each capsule image in the training data set into the teacher network and the student network in turn, obtaining output feature maps of the two networks, taking the average of the squares after subtracting the output feature maps of the two networks to obtain regression loss, and at the same time, masking multiple capsule images to obtain multiple mask images, inputting the multiple mask images into the student network to obtain multiple output features, taking the average of the squares of the multiple output features as the mask loss, iteratively training the student network using the gradient descent method, and saving the student network with the smallest sum of the regression loss and the mask loss; The difference map acquisition module is used to input the capsule image to be tested into the teacher network and the trained student network respectively, take the square of the difference between the two output results, and calculate the mean of the squares along the channel dimension. The output feature difference map of the teacher network and the student network is obtained according to the mean of the squares; The classification module is used to restore the output feature difference map to the size of the capsule image to be tested using bilinear interpolation, and the maximum pixel value in the restored output feature difference map is used as the anomaly score of the entire capsule, and the capsule is classified into good and defective products using the anomaly score; The positioning module is used to use the pixel value of each area in the restored output feature difference map as the defect score of the pixel, classify the defects at the pixel level according to the defect score, and locate the defects according to the defect classification result.

6. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded into a processor, the computer program can execute the steps of the method according to any one of claims 1 to 4.

Images