Industrial defect detection model construction method, industrial defect detection method, industrial defect detection device, industrial defect detection equipment and storage medium
Through the teacher network and student network combining artificial synthesis abnormal module and multi-scale feature fusion module, pseudo-defect sample images are generated and loss functions are optimized, which solves the problem of low accuracy in defect detection of complex structural industrial products and realizes high-precision defect detection.
Patent Information
- Application Number
- CN202411075647.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
When detecting complex structural industrial products, the existing defect detection algorithm has low accuracy and is difficult to obtain defect samples, resulting in poor robustness and prone to missed or missed inspections.
The teacher network and student network are combined with artificial synthesis exception module (SAM_AB) and multi-scale feature fusion module (MSFFM), and simulate unknown type defects by generating pseudo-defect sample images, introduce pseudo-defect features, and enhance the feature fusion and loss function optimization. Defect detection capabilities of the defect detection model.
The accuracy and segmentation accuracy of industrial defect detection have been significantly improved, and the image-level AUROC, pixel-level AUROC and AUCPRO have reached 98.4%, 97.2% and 95.1% respectively, improving the ability to detect defects of complex structural products.
Smart Images

Figure CN119090810A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of defect detection technology, and in particular to an industrial defect detection model construction method, detection method, device, equipment and storage medium. Background Art
[0002] Defect detection is an important field of machine vision research. It is widely used in food, medical, biological, industrial production, military technology and other fields. It has important research value and broad application prospects. In recent years, with the development of deep learning technology, many defect detection algorithms have been proposed, and research related to industrial defect detection has also made breakthrough progress. At present, the mainstream defect detection algorithms are divided into two categories: non-knowledge distillation and knowledge distillation. Among them, the knowledge distillation algorithm has been widely used in recent years due to its higher interpretability, flexibility and adaptability. However, although the detection model based on the knowledge distillation algorithm has improved the segmentation accuracy of industrial defects, it still only has normal samples as guidance during the training process, has poor robustness, and has poor defect detection effect on industrial products with complex structures.
[0003] With the improvement of the automation level of industrial production lines, enterprises have increasingly stringent requirements on product quality, and defect detection has become an important task to assist automated production. However, due to the continuous improvement of industrial production technology, defect samples are difficult to obtain, and the lack of guidance from defect samples can easily lead to missed detections. For industrial products with complex structures, various unknown types of defects may occur during the production process. Existing defect detection algorithms are prone to ignore or over-retain detailed features, resulting in false detections, making industrial defect detection still challenging. Summary of the invention
[0004] In view of this, the present application aims to provide an industrial defect detection model construction method, detection method, device, equipment and storage medium to solve the technical problem that the existing defect detection algorithm has poor defect detection accuracy for industrial products with complex structures and unknown defect types due to the complex structure of industrial products, various unknown types of defects that may occur during the production process, and the difficulty in obtaining defect samples.
[0005] On the one hand, the present application provides a method for building an industrial defect detection model, the model building method comprising:
[0006] S11, constructing a defect detection model, the defect detection model includes: a teacher network, a student network, an artificial synthesis anomaly module SAM_AB and a multi-scale feature fusion module MSFFM; the artificial synthesis anomaly module SAM_AB is used to generate a pseudo-defect sample image from a normal sample image, simulate an unknown type of defect, introduce pseudo-defect features in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has a stronger defect detection capability; the multi-scale feature fusion module MSFFM is used to fuse the shallow features and deep features output by the teacher network and the student network respectively;
[0007] S12, training the defect detection model, including: S121, generating a pseudo-defect sample image from a normal sample image through the artificially synthesized abnormality module SAM_AB; S122, inputting the normal sample image into the teacher network pre-trained on the ImageNet dataset, and inputting the pseudo-defect sample image into the student network, wherein the teacher network extracts features of the normal sample image, guides the student network to remove the defect features of the pseudo-defect sample image, and restores the normal sample image; S123, converting the shallow features output by the teacher network into the pseudo-defect sample image; and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. ; S124, calculating the fusion features of the teacher network and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map The image is visualized as a heat map, and different colors are used to distinguish between areas with higher abnormality scores and normal areas, thereby realizing defect detection and segmentation of the image and completing the training of the defect detection model.
[0008] Furthermore, in S121, the process of generating a pseudo-defect sample image from a normal sample image through the artificial synthetic abnormality module SAM_AB is as follows: the original image of the normal sample image is adaptively binarized to obtain a foreground position, and the Gaussian noise is thresholded to obtain a preliminary artificial defect position; then the foreground position is multiplied by the preliminary artificial defect position to obtain the artificial defect position; then the artificial defect position is multiplied by the texture image to obtain a more realistic synthetic defect, and finally the original image outside the defect position is added to the synthetic defect to obtain an artificially synthesized pseudo-defect sample image.
[0009] Furthermore, in S123, the multi-scale feature fusion module MSFFM selects the shallow features and deep features output by the teacher network and the student network after two average pooling layers as input features, so that the student network can better take into account detail information and semantic information. Specifically: the deep features of the student network After 1×1 convolution and ReLU activation function processing, the trainable weights are obtained , the shallow features of the student network After 1×1 convolution and bilinear interpolation downsampling, the deep features of the student network are obtained Intermediate features with the same size are processed by 1×1 convolution and ReLU activation function to obtain trainable weights , With the deep features of the student network , The features obtained by multiplying the intermediate features are added together to obtain the fusion features of the student network. ; The teacher network uses the same weights as the student network and , the shallow features of the teacher network After 1×1 convolution and bilinear interpolation downsampling, the deep features of the teacher network are obtained Intermediate features with the same size, With the deep features of the teacher network , The features obtained by multiplying the intermediate features are added together to obtain the fusion features of the teacher network. .
[0010] Furthermore, in S124, using the abnormal score map To represent the model's segmentation results for pixel-level defects, the abnormal score map The calculation formula is: ,in, The fusion features of the teacher network And the fusion characteristics of the student network The number of channels, and are the fusion features of the student network No. The features of the layer channels and the fusion features of the teacher network No. Characteristics of layer channels.
[0011] Furthermore, SSIM loss is introduced on the shallow features and deep features respectively, and the shallow features of the teacher network are calculated using the SSIM loss function and the shallow features of the student network , the deep features of the teacher network and the deep features of the student network The loss between.
[0012] Furthermore, the squared error is used as the fusion feature of the teacher network And the fusion characteristics of the student network The loss between them, SSIM loss and weighted average of squared difference are used as the overall loss in the distillation stage. , the calculation formula is: ,in, Anomaly score map of loss, is the SSIM loss, Used to balance the impact of SSIM loss on the overall loss function; Abnormal score graph Loss The calculation formula is: ,in, , The abnormal score graphs are The height and width of the SSIM loss are calculated as: ,in, Represents the feature map of the teacher network, The feature graph representing the student network, It is a shallow feature. When is a deep feature, , They are , The average brightness, , They are , The standard deviation of for and The covariance of and is a small constant that ensures that the denominator is non-zero.
[0013] On the other hand, the present application provides an industrial defect detection method, the defect detection method comprising:
[0014] S21, acquiring an image of the industrial product to be inspected;
[0015] S22, inputting the image of the industrial product to be inspected into the defect detection model constructed by the above-mentioned defect detection model construction method to perform feature extraction, and outputting the defect detection result.
[0016] On the other hand, the present application also provides an industrial defect detection device, the device comprising:
[0017] An image acquisition module, used to acquire images of industrial products to be inspected;
[0018] The defect detection module is used to perform defect detection on the image of the industrial product to be detected acquired by the image acquisition module using the defect detection model constructed by the above-mentioned defect detection model construction method, and output the defect detection result.
[0019] On the other hand, the present application also provides a device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the above-mentioned defect detection model construction method and / or the above-mentioned defect detection method is implemented.
[0020] On the other hand, the present application also provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the above-mentioned defect detection model construction method and / or the above-mentioned defect detection method are implemented.
[0021] One or more technical solutions proposed in this application have at least the following beneficial effects compared with the prior art:
[0022] Firstly, the normal sample images are converted into pseudo-defect sample images through the artificial synthesis anomaly module SAM_AB to simulate unknown types of defects. Pseudo-defect features are introduced in the knowledge distillation process of the teacher network and the student network to make the defect detection model have stronger defect detection ability. Then, the shallow features and deep features output by the teacher network and the student network are fused through the feature fusion module MSFFM to enhance the defect detection model's ability to extract detailed features of industrial products. At the same time, SSIM loss is introduced into the loss function to improve the segmentation accuracy of defects in industrial products with complex structures.
[0023] Through experiments on the industrial dataset MVTec AD, the image-level AUROC, pixel-level AUROC and AUCPRO of the defect detection model proposed in this application reached 98.4%, 97.2% and 95.1% respectively, effectively improving the accuracy and segmentation precision of industrial defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0025] Figure 1 The present invention is a flow chart of a defect detection model building method according to an exemplary embodiment.
[0026] Figure 2 It is a schematic structural diagram of a defect detection model according to an exemplary embodiment.
[0027] Figure 3 The figure is a flow chart of a defect detection model training method according to an exemplary embodiment.
[0028] Figure 4 It is a process diagram of generating a pseudo defect sample image by an artificial synthetic abnormality module SAM_AB according to an exemplary embodiment.
[0029] Figure 5 It is a schematic diagram showing the comparison results of human synthetic anomalies using a single threshold and adaptive binarization for industrial images with different background colors according to an exemplary embodiment.
[0030] Figure 6 FIG. 4 is a schematic diagram of a pixel extraction area according to an exemplary embodiment.
[0031] Figure 7 Schematic diagram of the structure of a multi-scale feature fusion module MSFFM according to an exemplary embodiment.
[0032] Figure 8 The figure is a flow chart of a defect detection method according to an exemplary embodiment. DETAILED DESCRIPTION
[0033] In order to better explain the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings of the specification and the specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of this application. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail to highlight the subject matter of the present application. Similarly, for clarity and conciseness, the description of known functions and structures is omitted in the following description.
[0034] Embodiment 1
[0035] Since industrial products have complex structures, various unknown types of defects may occur during the production process, and defect samples are difficult to obtain, industrial defect detection is still challenging. Therefore, this embodiment provides a method for building an industrial defect detection model.
[0036] See also Figures 1 to 7 , the model building method comprises:
[0037] S11, constructing a defect detection model, wherein the defect detection model includes: a teacher network, a student network, an artificial synthetic anomaly module SAM_AB and a multi-scale feature fusion module MSFFM.
[0038] If the student network only receives guidance from normal samples during the training process, when the normal samples change, for example, different lighting causes different sample colors, or the defect features are similar to normal features, the student network cannot distinguish between normal features and defect features well, and the model is prone to overfitting. Therefore, in order to introduce the guidance of defect features during the model training process and enhance the student network's ability to distinguish between normal features and defect features, the defect detection model proposed in this embodiment adds the artificial synthetic anomaly module SAM_AB. The artificial synthetic anomaly module SAM_AB generates pseudo-defect sample images from normal sample images to simulate unknown types of defects. Pseudo-defect features are introduced in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has stronger defect detection capabilities.
[0039] The shallow features of the image contain more detail information but more noise, while the deep features have stronger semantic information but lower resolution. In order to better detect defects in industrial products with complex structures, the defect detection model proposed in this embodiment adds the multi-scale feature fusion module MSFFM. The multi-scale feature fusion module MSFFM can fuse the shallow features and deep features output by the teacher network and the student network respectively, so that the student network can better take into account the detail information and semantic information, thereby enhancing the detection capability of defects in industrial products with complex structures.
[0040] S12, training the defect detection model to obtain the trained defect detection model. Specifically, the following steps are included:
[0041] S121, the normal sample image is generated into a pseudo defect sample image through the artificial synthesis abnormal module SAM_AB. The specific process is: the original image of the normal sample image The foreground position is obtained by adaptive binarization , Gaussian noise The initial artificial defect position is obtained by thresholding ; Then set the foreground position and preliminary artificial defect location Multiply to get the position of the artificial defect ; Then the artificial defect position With texture image Multiply together to get a more realistic composite defect Finally, the original image outside the defect position is added to the synthetic defect to obtain the artificially synthesized pseudo-defect sample image. .
[0042] The defects simulated by the above method will only appear in the target area, which can better simulate the abnormal conditions in the real world and provide abnormal feature guidance to the defect detection model, so that the student network can better restore normal features.
[0043] Among them, adaptive binarization is to solve the problem that a single threshold cannot extract the foreground position when the background color changes. The comparison results of using a single threshold and adaptive binarization for human synthetic anomalies are as follows: Figure 5 As shown, from Figure 5 It can be seen that a single threshold can only extract light-colored areas. When the defect detection model is applied to the capsule category, the background area will be used as the foreground, and the simulated defects will be added to the background area, which violates the laws of nature. When adaptive binarization is used, whether it is a metal nut with a dark background or a capsule with a light background, it can ensure that the simulated defects are generated in the product area, which is consistent with the defect generation situation in the production process. The calculation of adaptive binarization is shown in formula (1).
[0044] (1)
[0045] in, The foreground image is extracted from the original image The threshold value, Generate a binary image when the background is dark The threshold value of Generate a binary image when the background is light The threshold value of is the threshold for determining whether the object is a texture product; is the threshold for determining whether the background is dark; is the mean value of pixels in the center area of the image; is the mean pixel value in the corner area, and the corresponding pixel extraction area is Figure 6 shown.
[0046] S122, input the normal sample image into the teacher network pre-trained on the ImageNet dataset, input the pseudo-defect sample image into the student network, the teacher network extracts the features of the normal sample image, guides the student network to remove the defect features of the pseudo-defect sample image, and restores the normal sample image.
[0047] S123, the shallow features output by the teacher network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. .
[0048] Specifically: In S123, the multi-scale feature fusion module MSFFM selects the shallow features and deep features output by the teacher network and the student network after two average pooling layers as input features, so that the student network can better take into account detail information and semantic information. Specifically: the deep features of the student network After 1×1 convolution and ReLU activation function processing, the trainable weights are obtained , the shallow features of the student network After 1×1 convolution and bilinear interpolation downsampling, the deep features of the student network are obtained Intermediate features with the same size are processed by 1×1 convolution and ReLU activation function to obtain trainable weights , With the deep features of the student network , The features obtained by multiplying the intermediate features are added together to obtain the fusion features of the student network. ; The teacher network uses the same weights as the student network and , the shallow features of the teacher network After 1×1 convolution and bilinear interpolation downsampling, the deep features of the teacher network are obtained Intermediate features with the same size, With the deep features of the teacher network , The features obtained by multiplying the intermediate features are added together to obtain the fusion features of the teacher network. The calculation is shown in formula (2).
[0049] (2)
[0050] in, is the trainable weight of the shallow features, is the trainable weight of the deep feature, calculated as shown in formula (3), represents convolution, Stands for Bilinear Interpolation Downsampling.
[0051] (3)
[0052] S124, calculating the fusion features of the teacher network and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map The image is visualized as a heat map, and different colors are used to distinguish between areas with higher abnormality scores and normal areas, thereby realizing defect detection and segmentation of the image and completing the training of the defect detection model.
[0053] In this embodiment, the abnormal score map is used To represent the model's segmentation results for pixel-level defects, the abnormal score map The calculation of is as shown in formula (4).
[0054] (4)
[0055] in, The fusion features of the teacher network And the fusion characteristics of the student network The number of channels, and are the fusion features of the student network No. The features of the layer channels and the fusion features of the teacher network No. Characteristics of layer channels.
[0056] Since the shallow features and deep features contain different information, in order to better distill the normal sample features into the student network, in this embodiment, SSIM loss is introduced on the shallow features and deep features respectively, and the shallow features of the teacher network are calculated using the SSIM loss function. and the shallow features of the student network , the deep features of the teacher network and the deep features of the student network The loss between the two, taking into account the brightness, contrast and structure of the image, can better reflect the intuitive feeling of human vision on the image quality.
[0057] Use the squared error as the fusion feature of the teacher network And the fusion characteristics of the student network The loss between them, SSIM loss and weighted average of squared difference are used as the overall loss in the distillation stage. , calculated as formula (5).
[0058] (5).
[0059] in, Anomaly score map of loss, is the SSIM loss, Used to balance the impact of SSIM loss on the overall loss function.
[0060] Anomaly score graph Loss The calculation is as in formula (6).
[0061] (6)
[0062] in, , The abnormal score graphs are The height and width of the
[0063] The calculation of SSIM loss is as shown in formula (7).
[0064] (7)
[0065] in, Represents the feature map of the teacher network, The feature graph representing the student network, It is a shallow feature. When is a deep feature, , They are , The average brightness, , They are , The standard deviation of for and The covariance of and is a small constant that ensures that the denominator is non-zero.
[0066] In order to prove the effectiveness of the improved loss function in this embodiment, ablation experiments are carried out on the defect detection model proposed in this embodiment under different loss functions, and the results are shown in Table 1.
[0067] Table 1 Loss function hyperparameter ablation experiment
[0068]
[0069] Use alone When used as a loss function, the image-level and pixel-level AUROC and AUCPRO are 96.4%, 95.7%, and 90.6%, respectively; when the loss function is used between shallow features and deep features , the overall loss function is + When used alone When , the image-level and pixel-level AUROC and AUCPRO increased by 1.1%, 1.2%, and 2.1%, respectively, indicating that the use of loss function between shallow features and deep features helps to improve the defect detection performance; when the loss function is used between shallow features and deep features , hyperparameters is 0.3, and the overall loss function is When , the loss function is + Compared with the previous study, the image-level AUROC and AUCPRO increased by 0.3% and 0.8% respectively, and the pixel-level AUROC decreased by 0.7%, indicating that the use of Compare It can effectively improve the defect detection and positioning effect; when the hyperparameter is 0.7, and the loss function is When the hyperparameter is 0.3, the pixel-level AUROC and AUCPRO increase by 1.2% and 1.9% respectively, but the image-level AUROC decreases by 0.5%, indicating that the increase The proportion of can greatly improve the positioning performance of defect detection; hyperparameters When it is 0.5, When the value is 0.7, the image-level AUROC increases by 1.1%, but the pixel-level AUROC and AUCPRO decrease by 0.2% and 0.3% respectively. The pixel-level index decreases little, but the image-level AUROC increases significantly, indicating that the detection effect is significantly improved when the positioning effect is not seriously lost. Considering the comprehensive performance of defect detection, the hyperparameter is selected in this embodiment. is 0.5.
[0070] Embodiment 2
[0071] See also Figures 1 to 8 This embodiment provides an industrial defect detection method, which is applied to the liquid filling industry and can detect foreign matter in the filling barrel. Of course, it can be applied to other defect detection in the liquid filling industry and other industrial industries in the same way. The defect detection method includes:
[0072] S21, acquiring an image of the industrial product to be inspected.
[0073] In this embodiment, the image inside the filling barrel can be acquired in real time by an industrial camera arranged in the filling machine.
[0074] S22, inputting the image of the industrial product to be inspected into the defect detection model constructed by the above-mentioned defect detection model construction method to perform feature extraction, and outputting the defect detection result.
[0075] In this embodiment, the image inside the filling barrel obtained from the industrial camera is input into the defect detection model, and defect detection is performed on the image inside the filling barrel. The image inside the filling barrel with foreign objects is regarded as a normal image, and the foreign objects in the filling barrel are regarded as defects, so as to determine whether there are foreign objects in the filling barrel. At the same time, the defect detection result can also be returned to the host computer of the filling machine, and the detection result can be displayed on the software. The user can confirm the location of the foreign objects in the barrel through the software.
[0076] The defect detection model includes: a teacher network, a student network, an artificial synthetic anomaly module SAM_AB and a multi-scale feature fusion module MSFFM; the artificial synthetic anomaly module SAM_AB is used to generate pseudo-defect sample images from normal sample images, simulate unknown types of defects, and introduce pseudo-defect features in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has stronger defect detection capabilities; the multi-scale feature fusion module MSFFM is used to fuse the shallow features and deep features output by the teacher network and the student network respectively, so that the student network can better take into account detail information and semantic information, thereby enhancing the detection capability of defects in industrial products with complex structures.
[0077] First, the obtained image inside the filling barrel is used to generate a pseudo defect sample image through the artificial synthesis abnormality module SAM_AB. The specific process is as follows: the original image of the image inside the filling barrel is obtained The foreground position is obtained by adaptive binarization , Gaussian noise The initial artificial defect position is obtained by thresholding ; Then set the foreground position and preliminary artificial defect location Multiply to get the position of the artificial defect ; Then the artificial defect position With texture image Multiply together to get a more realistic composite defect Finally, the original image outside the defect position is added to the synthetic defect to obtain the artificially synthesized pseudo-defect sample image. .
[0078] Then, the normal image of the image inside the filling barrel is input into the teacher network pre-trained on the ImageNet dataset, and the pseudo-defect sample image is input into the student network. The teacher network extracts the normal image features of the image inside the filling barrel, guides the student network to remove the defect features of the pseudo-defect sample image, and restores the normal sample image of the image inside the filling barrel.
[0079] Then the shallow features output by the teacher network are and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. .
[0080] Finally, the fusion features of the teacher network are calculated and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map Visualized as a heat map, different colors are used to distinguish between areas with higher anomaly scores and normal areas, thereby achieving defect detection and segmentation of images inside the filling barrel, and then determining whether there are foreign objects in the filling barrel.
[0081] Embodiment 3
[0082] This embodiment also provides an industrial defect detection device, the device comprising:
[0083] An image acquisition module, used to acquire images of industrial products to be inspected;
[0084] The defect detection module is used to perform defect detection on the image of the industrial product to be detected acquired by the image acquisition module using the defect detection model constructed by the above-mentioned defect detection model construction method, and output the defect detection result.
[0085] The defect detection model includes: a teacher network, a student network, an artificial synthetic anomaly module SAM_AB and a multi-scale feature fusion module MSFFM; the artificial synthetic anomaly module SAM_AB is used to generate pseudo-defect sample images from normal sample images, simulate unknown types of defects, and introduce pseudo-defect features in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has stronger defect detection capabilities; the multi-scale feature fusion module MSFFM is used to fuse the shallow features and deep features output by the teacher network and the student network respectively, so that the student network can better take into account detail information and semantic information, thereby enhancing the detection capability of defects in industrial products with complex structures.
[0086] First, the acquired industrial product image to be detected is used to generate a pseudo defect sample image through the artificial synthetic abnormality module SAM_AB. The specific process is as follows: the original image of the industrial product image to be detected is acquired The foreground position is obtained by adaptive binarization , Gaussian noise The initial artificial defect position is obtained by thresholding ; Then set the foreground position and preliminary artificial defect location Multiply to get the position of the artificial defect ; Then the artificial defect position With texture image Multiply together to get a more realistic composite defect Finally, the original image outside the defect position is added to the synthetic defect to obtain the artificially synthesized pseudo-defect sample image. .
[0087] The acquired normal image of the industrial product image to be inspected is then input into the teacher network pre-trained on the ImageNet dataset, and the pseudo-defect sample image is input into the student network. The teacher network extracts the normal image features of the industrial product image to be inspected, guides the student network to remove the defect features of the pseudo-defect sample image, and restores the normal sample image of the industrial product image to be inspected.
[0088] Then the shallow features output by the teacher network are and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. .
[0089] Finally, the fusion features of the teacher network are calculated and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map Visualized as a heat map, different colors are used to distinguish areas with higher anomaly scores from normal areas, thus achieving defect detection and segmentation of industrial product images to be inspected.
[0090] Embodiment 4
[0091] This embodiment also provides a device, including a memory and a processor.
[0092] The memory stores a computer program.
[0093] The processor can call the computer program in the memory to execute the above-mentioned defect detection model construction method, which includes: constructing a defect detection model, the defect detection model includes: a teacher network, a student network, an artificial synthesis abnormality module SAM_AB and a multi-scale feature fusion module MSFFM; the artificial synthesis abnormality module SAM_AB is used to generate a pseudo-defect sample image from a normal sample image, simulate an unknown type of defect, and introduce pseudo-defect features in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has a stronger defect detection ability; the multi-scale feature fusion module MSFFM is used to fuse the shallow features and deep features output by the teacher network and the student network respectively; training the defect detection model, including: generating a pseudo-defect sample image from a normal sample image through the artificial synthesis abnormality module SAM_AB; inputting the normal sample image into the teacher network pre-trained on the ImageNet data set, and the pseudo-defect sample image into the student network, the teacher network extracting the normal sample image features, guiding the student network to remove the defect features of the pseudo-defect sample image, and restoring the normal sample image; the shallow features output by the teacher network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. ; Calculate the fusion features of the teacher network and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map The image is visualized as a heat map, and different colors are used to distinguish between areas with higher abnormality scores and normal areas, thereby realizing defect detection and segmentation of the image and completing the training of the defect detection model.
[0094] The processor can also call the computer program in the memory to execute the above-mentioned image classification method, which includes: acquiring an image of the industrial product to be inspected; inputting the image of the industrial product to be inspected into the defect detection model constructed by the above-mentioned defect detection model construction method to perform feature extraction, and outputting the defect detection result.
[0095] In addition, the computer program in the memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0096] When the above software is applied to a liquid filling machine, the software is first copied to the specified location of the image industrial computer of the filling machine. The software is configured to communicate with the PLC of the filling machine, and the image inside the filling barrel is obtained in real time through the Hikvision web camera. The software receives the detection signal sent by the PLC, starts to detect foreign objects at the bottom of the barrel, and then returns the detected results to the PLC.
[0097] The software can also be set according to different barrel types. For example, when detecting foreign objects at the bottom of 20L plastic barrels and 25L plastic barrels, the input image color can be set to a color image; when detecting foreign objects at the bottom of 200L paint barrels, the input image color can be set to black and white.
[0098] Embodiment 5
[0099] This embodiment also provides a computer-readable storage medium on which computer program instructions are stored.
[0100] When the computer program instructions are executed by the processor, the above-mentioned defect detection model construction method is implemented, and the method includes: constructing a defect detection model, the defect detection model includes: a teacher network, a student network, an artificial synthesis abnormality module SAM_AB and a multi-scale feature fusion module MSFFM; the artificial synthesis abnormality module SAM_AB is used to generate a pseudo-defect sample image from a normal sample image, simulate an unknown type of defect, and introduce pseudo-defect features in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has a stronger defect detection ability; the multi-scale feature fusion module MSFFM is used to fuse the shallow features and deep features output by the teacher network and the student network respectively; training the defect detection model, including: generating a pseudo-defect sample image from a normal sample image through the artificial synthesis abnormality module SAM_AB; inputting the normal sample image into the teacher network pre-trained on the ImageNet data set, and the pseudo-defect sample image into the student network, the teacher network extracting the normal sample image features, guiding the student network to remove the defect features of the pseudo-defect sample image, and restoring the normal sample image; the shallow features output by the teacher network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. ; Calculate the fusion features of the teacher network and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map The image is visualized as a heat map, and different colors are used to distinguish between areas with higher abnormality scores and normal areas, thereby realizing defect detection and segmentation of the image and completing the training of the defect detection model.
[0101] When the computer program instructions are executed by the processor, the above-mentioned image classification method can also be implemented, which includes: acquiring an image of the industrial product to be inspected; inputting the image of the industrial product to be inspected into the defect detection model constructed by the above-mentioned defect detection model construction method to perform feature extraction, and outputting the defect detection result.
[0102] The above is an industrial defect detection model construction method, detection method, device, equipment and storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.
Claims
1. A method for constructing an industrial defect detection model, characterized in that: The model building method comprises: S11, constructing a defect detection model, the defect detection model includes: a teacher network, a student network, an artificial synthesis anomaly module SAM_AB and a multi-scale feature fusion module MSFFM; the artificial synthesis anomaly module SAM_AB is used to generate a pseudo-defect sample image from a normal sample image, simulate an unknown type of defect, introduce pseudo-defect features in the knowledge distillation process of the teacher network and the student network, so that the defect detection model has a stronger defect detection capability; the multi-scale feature fusion module MSFFM is used to fuse the shallow features and deep features output by the teacher network and the student network respectively; S12, training the defect detection model, including: S121, generating a pseudo-defect sample image from a normal sample image through the artificially synthesized abnormality module SAM_AB; S122, inputting the normal sample image into the teacher network pre-trained on the ImageNet dataset, and inputting the pseudo-defect sample image into the student network, wherein the teacher network extracts features of the normal sample image, guides the student network to remove the defect features of the pseudo-defect sample image, and restores the normal sample image; S123, converting the shallow features output by the teacher network into the pseudo-defect sample image; and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the teacher network. ; Then the shallow features output by the student network and deep features The concatenation is performed and input into the multi-scale feature fusion module MSFFM to obtain the fusion features of the student network. ; S124, calculating the fusion features of the teacher network and the fusion characteristics of the student network The square difference of , the area with high anomaly score is considered to be the defect area, and finally the anomaly score map The image is visualized as a heat map, and different colors are used to distinguish between areas with higher abnormality scores and normal areas, thereby realizing defect detection and segmentation of the image and completing the training of the defect detection model.
2. The industrial defect detection model construction method according to claim 1, characterized in that: In S121, the process of generating a pseudo-defect sample image from a normal sample image through the artificial synthesis abnormality module SAM_AB is as follows: the original image of the normal sample image is adaptively binarized to obtain a foreground position, and Gaussian noise is thresholded to obtain a preliminary artificial defect position; then the foreground position is multiplied by the preliminary artificial defect position to obtain the artificial defect position; Then the artificial defect position is multiplied by the texture image to obtain a more realistic synthetic defect. Finally, the original image except the defect position is added to the synthetic defect to obtain an artificially synthesized pseudo-defect sample image.
3. The industrial defect detection model construction method according to claim 1, characterized in that: In S123, the multi-scale feature fusion module MSFFM selects the shallow features and deep features output by the teacher network and the student network after two average pooling layers as input features, so that the student network can better take into account detail information and semantic information. Specifically: the deep features of the student network After 1×1 convolution and ReLU activation function processing, the trainable weights are obtained , the shallow features of the student network After 1×1 convolution and bilinear interpolation downsampling, the deep features of the student network are obtained Intermediate features with the same size are processed by 1×1 convolution and ReLU activation function to obtain trainable weights , With the deep features of the student network , The features obtained by multiplying the intermediate features are added together to obtain the fusion features of the student network. ; The teacher network uses the same weights as the student network and , the shallow features of the teacher network After 1×1 convolution and bilinear interpolation downsampling, the deep features of the teacher network are obtained Intermediate features with the same size, With the deep features of the teacher network , The features obtained by multiplying the intermediate features are added together to obtain the fusion features of the teacher network. .
4. The method for constructing an industrial defect detection model according to claim 3, characterized in that: In S124, the abnormal score map is used To represent the model's segmentation results for pixel-level defects, the abnormal score map The calculation formula is: ,in, The fusion features of the teacher network And the fusion characteristics of the student network The number of channels, and are the fusion features of the student network No. The features of the layer channels and the fusion features of the teacher network No. Characteristics of layer channels.
5. The industrial defect detection model construction method according to claim 4, characterized in that: SSIM loss is introduced on shallow features and deep features respectively, and the shallow features of the teacher network are calculated using the SSIM loss function. and the shallow features of the student network , the deep features of the teacher network and the deep features of the student network The loss between.
6. The method for constructing an industrial defect detection model according to claim 5, characterized in that: Use the squared error as the fusion feature of the teacher network And the fusion characteristics of the student network The loss between them, SSIM loss and weighted average of squared difference are used as the overall loss in the distillation stage. , the calculation formula is: ,in, Anomaly score map of loss, is the SSIM loss, Used to balance the impact of SSIM loss on the overall loss function; Abnormal score graph Loss The calculation formula is: ,in, , The abnormal score graphs are The height and width of the SSIM loss are calculated as: ,in, Represents the feature map of the teacher network, The feature graph representing the student network, It is a shallow feature. When is a deep feature, , They are , The average brightness, , They are , The standard deviation of for and The covariance of and is a small constant that ensures that the denominator is non-zero.
7. An industrial defect detection method, characterized in that: The defect detection method comprises: S21, acquiring an image of the industrial product to be inspected; S22, inputting the image of the industrial product to be inspected into the defect detection model constructed by the defect detection model construction method according to any one of claims 1 to 6 to perform feature extraction, and outputting the defect detection result.
8. An industrial defect detection device, characterized in that: The device comprises: An image acquisition module, used to acquire images of industrial products to be inspected; A defect detection module is used to perform defect detection on the image of the industrial product to be detected acquired by the image acquisition module using the defect detection model constructed by the defect detection model construction method described in any one of claims 1 to 6, and output the defect detection result.
9. A device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the defect detection model construction method described in any one of claims 1 to 6 and / or the defect detection method described in claim 7.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the defect detection model construction method described in any one of claims 1 to 6 and / or the defect detection method described in claim 7 are implemented.
Citation Information
Cited By
Power transmission line defect detection method and system based on knowledge distillation
CN119941714A
Industrial defect classification detection method and device, medium and electronic equipment
CN120374628A
Product defect detection method based on adaptive complementary channel and full-span enhancement
CN120725981A
Unsupervised industrial defect detection network construction method
CN120953222A
