Industrial visual defect detection method in combination with Mask information under complex background

Abstract

The invention relates to an industrial visual defect detection method in combination with Mask information under a complex background. The method comprises the following steps: S1, introducing an attention mechanism module guided by Mask; s2, training a classification network of an attention mechanism fused with Mask guidance; s3, the background of the defect sample is erased; and S4, defect prediction: inputting a to-be-detected image into the trained model to perform reasoning on a defect image. According to the method, a mask-guided attention mechanism module is introduced into a classification network, so that the representation of defect information in features is enhanced; a defect sample background erasure data enhancement method based on mask is provided, in the model training process, by combining mask labeling, the defect area is avoided, the image background is erased randomly, and the influence of the complex background on the model is weakened; after the method is used, the defect detection accuracy in an industrial defect detection task under a complex background can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image vision detection, and in particular to an industrial vision defect detection method under a complex background combined with Mask information. Background Art

[0002] Currently, there is no public method in the field of industrial visual defect detection that can improve the accuracy of defect detection in complex backgrounds. In terms of natural images, there are similar image erasure data enhancement methods to alleviate the overfitting problem of the model; in addition, there are related studies on the use of mask-guided attention mechanisms in tasks related to pedestrian detection.

[0003] For image erasing data enhancement methods in natural images: including Cutout, Random Erasing, Hide-and-Seek and other methods used in natural images, they randomly select rectangular areas on the image through different strategies and cover the selected areas with 0 or random values. These methods all have the characteristic of randomness in erasing areas.

[0004] However, image erasure data enhancement methods in natural images may cause loss of defect area information by randomly selecting masked areas, due to the characteristics of industrial defect images in which the recognition target accounts for a small proportion of the image and the semantic information of the recognition target is weak, which has uncontrollable impact on model training.

[0005] In the task of pedestrian detection, there are related studies on the use of mask-guided attention mechanism in pedestrian re-identification and occluded pedestrian detection.

[0006] However, there is no application of the mask-guided attention mechanism in the field of industrial defect detection. Summary of the invention

[0007] The technical problem to be solved by the present invention is: to provide an industrial visual defect detection method under a complex background combined with Mask information, so as to solve the problem that in the industrial visual defect detection task, the surface of the material with physical texture will form a complex background image when imaging. These complex backgrounds will have an adverse effect on the training of the model, resulting in the missed detection of some targets and the over-detection of normal samples.

[0008] The technical solution adopted by the present invention to solve the technical problem is: a method for detecting industrial visual defects under a complex background combined with Mask information, comprising the following steps:

[0009] S1. Introduce the Mask-guided attention mechanism module;

[0010] S2, training the classification network integrating the Mask-guided attention mechanism;

[0011] S3, background erasing of defective samples;

[0012] S4, defect prediction, input the image to be detected into the trained model to infer the defect image.

[0013] Furthermore, in step S1 described in the present invention, the Mask-guided attention mechanism module is introduced after the feature map enters the last stage of the backbone network and before entering the classifier.

[0014] Furthermore, in step S1 of the present invention, the introduction method is:

[0015] S11, the input image passes through the feature extraction network and the feature map output in the final stage is F1; the size of F1 is B×C×H×W;

[0016] S12, feature map F1 passes through a convolution module with an input channel of C and an output channel of C to obtain a feature map F2 with a scale of B×C×H×W;

[0017] S13, input the feature map F2 into a convolution layer with an input channel of C and an output channel of 1, and obtain a weight matrix ω of a scale of B×1×H×W;

[0018] S14. Pass the weight matrix ω through the sigmoid function to obtain ω′, and then multiply it with the feature map F1 to obtain the feature map F3 that enters the classifier.

[0019] Furthermore, the feature map F3 described in the present invention is passed through a classifier to obtain the category probability probs; the weight matrix ω is used to perform loss calculation with the scaled mask annotated image during the network training process.

[0020] Furthermore, in step S2 described in the present invention, the attention mechanism is guided by a mask-annotated image; the binary image of the mask-annotated image after being reduced is M; the loss function of the classification part during the training process adopts a cross-entropy loss function; the input of the loss function is the category probability probs output by the classifier and the category truth value gt_class; the mask guidance part adopts a binary cross-entropy loss function, and the input of the loss function is the reduced mask-annotated image M and the weight matrix ω generated by the attention mechanism module; the final loss function adopts a combination of classification loss and mask loss.

[0021] Furthermore, in step S2 of the present invention, only geometric transformation and defect sample background erasing method are used for data enhancement in the training phase; the geometric transformation includes: random flipping up and down, random flipping left and right, random offset and random rotation.

[0022] Further, in step S3 described in the present invention, the defect sample background erasing method is as follows: let the binary image marked with Mask be A, first set the initial parameters of the algorithm: the maximum number of rectangles N, the number of loops n=0; initialize the 0 matrix R of the image input size, randomly generate a rectangle with a height of h, a width of w, and a rotation angle of r, and draw it on the matrix R; calculate the intersection of the matrix A and the matrix R; if it is empty, retain the rectangle drawn this time; if it is not empty, delete the rectangle drawn this time; execute N times, and the generated matrix R is the final masking area; assign 0 pixels to the position corresponding to the matrix R in the image.

[0023] The beneficial effect of the present invention is that it solves the defects existing in the background technology. The present invention strengthens the representation of defect information in features by introducing a mask-guided attention mechanism module (Mask Guide Attention Head, MAGHead) in the classification network; and proposes a mask-based defect sample background erasing data enhancement method, which combines mask annotation in the model training process to randomly erase the image background avoiding the defect area, thereby weakening the influence of the complex background on the model; after using the present invention, the defect detection accuracy in complex backgrounds in industrial defect detection tasks can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the network training process of the present invention;

[0025] Figure 2 It is a schematic diagram of the structure of the mask-guided attention mechanism module of the present invention;

[0026] Figure 3 It is a schematic diagram of the defect sample background erasing process of the present invention. DETAILED DESCRIPTION

[0027] The present invention will now be described in further detail in conjunction with the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0028] like Figure 1-Figure 3 The figure shows an industrial visual defect detection method under complex background combined with Mask information. By combining mask information, the influence of image background in the model training process is weakened, while emphasizing the representation of defects in features, thereby improving the defect detection accuracy of industrial visual images under complex background.

[0029] The specific steps are as follows:

[0030] 1. Introduce the Mask-guided Attention Mechanism Module

[0031] After the input image passes through the feature extraction network, the feature map F1 output in the final stage is obtained. Before entering the classifier, the Mask Guide Attention Head (MAGHead) is introduced. Assume that the size of F1 is B×C×H×W. First, it passes through a convolution module with an input channel of C and an output channel of C to obtain a feature map F2 with a scale of B×C×H×W. Then the feature map is input into a convolution layer with an input channel of C and an output channel of 1 to obtain a weight matrix ω with a scale of B×1×H×W. The weight matrix is ​​passed through the sigmoid function to obtain ω′, which is then multiplied by F1 to obtain the feature map F3 that enters the classifier. F3 passes through the classifier to obtain the category probability probs. The weight matrix ω will be used for loss calculation with the scaled mask annotated image during network training.

[0032] 2. Training a classification network with Mask-guided attention mechanism

[0033] Use the masked image to guide the attention mechanism. Assume that the true value of the category is gt_class, and the binary image of the masked image after being reduced is M. During the training process, the classification part loss function uses the cross entropy loss function, and the input of the loss function is the category probability probs output by the classifier and the true value of the category gt_class:

[0034] CELoss(gt_class,probs)=-∑gt_class i log(probs i ) (1)

[0035] The mask guidance part uses a binary cross entropy loss function, and the input of the loss function is the reduced mask annotation image M and the weight matrix ω generated by MGAHead:

[0036] BCELoss(M,ω)=-[Mlogω +(1-M)log(1-ω)] (2)

[0037] The final loss function uses a combination of classification loss and mask loss:

[0038] loss(gt_class,probs,M,ω)=

[0039] CELoss(gt_class,probs)+0.5*BCELoss(M,ω)(3)

[0040] The training optimizer uses Adam, and the initial learning rate is set to 1e-4. In order to avoid the negative impact of some image enhancement methods that modify pixel values ​​on the quality of defective images, only geometric transformation and the defective sample background erasing method designed by the present invention are used for data enhancement during the training stage. The geometric transformation includes: random flipping up and down, random flipping left and right, random offset, and random rotation.

[0041] 3. Defective sample background erasure

[0042] Combined with the pixel-level annotation of the training image, the mask area is generated. Assume that the binary image with Mask annotation is A. First, set the initial parameters of the algorithm: the maximum number of rectangles N, the number of loops n = 0. First, initialize the 0 matrix R of the image input size, randomly generate a rectangle with a height of h, a width of w, and a rotation angle of r, and draw it on the matrix R. Calculate the intersection of matrix A and matrix R. If it is empty, keep the rectangle drawn this time; if it is not empty, delete the rectangle drawn this time. This process is performed N times, and the generated matrix R is the final mask area. Assign the position in the image corresponding to the matrix R to 0 pixels.

[0043] 4. Defect Prediction

[0044] The image to be detected is input into the trained model to perform inference on the defect image.

[0045] This solution includes a mask-guided attention mechanism module and a mask-based defect sample background erasure method. The mask-guided attention mechanism module can assist the network to pay attention to the defect area of ​​the image during training. The mask-based defect sample background erasure method can weaken the role of the background in network training.

[0046] The above description only describes the specific implementation mode of the present invention. Various examples do not limit the essential content of the present invention. After reading the description, ordinary technicians in the relevant technical field can modify or deform the specific implementation modes described above without departing from the essence and scope of the invention.

Claims

1. A method for detecting industrial visual defects under complex backgrounds combined with Mask information, characterized in that: The following steps are included: S1. Introduce the Mask-guided attention mechanism module; S2, training the classification network integrating the Mask-guided attention mechanism; S3, background erasing of defective samples; S4, defect prediction, input the image to be detected into the trained model to infer the defect image.

2. The method for detecting industrial visual defects under complex backgrounds combined with Mask information according to claim 1, characterized in that: In the step S1, the Mask-guided attention mechanism module is introduced after the feature map enters the last stage of the backbone network and before entering the classifier.

3. The method for detecting industrial visual defects under complex backgrounds combined with Mask information as claimed in claim 2, characterized in that: In the step S1, the introduction method is: S11, the input image passes through the feature extraction network and the feature map output in the final stage is F1; the size of F1 is B×C×H×W; S12, feature map F1 passes through a convolution module with an input channel of C and an output channel of C to obtain a feature map F2 with a scale of B×C×H×W; S13, input the feature map F2 into a convolution layer with an input channel of C and an output channel of 1, and obtain a weight matrix ω of a scale of B×1×H×W; S14. Pass the weight matrix ω through the sigmoid function to obtain ω′, and then multiply it with the feature map F1 to obtain the feature map F3 that enters the classifier.

4. The method for detecting industrial visual defects under complex backgrounds combined with Mask information as claimed in claim 3, characterized in that: The feature graph F3 is passed through a classifier to obtain the category probability probs; The weight matrix ω is used to calculate the loss with the scaled mask annotated image during network training.

5. The method for detecting industrial visual defects under complex backgrounds combined with Mask information as claimed in claim 4, characterized in that: In the step S2, the attention mechanism is guided by the mask annotated image; the binary image of the mask annotated image after being reduced is M; the loss function of the classification part during the training process adopts the cross entropy loss function; the input of the loss function is the category probability probs output by the classifier and the category true value gt_class; the mask guidance part adopts the binary cross entropy loss function, and the input of the loss function is the reduced mask annotated image M and the weight matrix ω generated by the attention mechanism module; the final loss function adopts a combination of classification loss and mask loss.

6. The method for detecting industrial visual defects under complex backgrounds combined with Mask information according to claim 1, characterized in that: In the step S2, only geometric transformation and defect sample background erasing method are used for data enhancement in the training phase; the geometric transformation includes: random up and down flipping, random left and right flipping, random offset and random rotation.

7. The method for detecting industrial visual defects under complex backgrounds combined with Mask information according to claim 1, characterized in that: In the step S3, the defect sample background erasing method is as follows: let the binary image marked by Mask be A, first set the initial parameters of the algorithm: the maximum number of rectangles N, the number of loops n=0; initialize the 0 matrix R of the image input size, randomly generate a rectangle with a height of h, a width of w, and a rotation angle of r, and draw it on the matrix R; calculate the intersection of the matrix A and the matrix R; if it is empty, keep the rectangle drawn this time; if it is not empty, delete the rectangle drawn this time; execute N times, and the generated matrix R is the final masking area; assign 0 pixels to the position corresponding to the matrix R in the image.

Images