An industrial defect detection method and apparatus

Abstract

The application discloses a kind of industry defect detection methods, the method comprises the following steps: selecting the CNN classification network that different resolution images can be input;Using training set to train CNN classification network;Input preprocessed image to be measured, utilize CNN classification network to identify the defect in image to be measured, and classify defect;Wherein, CNN classification network has global average pooling layer.The application also discloses a kind of industry defect detection device.The application utilizes CNN classification network with global average pooling layer, in training and testing process, the width and height of the image input model can change, increase the diversity of training sample size, and increase the classification accuracy of model to various size images.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection technology, and more specifically, to an industrial defect detection method and apparatus. Background Technology

[0002] Currently, Convolutional Neural Network (CNN) classification models are widely used in the industrial quality inspection field. However, current CNN classification models use fixed input image resolutions and equal width and height during training and testing, for example, 224*224 resolution. This method requires scaling the original image to 224*224 resolution before feeding it into the CNN classification model, regardless of its original resolution. This can actually reduce the classification accuracy of some images. For instance, some images with width and height less than 30 pixels will become severely blurry when scaled to 224*224, reducing classification accuracy. Similarly, images with significant width and height differences (e.g., 25 pixels wide and 300 pixels high) will appear significantly wider when scaled to 224*224, causing noticeable distortion and further reducing classification accuracy. Summary of the Invention

[0003] To address at least one deficiency or improvement need in the existing technology, the present invention provides an industrial defect detection method and apparatus. It utilizes a CNN classification network with a global average pooling layer, where the width and height of the input model image can vary during training and testing, increasing the diversity of training sample sizes and improving the model's classification accuracy for images of various sizes.

[0004] To achieve the above objectives, according to a first aspect of the present invention, an industrial defect detection method is provided, the method comprising:

[0005] Choose a CNN classification network that can accept images of different resolutions;

[0006] The CNN classification network is trained using the training set;

[0007] Input a preprocessed image to be tested, use the CNN classification network to identify defects in the image to be tested, and classify the defects;

[0008] The CNN classification network has a global average pooling layer.

[0009] Furthermore, the aforementioned industrial defect detection methods also include:

[0010] The CNN classification network is ResNet, MobileNet v1, MobileNet v2, or MobileNet v3.

[0011] Furthermore, the aforementioned industrial defect detection methods also include:

[0012] The preprocessing includes:

[0013] A detection model is used to detect the location of defects in the original image to be tested, and the image to be tested is cropped based on the location of the defects.

[0014] Furthermore, the aforementioned industrial defect detection methods also include:

[0015] Training the CNN classification network using the training set specifically includes:

[0016] The training batch size is set to 1. After training with N samples, the accumulated gradient is divided by N, and then the parameters of the CNN classification network are updated.

[0017] Furthermore, the aforementioned industrial defect detection methods also include:

[0018] Save the preprocessed test images and construct or update the training set through data equalization and / or data augmentation.

[0019] According to a second aspect of the present invention, an industrial defect detection device is also provided, comprising:

[0020] A storage module configured to store at least one CNN classification network and a training set; wherein the CNN classification network has a global average pooling layer;

[0021] A camera module, configured to photograph the object under test, to acquire a first image of the object under test with industrial defects;

[0022] An image input module is configured to receive the first image to be tested, preprocess the image to be tested, and output a second image to be tested.

[0023] A training module is configured to call a CNN classification network from the storage module and train the CNN classification network using the training set;

[0024] The computing module is configured to receive the second image to be tested, identify defects in the second image to be tested using a trained CNN classification network, and classify the defects.

[0025] Furthermore, the aforementioned industrial defect detection device also includes:

[0026] The CNN classification network is ResNet, MobileNet v1, MobileNet v2, or MobileNet v3.

[0027] Furthermore, the aforementioned industrial defect detection device also includes:

[0028] The image input module includes a detection model for detecting the location of defects in the first image to be tested, cropping the first image to be tested based on the defect location as a second image to be tested, and outputting the second image to be tested.

[0029] Furthermore, the aforementioned industrial defect detection device also includes:

[0030] The training module trains the CNN classification network using samples, specifically including:

[0031] In the training module, the batch size is set to 1. After training with N samples, the accumulated gradient is divided by N, and then the parameters of the CNN classification network are updated.

[0032] Furthermore, the aforementioned industrial defect detection device also includes:

[0033] The data processing module is used to perform data equalization and / or data enhancement on the second image to be tested to obtain the third image to be tested;

[0034] The storage module stores the third image to be tested and is used to build or update the training set.

[0035] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0036] The industrial defect detection method provided by this invention allows for variable resolution and non-uniform width and height of the input model images during training and testing. When training a CNN classification network, the resolution of the input model images can be scaled, increasing the diversity of training sample resolutions and improving the model's accuracy in classifying objects of different sizes. When testing the CNN classification network, the resolution of the input model images can be varied as needed, providing greater flexibility and improving the classification accuracy of images during testing. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic flowchart of an industrial defect detection method provided in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of an industrial defect detection device provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0041] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0042] like Figure 1 As shown, as a first embodiment of the present invention, an industrial defect detection method is provided, comprising:

[0043] Choose a CNN classification network that can accept images of different resolutions;

[0044] Train a CNN classification network using the training set;

[0045] Input the image to be tested, use a CNN classification network to identify defects in the image, and classify the defects;

[0046] Among them, the CNN classification network has a global average pooling layer.

[0047] Traditional classification models require a fixed input size; for example, the resolution of the input image needs to be adjusted to 224*224. If the defect area is very small, for example, no more than 30*30, or if the width and height of the detected defect area have a very large difference, such as an aspect ratio of 15, a relatively large area needs to be expanded to crop a square region in order to feed the extracted defect into the classification model. This introduces a relatively large background area. The background area becomes noise in the classification process, reducing the accuracy of classification.

[0048] Now, by training a CNN classification model using methods with variable width and height and non-fixed aspect ratio, for defects detected by the detection model that have very small areas or very large differences between width and height, the corresponding region can be extracted by simply expanding it outwards. Then, it can be scaled up as needed without causing obvious blurring or distortion of the image before being fed into the CNN classification model for classification. This method can improve classification accuracy.

[0049] In this invention, a CNN classification network with a global average pooling layer is selected as the classification network. Taking ResNet and MobileNet (including MobileNet v1, MobileNet v2, or MobileNet v3) as examples, the final part of the CNN classification model includes a global average pooling layer. This way, regardless of the resolution of the input image to the CNN, the feature map resolution becomes 1*1 after passing through this layer. Since the number of channels of the feature map output by the global average pooling layer is a fixed value designed by the model, the resolution and number of channels of the feature map output by the global average pooling layer are fixed. Thus, the total number of parameters of the subsequent fully connected layers will not change. Therefore, the resolution of the image input to the CNN classification model can be flexible during training and testing.

[0050] Taking MobileNet v1 as an example, since there are five convolutions with a stride of 2 before the global average pooling layer, and the remaining convolutions have a stride of 1, the feature map is downsampled by a factor of 2 five times, totaling a factor of 32. Therefore, the width and height of the input image to MobileNet v1 only need to be integer multiples of 32, and the width and height do not need to be equal. For example, the input image resolution can be 128 pixels wide and 160 pixels high. During training, the width and height of the input image can be continuously changed. During testing, the width and height of the input image can also be changed as needed.

[0051] Furthermore, since the aforementioned CNN classification network is used, there is no need to fix the width and height of the input image. The input image can then be preprocessed. First, a detection model is used to identify the specific location of defects in the image. Unlike traditional CNN classification networks, which stretch or expand the defect image, the image cropped to primarily represent the defect can be directly input into the CNN classification network for training and testing. In other words, the aspect ratio of the defect image can be changed as little as possible, avoiding the introduction of blur or distortion.

[0052] Furthermore, since there are no widely available public training sets in industrial inspection, it is necessary to collect relevant defect images as training sets during the inspection process. Specifically, for defect images, data processing techniques such as data balancing and data augmentation are also required.

[0053] Furthermore, during training, input images do not need to be scaled to the same size or to have equal width and height. Images can be scaled, but the width and height must meet certain conditions. For example, for MobileNet v1, the width and height of the input images must be integer multiples of 32. Because current deep learning frameworks require images within the same batch to have equal width and height, if some images are not suitable for scaling to the image size set for a particular batch, the training batchsize can be set to 1. Each image can be scaled appropriately, and after training with N samples, the accumulated gradient is divided by N, and the parameters of the CNN classification network are updated again. This involves updating the parameters once in the forward and backward directions multiple times. This achieves both the goal of reasonably scaling all images and obtaining the desired equivalent batch size for training.

[0054] like Figure 2 As shown, as a second embodiment of the present invention, an industrial defect detection device is also provided, comprising:

[0055] A storage module configured to store at least one CNN classification network and a training set; wherein the CNN classification network has a global average pooling layer;

[0056] A camera module, configured to photograph the object under test, to acquire a first image of the object under test with industrial defects;

[0057] An image input module is configured to receive the first image to be tested, preprocess the image to be tested, and output a second image to be tested.

[0058] A training module is configured to call a CNN classification network from the storage module and train the CNN classification network using the training set;

[0059] The computing module is configured to receive the second image to be tested, identify defects in the second image to be tested using a trained CNN classification network, and classify the defects.

[0060] Furthermore, the aforementioned industrial defect detection device also includes:

[0061] The CNN classification network is ResNet, MobileNet v1, MobileNet v2, or MobileNet v3.

[0062] Furthermore, the aforementioned industrial defect detection device also includes:

[0063] The image input module includes a detection model for detecting the location of defects in the first image to be tested, cropping the first image to be tested based on the defect location as a second image to be tested, and outputting the second image to be tested.

[0064] Furthermore, the aforementioned industrial defect detection device also includes:

[0065] The training module trains the CNN classification network using samples, specifically including:

[0066] In the training module, the batch size is set to 1. After training with N samples, the accumulated gradient is divided by N, and then the parameters of the CNN classification network are updated.

[0067] Furthermore, the aforementioned industrial defect detection device also includes:

[0068] The data processing module is used to perform data equalization and / or data enhancement on the second image to be tested to obtain the third image to be tested;

[0069] The storage module stores the third image to be tested and is used to build or update the training set.

[0070] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0071] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An industrial defect detection method characterized by, The method comprises the following steps: selecting a CNN classification network capable of inputting images with different resolutions; training the CNN classification network using a training set; inputting a preprocessed test image, identifying defects in the test image using the CNN classification network, and classifying the defects; wherein the CNN classification network has a global average pooling layer; the feature map resolution and the number of channels output by the global average pooling layer are fixed; training the CNN classification network using a training set, specifically comprising: setting the batchsize of the training to 1, dividing the accumulated gradient by N after training N samples, and then performing parameter update of the CNN classification network.

2. The industrial defect detection method of claim 1, wherein: the CNN classification network is Resnet, Mobilenet v1, Mobilenet v2, or Mobilenet v3.

3. The industrial defect detection method of claim 1, wherein: the preprocessing comprises: detecting the positions of defects in the original test image using a detection model, and cutting the test image according to the positions of the defects.

4. The industrial defect detection method of claim 1, wherein: saving the preprocessed test image, and constructing or updating the training set through data balancing and / or data enhancement.

5. An industrial defect detection apparatus for implementing the industrial defect detection method according to any one of claims 1 to 4, characterized by, The method comprises the following steps: a storage module configured to store at least one CNN classification network and a training set; wherein the CNN classification network has a global average pooling layer; a camera module configured to take a picture of a test object to obtain a first test image having industrial defects; an image input module configured to receive the first test image, preprocess the test image, and output a second test image; a training module configured to call a CNN classification network from the storage module, and train the CNN classification network using the training set; a calculation module configured to receive the second test image, identify defects in the second test image using the trained CNN classification network, and classify the defects.

6. The industrial defect detection device of claim 5, wherein: the CNN classification network is Resnet, Mobilenet v1, Mobilenet v2, or Mobilenet v3.

7. The industrial defect detection device of claim 5, wherein: the image input module comprises a detection model for detecting the positions of defects in the first test image, cutting the first test image according to the positions of the defects to obtain a second test image, and outputting the second test image.

8. The industrial defect detection device of claim 5, wherein: the training module trains the CNN classification network using samples, specifically comprising: in the training module, setting the batchsize of the training to 1, dividing the accumulated gradient by N after training N samples, and then performing parameter update of the CNN classification network.

9. The industrial defect detection device of claim 5, wherein: A data processing module is configured to perform data equalization and / or data enhancement on the second to-be-tested image to obtain a third to-be-tested image. A storage module is configured to store the third to-be-tested image, and is configured to construct or update a training set.

Images