Deep Learning-Based Defect Detection Method, System and Equipment for Small Hardware Parts

Abstract

The present invention provides a method for detecting defects in small hardware parts based on deep learning, which comprises the following steps: selecting sample images of small hardware parts, performing preprocessing to construct a CIFAR-100 sample set, and obtaining a defect recognition model based on a sample convolutional neural network; performing image acquisition on the small hardware parts and gray-scaling processing to obtain a grayscale image, performing image segmentation on the grayscale image, and obtaining a target motion foreground; calculating the target motion foreground image based on a nine-square grid algorithm, segmenting the obtained image of the small hardware parts to be detected into several superpixel blocks, calculating the significance value of each superpixel block, and then obtaining a significance map by fusing the significance of multi-scale superpixel blocks; constructing a minimum circumscribed square for the significance map, normalizing and resizing it, and inputting it into a defect recognition model to determine defects in the image of the small hardware parts to be detected. The present invention is applicable to a variety of small target hardware parts and other parts, is relatively simple to operate, and also improves detection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of hardware surface defect detection technology, and in particular to a method, system and equipment for small hardware defect detection based on deep learning. Background Art

[0002] Image target detection (also known as object detection) is a method of locating and identifying objects of a specific category from an image. It is an important part of machine automation processing. Its function is to locate the position of the target to be found and identify the target category to assist the machine in further processing. Target detection methods are widely used in medicine, transportation, the Internet, data statistics, military and other fields, and have successfully realized automatic camera focusing, face detection, vehicle detection, drone detection, building detection, etc.

[0003] In power systems, most equipment has a wide variety of target sizes. Small target detection, as a crucial component of target detection, is an equally important requirement. Furthermore, small target detection is particularly important in common imagery, such as long-range target images, large field-of-view target images, and densely packed target images, requiring high precision. Therefore, its accuracy determines the overall target detection accuracy, as seen in remote sensing imagery, drone aerial photography, and pedestrian detection. Furthermore, important and special target detection often involves small target detection. Small hardware defects are common in power equipment and are a widespread and crucial component of power equipment defect detection. Existing deep learning-based small target detection methods are cumbersome, resulting in low efficiency and poor accuracy when detecting small targets. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system and equipment for detecting defects in small hardware parts based on deep learning, so as to solve the problems of the existing small target detection method based on deep learning in the background technology, which is cumbersome to operate, has low efficiency and poor accuracy when detecting small targets.

[0005] The embodiment of the present invention is implemented through the following technical solution: a method for detecting defects in small hardware based on deep learning, comprising the following steps:

[0006] Step 1: Select sample images of small hardware parts, preprocess them to construct the CIFAR-100 sample set, train the convolutional neural network based on the CIFAR-100 sample set, and obtain a trained defect recognition model;

[0007] Step 2: Capture images of small hardware parts and perform grayscale processing to obtain a grayscale image, perform image segmentation on the grayscale image, and obtain the target motion foreground;

[0008] Step 3: Calculate the target motion foreground image based on the Nine-squares algorithm to obtain the image of the small hardware to be detected and divide it into several superpixel blocksS up , calculate each superpixel block S up The significance value of s , and then obtain the saliency map by fusing the saliency of multi-scale super-pixel blocks;

[0009] Step 4: construct a minimum circumscribed square for the saliency map, normalize it, resize it to 32×32, input it into the defect recognition model, and determine the defects in the image of the small hardware to be detected.

[0010] According to a preferred embodiment, the step 1 specifically includes:

[0011] 1.1. For the images in the sample set, the image resolution is gradually reduced by dilation followed by erosion to obtain the first small target feature map;

[0012] 1.2. Copy the image processed in step 1.1, segment the defect mask, and obtain the second small target feature map;

[0013] 1.3. Copy the first small target feature map obtained after step 1.1 and obtain the third small target feature map by rotating it;

[0014] 1.4. Copy the first small target feature map obtained after step 1.1 and scale it to obtain the fourth small target feature map;

[0015] 1.5. All small target feature maps are fused to obtain the feature pyramid network FPN, which serves as the backbone network of the defect recognition model.

[0016] According to a preferred embodiment, the convolutional neural network in step 1 includes:

[0017] Input layer, input 32×32 image;

[0018] Convolutional layer C1 outputs 6 5*5 convolution kernels with a stride of 1;

[0019] The downsampling layer S1 uses the maximum pooling method to output a kernel of size 2*2 with a stride of 2;

[0020] Convolutional layer C2 outputs 12 5*5 convolution kernels with a stride of 1;

[0021] The downsampling layer S2 uses the maximum pooling method to output a kernel of size 2*2 with a stride of 2;

[0022] The full connection layer uses ReLU as the activation function and outputs N neurons. ;

[0023] Output layer, outputs a four-dimensional vector.

[0024] According to a preferred embodiment, in the step 2, a mixture Gaussian model is combined with a three-frame difference method to detect the target motion foreground image.

[0025] According to a preferred embodiment, the maximum pooling method in the downsampling layer S1 and the downsampling layer S2 is replaced by an average pooling method or a random pooling method.

[0026] According to a preferred embodiment, in step 3, the super pixel block is calculated in the following manner: S up The significance value of s :

[0027] Superpixel blocks S up Perform clustering and obtain n clusters, let the cluster center eigenvector of the first cluster be B 1. The cluster center eigenvector of the second cluster is B 2. Similarly, we get the cluster center eigenvector ;

[0028] The superpixel block is calculated using the following formula S up The significance value of s :

[0029]

[0030] in:

[0031]

[0032] In the above formula, Superpixel block S up Hedi i The cluster center eigenvectors of clusters B i The distance between , is the scale factor, w is the weight, ( x , y ) is a super pixel block S up The center coordinates of is the center coordinate of the image of the small hardware to be detected.

[0033] According to a preferred embodiment, in step 4, bicubic interpolation is used for normalization.

[0034] The present invention also provides a small hardware defect detection system based on deep learning, which is applied to the above-mentioned method, including:

[0035] The model training unit is used to select sample images of small hardware parts, preprocess them to construct the CIFAR-100 sample set, and train the convolutional neural network based on the CIFAR-100 sample set to obtain a trained defect recognition model;

[0036] The image segmentation unit is used to collect images of small hardware parts and perform grayscale processing to obtain a grayscale image, and then detect the target moving foreground image by combining a mixed Gaussian model with a three-frame difference method;

[0037] The saliency map acquisition unit is used to calculate the target motion foreground image based on the nine-square grid algorithm to obtain the image of the small hardware to be detected and divide it into several super pixel blocks S up , calculate each superpixel block S up The significance value of s , and then obtain the saliency map by fusing the saliency of multi-scale super-pixel blocks;

[0038] The input unit is used to construct a minimum circumscribed square for the saliency map, and then adjust its size to 32×32 through bicubic interpolation, and then input it into the defect recognition model to determine the defects in the image of the small hardware to be detected.

[0039] According to a preferred embodiment, the model training module further includes a feature pyramid network (FPN) construction module for reducing the image resolution of the image in the sample set by first dilating and then corroding to obtain a first small target feature map;

[0040] Copy the first small target feature map, segment the defect mask, and obtain the second small target feature map;

[0041] Copy the first small target feature map and obtain the third small target feature map by rotating it;

[0042] Copy the first small target feature map and obtain the fourth small target feature map by scaling;

[0043] All small target feature maps are fused to obtain the feature pyramid network FPN, which is used as the backbone network of the defect recognition model.

[0044] The present invention also provides a small hardware defect detection device based on deep learning, comprising:

[0045] Memory for storing computer programs;

[0046] A processor is used to implement the steps of the small hardware defect detection method based on deep learning as described above when executing the computer program.

[0047] The technical solution of the embodiment of the present invention has at least the following advantages and beneficial effects: (1) The defect detection of the present invention is applicable to a variety of small-target hardware and other parts; (2) By using deep convolutional neural network recognition, the number of layers can be increased according to actual conditions, and more abstract features of defects can be extracted, with higher recognition accuracy; (3) By using the ReLU activation function, the training process will not be fitted, the calculation is simple and effective, and the convergence speed is fast; (4) The data set is processed to make the features more obvious, thereby improving the performance of target detection, the operation is relatively simple, and the detection efficiency is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic flow chart of a method for detecting defects in small hardware parts based on deep learning provided in Example 1 of the present invention;

[0049] Figure 2 This is a diagram of the convolutional neural network structure provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0051] Example 1

[0052] See Figure 1 As shown, Figure 1 A schematic flow chart of a method for detecting defects in small hardware parts based on deep learning provided by an embodiment of the present invention is shown.

[0053] The applicant has found through research that in the power system, most of the power system equipment has a diversity of detection target sizes, and small target detection, as a part of target detection, has an equally important practical need. Not only that, small target detection has a large proportion and high precision requirements in common images such as long-distance target images, large field of view target images, and target dense images. Therefore, its precision determines the overall target detection accuracy, such as remote sensing image target detection, drone aerial photography target detection, and pedestrian target detection. In addition, important special target detection is also mostly small target detection. Power equipment often has small hardware defects, which are an extremely widespread and important part of power equipment defect detection. The existing small target detection method based on deep learning is relatively cumbersome to operate, and has low efficiency and poor accuracy when detecting small targets. Therefore, the present invention provides a small hardware defect detection method based on deep learning to solve the above problems. The specific solution includes the following steps:

[0054] Step 1: Select sample images of small hardware parts, pre-process them to construct a CIFAR-100 sample set, train a convolutional neural network based on the CIFAR-100 sample set, and obtain a trained defect recognition model. In one implementation of this embodiment, step 1 specifically includes:

[0055] For the images in the sample set, the image resolution is reduced by first dilating and then eroding to obtain the first small target feature map. The first small target feature map is copied and used to segment the defect mask to obtain the second small target feature map. The first small target feature map is copied and rotated to obtain the third small target feature map. The first small target feature map is copied and scaled to obtain the fourth small target feature map. All small target feature maps are fused to form the Feature Pyramid Network (FPN), which serves as the backbone network of the defect recognition model. These different stages include shallow and deep features. Through the Feature Pyramid Network (FPN), the model can handle the multi-scale variation problem in small target detection with minimal increase in computational effort, further improving detection speed.

[0056] See Figure 2 As shown, the convolutional neural network in step 1 includes: input layer, input 32×32 image; convolution layer C1, output 6 5*5 convolution kernels with a step size of 1; downsampling layer S1, using the maximum pooling method to output 2*2 size kernels with a step size of 2; convolution layer C2, output 12 5*5 convolution kernels with a step size of 1; downsampling layer S2, using the maximum pooling method to output 2*2 size kernels with a step size of 2; full connection layer, using ReLU as the activation function, output N neurons, The output layer outputs a four-dimensional vector. It should be noted that when using deep convolutional neural network recognition, the number of layers can be increased according to actual conditions to extract more abstract defect features and achieve higher recognition accuracy. Furthermore, the maximum pooling method in downsampling layers S1 and S2 can be replaced with average pooling or random pooling, which will not be discussed further here.

[0057] Step 2: Capture images of small hardware parts and perform grayscale processing to obtain a grayscale image. Use a mixed Gaussian model combined with a three-frame difference method to detect the target moving foreground image. Perform image segmentation on the grayscale image to obtain the target moving foreground. Furthermore, in this embodiment, the detection process using a mixed Gaussian model combined with a three-frame difference method is as follows: Use the three-frame difference method to calculate the inter-frame difference image of adjacent grayscale images, then perform thresholding, morphological processing, and operations to obtain the moving foreground. The inter-frame difference image calculation formula is as follows:

[0058]

[0059]

[0060] Furthermore, the moving foreground is input, and the background is modeled according to the mixed Gaussian model, and k Gaussian models are established for each pixel in the image; further, the K Gaussian distributions of each pixel are arranged from large to small, and the closer the distribution is to the front, the greater the possibility that the distribution describes the background; the n distributions arranged in front are selected as the background pixel model according to the threshold, wherein the threshold takes the empirical value of 0.75; further, the current frame is obtained, and each pixel value is matched with the K Gaussian distributions respectively. If the match is successful, the parameters of the first matching Gaussian distribution are updated. If the match fails, the mean of the Gaussian distribution with the smallest weight is modified to the current pixel value, thereby realizing the segmentation of the moving foreground.

[0061] Step three, calculate the target motion foreground image based on the nine-square grid algorithm to obtain a stretched image. It should be noted that the image stretching and scaling by the nine-square grid algorithm can reduce the degree of image distortion. The specific calculation method is as follows: keep the four corners of the motion foreground from being stretched, define an image block around the area block of the suspected target in the motion foreground as the center, set the multiple of the image block and the area block, and this embodiment sets the side length of the image block to 3 times the area block; further, take the image block as the center and its surrounding area as the background area to calculate the local contrast value of the image block; judge according to the calculation result, if the image block is the target area, stretch the image block, if the image block is not the target area, suppress the background area, and then generate a stretched image. Further, obtain the image of the small hardware to be detected and divide it into several super pixel blocks S up , calculate each superpixel block S up The significance value of s , and then obtain the saliency map by fusing the saliency of multi-scale super pixel blocks. In one implementation of this embodiment, the following method is used to calculate the super pixel block S up The significance value of s :

[0062] Superpixel blocks S up Perform clustering and obtain n clusters, let the cluster center eigenvector of the first cluster be B 1. The cluster center eigenvector of the second cluster is B 2. Similarly, we get the cluster center eigenvector ;

[0063] The superpixel block is calculated using the following formula S up The significance value of s :

[0064]

[0065] in:

[0066]

[0067] In the above formula, Superpixel block S up Hedi i The cluster center eigenvectors of clusters B i The distance between , is the scale factor, w is the weight, ( x , y ) is a super pixel block S up The center coordinates of is the center coordinate of the image of the small hardware to be detected.

[0068] Step 4: Construct a minimum circumscribed square for the saliency map. It should be noted that the image size of the CIFAR-100 dataset is 32×32. To further improve recognition accuracy, this embodiment uses bicubic interpolation to normalize the image, adjust the size to 32×32, and input it into the defect recognition model to determine the defects in the image of the small hardware to be detected.

[0069] An embodiment of the present invention also provides a small hardware defect detection system based on deep learning, which is applied to the above-mentioned method, including:

[0070] The model training unit is used to select sample images of small hardware parts, preprocess them to construct the CIFAR-100 sample set, and train the convolutional neural network based on the CIFAR-100 sample set to obtain a trained defect recognition model;

[0071] The image segmentation unit is used to collect images of small hardware parts and perform grayscale processing to obtain a grayscale image, and then detect the target moving foreground image by combining a mixed Gaussian model with a three-frame difference method;

[0072] The saliency map acquisition unit is used to calculate the target motion foreground image based on the nine-square grid algorithm to obtain the image of the small hardware to be detected and divide it into several super pixel blocks S up , calculate each superpixel block S up The significance value of s , and then obtain the saliency map by fusing the saliency of multi-scale super-pixel blocks;

[0073] The input unit is used to construct a minimum circumscribed square for the saliency map, and then adjust its size to 32×32 through bicubic interpolation, and then input it into the defect recognition model to determine the defects in the image of the small hardware to be detected.

[0074] Furthermore, in this embodiment, the model training module further includes a feature pyramid network (FPN) construction module for reducing the image resolution of the images in the sample set by first dilating and then eroding to obtain a first small object feature map;

[0075] Copy the first small target feature map, segment the defect mask, and obtain the second small target feature map;

[0076] Copy the first small target feature map and obtain the third small target feature map by rotating it;

[0077] Copy the first small target feature map and obtain the fourth small target feature map by scaling;

[0078] All small target feature maps are fused to obtain the feature pyramid network FPN, which is used as the backbone network of the defect recognition model.

[0079] The embodiment of the present invention also provides a small hardware defect detection device based on deep learning, including:

[0080] Memory for storing computer programs;

[0081] A processor is used to implement the steps of the small hardware defect detection method based on deep learning as described above when executing the computer program.

[0082] In summary, the technical solutions of the embodiments of the present invention have at least the following advantages and beneficial effects: (1) The defect detection of the present invention is applicable to a variety of small-target hardware and other parts; (2) By using deep convolutional neural network recognition, the number of layers can be increased according to actual conditions, and more abstract features of defects can be extracted, resulting in a higher recognition accuracy; (3) By using the ReLU activation function, the training process will not be fitted, the calculation is simple and effective, and the convergence speed is fast; (4) The data set is processed to make the features more obvious, thereby improving the performance of target detection, the operation is relatively simple, and the detection efficiency is also improved.

[0083] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for detecting defects in small hardware parts based on deep learning, characterized in that: The steps include: Step 1: Select sample images of small hardware parts, preprocess them to construct the CIFAR-100 sample set, train the convolutional neural network based on the CIFAR-100 sample set, and obtain a trained defect recognition model; Step 2: Capture images of small hardware parts and perform grayscale processing to obtain a grayscale image, perform image segmentation on the grayscale image, and obtain the target motion foreground; Step 3: Calculate the target motion foreground image based on the Nine-squares algorithm to obtain the image of the small hardware to be detected and divide it into several superpixel blocks S up , calculate each superpixel block S up The significance value of s , and then obtain a saliency map by fusing the saliency of multi-scale super-pixel blocks, wherein the calculation method of the target motion foreground image based on the nine-square grid algorithm is as follows: keep the four corners of the motion foreground from being stretched, define an image block around the area block of the suspected target in the motion foreground, set the multiple of the image block and the area block, take the image block as the center, and use its adjacent area as the background area to calculate the local contrast value of the image block. Make a judgment based on the calculation result. If the image block is the target area, stretch the image block. If the image block is not the target area, suppress the background area to generate a stretched image; Step 4: construct a minimum circumscribed square for the saliency map, normalize it, resize it to 32×32, input it into the defect recognition model, and determine the defects in the image of the small hardware to be detected; In step 3, the super pixel block is calculated as follows: S up The significance value of s : Superpixel blocks S up Perform clustering and obtain n clusters, let the cluster center eigenvector of the first cluster be B 1. The cluster center eigenvector of the second cluster is B 2. Similarly, we get the cluster center eigenvector ; The superpixel block is calculated using the following formula S up The significance value of s : in: In the above formula, Superpixel block S up Hedi i The cluster center eigenvectors of clusters B i The distance between , is the scale factor, w is the weight, ( x , y ) is a super pixel block S up The center coordinates of is the center coordinate of the image of the small hardware to be detected.

2. The method for detecting defects in small hardware parts based on deep learning according to claim 1, characterized in that: The step 1 specifically includes: 1.

1. For the images in the sample set, the image resolution is gradually reduced by dilation followed by erosion to obtain the first small target feature map; 1.

2. Copy the image processed in step 1.1, segment the defect mask, and obtain the second small target feature map; 1.

3. Copy the first small target feature map obtained after step 1.1 and obtain the third small target feature map by rotating it; 1.

4. Copy the first small target feature map obtained after step 1.1 and scale it to obtain the fourth small target feature map; 1.

5. All small target feature maps are fused to obtain the feature pyramid network FPN, which serves as the backbone network of the defect recognition model.

3. The method for detecting defects in small hardware parts based on deep learning according to claim 1, characterized in that: The convolutional neural network in step 1 includes: Input layer, input 32×32 image; Convolutional layer C1 outputs 6 5*5 convolution kernels with a stride of 1; The downsampling layer S1 uses the maximum pooling method to output a kernel of size 2*2 with a stride of 2; Convolutional layer C2 outputs 12 5*5 convolution kernels with a stride of 1; The downsampling layer S2 uses the maximum pooling method to output a kernel of size 2*2 with a stride of 2; The full connection layer uses ReLU as the activation function and outputs N neurons. ; Output layer, outputs a four-dimensional vector.

4. The method for detecting defects in small hardware parts based on deep learning according to claim 1, wherein: In the step 2, a mixed Gaussian model is combined with a three-frame difference method to detect the target motion foreground image.

5. The method for detecting defects in small hardware parts based on deep learning according to claim 3, wherein: The maximum pooling method in the downsampling layer S1 and the downsampling layer S2 is replaced by an average pooling method or a random pooling method.

6. The method for detecting defects in small hardware parts based on deep learning according to claim 1, wherein: In the step 4, bicubic interpolation is used for normalization.

7. A small hardware defect detection system based on deep learning, applied to the method according to any one of claims 1 to 6, characterized in that: include: The model training unit is used to select sample images of small hardware parts, preprocess them to construct the CIFAR-100 sample set, and train the convolutional neural network based on the CIFAR-100 sample set to obtain a trained defect recognition model; The image segmentation unit is used to collect images of small hardware parts and perform grayscale processing to obtain a grayscale image, and then detect the target moving foreground image by combining a mixed Gaussian model with a three-frame difference method; The saliency map acquisition unit is used to calculate the target motion foreground image based on the nine-square grid algorithm to obtain the image of the small hardware to be detected and divide it into several super pixel blocks S up , calculate each superpixel block S up The significance value of s , and then obtain the saliency map by fusing the saliency of multi-scale super-pixel blocks; The input unit is used to construct a minimum circumscribed square for the saliency map, and then adjust its size to 32×32 through bicubic interpolation, and then input it into the defect recognition model to determine the defects in the image of the small hardware to be detected.

8. The small hardware defect detection system based on deep learning according to claim 7, characterized in that: The model training module also includes a feature pyramid network (FPN) construction module for reducing the image resolution of the image in the sample set by first dilating and then corroding to obtain a first small target feature map; Copy the first small target feature map, segment the defect mask, and obtain the second small target feature map; Copy the first small target feature map and obtain the third small target feature map by rotating it; Copy the first small target feature map and obtain the fourth small target feature map by scaling; All small target feature maps are fused to obtain the feature pyramid network FPN, which is used as the backbone network of the defect recognition model.

9. Small hardware defect detection equipment based on deep learning, characterized by: include: Memory for storing computer programs; A processor, configured to implement the steps of the small hardware defect detection method based on deep learning as described in any one of claims 1 to 6 when executing the computer program.

Images