Defect extraction method based on target detection

Abstract

The present invention relates to a defect extraction method based on target detection, comprising the following steps: S1, based on the annotation tool, pixel-level annotation and / or rectangular frame annotation are performed on the defects; S2, with the defect as the center, according to the set window size, small image slices are cut out on the original large image and saved in the corresponding category folder; S3, the number of samples in each folder is balanced; S4, the samples are processed in the data format of the coco data set to obtain a txt label file; the label file includes the category of the defect, the coordinates of the center of the defect, and the normalized information of the length and width of the circumscribed rectangle of the defect; S5, the samples are subjected to post-inference processing. The present invention not only reduces the overkill problem of defect detection and reduces the difficulty of the post-processing algorithm; but also reduces the detection time of a single image, greatly improving the operation efficiency of the machine.

Description

Technical Field

[0001] The invention relates to the technical field of image vision detection, and in particular to a defect extraction method based on target detection. Background Art

[0002] Defect detection is a key link in modern industry, and plays a decisive role in improving the factory quality of products and the quality of finished components. However, the previous manual inspection was cumbersome, with inconsistent standards and a large impact on human subjectivity. In an era of increasing labor costs, it seems inadequate; while the later machine inspection has the advantages of unified standards, low labor costs, and fast inspection speed. Therefore, visual inspection solutions have become the first choice of major inspection companies. Commonly used methods include defect detection methods based on traditional visual algorithms and defect detection methods based on deep learning algorithms.

[0003] However, the above visual inspection scheme has the following disadvantages:

[0004] 1. The defect detection method based on traditional visual algorithms has the disadvantages of requiring customization and being highly dependent on the personal ability of algorithm engineers. In addition, the customized algorithm is less versatile and highly dependent on hardware conditions such as lighting and mechanism movement. The detection CT of a single image is relatively long.

[0005] 2. The defect detection method based on deep learning algorithm has the disadvantage that although the defect detection capability is acceptable, the over-inspection is very serious, the defect detection capability is weakened under complex backgrounds, the overall control of defects is weak, and the detection time is relatively long. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a defect extraction method based on target detection to solve the over-detection problem of industrial defect detection and the detection speed problem of large-size images.

[0007] The technical solution adopted by the present invention to solve the technical problem is: a defect extraction method based on target detection, comprising the following steps:

[0008] S1. Based on the annotation tool, the defects are annotated at the pixel level and with rectangular boxes;

[0009] S2, taking the defect as the center, cut out small slices from the original large image according to the set window size and save them to the corresponding category folder;

[0010] S3, balance the number of samples in each folder;

[0011] S4. Process the sample in the data format of the coco data set to obtain a txt label file; the label file includes the category of the defect, the coordinates of the center of the defect, and the normalized information of the length and width of the circumscribed rectangle of the defect;

[0012] S5. Perform post-processing on the samples.

[0013] Furthermore, in step S1 of the present invention, the pixel-level annotation covers all defective pixels, and non-defective areas are not annotated.

[0014] Furthermore, in step S1 of the present invention, for relatively discrete and concentrated defects, pixel-level annotation is performed first, and then rectangular frame annotation is performed.

[0015] Furthermore, in step S2 of the present invention, if the defects are of the same category and are relatively close in distance, the small image slices are saved in the same folder; if the defects are relatively far in distance, the small image slices are saved in different folders.

[0016] Furthermore, in step S2 of the present invention, the rectangular frame annotation is firstly subjected to a cut image processing; the cut small rectangular frame contains the pixel-level annotation, and the small rectangular frame that does not contain the pixel-level annotation is discarded.

[0017] Furthermore, in step S2 of the present invention, after the rectangular frame annotation is processed, the pixel-level annotations of different categories within the rectangular frame annotation are retained and enter the subsequent pixel-level annotation processing.

[0018] Furthermore, in step S3 of the present invention, the intra-sample balance is maintained by a replication method or the intra-sample balance is achieved by a defect creation method.

[0019] Furthermore, in step S4 described in the present invention, if the width w of the circumscribed rectangle of the defect is greater than the height h*2, the defect is considered to be a longer horizontal defect, and the defect is divided into several small defects with a width of 2*h and a height of h; if the height h is greater than the width w*2, it is considered to be a longer vertical defect, and the defect is divided into several small defects with a width of w and a height of 2*w.

[0020] The beneficial effects of the present invention are that it solves the defects existing in the background technology, avoids the over-detection problem of the traditional pixel-level detection algorithm by the ability of overall detection of the target object based on target detection; reduces the detection speed problem of large-size images by cropping pictures and multi-batch processing of GPU; not only reduces the over-killing problem of defect detection and reduces the difficulty of post-processing algorithms; but also reduces the detection time of a single image, greatly improving the operation efficiency of the machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0022] The present invention will now be described in further detail in conjunction with the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0023] like Figure 1 A defect extraction method based on target detection is shown, comprising the following steps:

[0024] 1. Labeling strategy

[0025] Based on the annotation tool, defects are annotated at the pixel level and with rectangular frames. The annotation situations can be roughly divided into the following two categories.

[0026] 1.1 Pixel-level annotation

[0027] Pixel-level annotation is effective for all defects. The annotation principle is complete, clear, and covers all defective pixels, and does not annotate non-defective areas.

[0028] 1.2 Rectangular box annotation + pixel level annotation

[0029] For defects that are relatively discrete and concentrated, the principle of pixel-level annotation + rectangular frame annotation is used for annotation. For defects that are relatively discrete, pixel-level annotation must be used first. If it is believed that such defects exist as a whole, a rectangular frame needs to be added (Note: there is no limit on the size of the box for box annotation, which is based on the actual edge of the defect).

[0030] 2. Sample cutting + balancing strategy

[0031] 2.1 Sample Cutting

[0032] Traverse all defects, take the defect as the center, use the model size of 640*640 as the window size, draw a small slice on the original large image, and put it into the corresponding category folder; if the original image is smaller than 640*640, pad it to 640*640 size; for example, if two defects are of the same category and are close to each other, two slices will be generated in the same folder; if two defects are relatively far away, two slices will be generated and put into two different folders respectively.

[0033] 2.2 Sample cutting priority

[0034] In the sample cut image, there may be rectangular box annotations and pixel-level annotations. In the cut image processing, the rectangular box annotations are processed first. In the rectangular box cutting process, the pixel-level annotations must prevail.

[0035] If the cut-off condition is met, the cut-out small rectangular box must contain pixel-level annotations, and the small rectangular box that does not contain pixel-level annotations will be discarded;

[0036] If the cut-off condition is not met, it is processed according to the original frame;

[0037] The rectangular box and the pixel-level annotation are not repeated, and the pixel-level annotation within the rectangular box will not enter the next step of pixel-level annotation and image cutting processing.

[0038] When different categories of annotations appear in the rectangular box, after the rectangular box is processed, the pixel-level annotations of different categories in the rectangular box should be retained and enter the subsequent pixel-level annotation processing.

[0039] 2.3 Balance between sample classes

[0040] After the above sample cutting, a series of sample folders of different categories can be obtained;

[0041] Due to the natural imbalance of samples, the number of sample slices in each folder varies, so the number of samples in the folder needs to be balanced. The balance of quantity can be maintained by copying or by creating defects. The following uses the copying method to illustrate how to maintain the balance of quantity.

[0042] According to the number of input samples, such as 5000, assuming there are 5 defect categories, that is, each category has 5000 / 5 = 1000 samples. According to the cut situation,

[0043] If the number of files in each category is less than 1000, the number of files in each category folder will be increased to 1000 by copying.

[0044] If a class is larger than 1000, for example, 1500, then 1500 is used as the benchmark, and the remaining classes are copied to make up to 1500, and the final total number of samples is 1500*5=7500.

[0045] 3. Aspect ratio processing

[0046] After the above processing, we can get a small image and a label image sliced ​​from the original image. The label in the target detection is in the format of the position of the box, so the label image needs to be processed to get a txt label file in the coco data format.

[0047] 3.1 Annotation Data Format

[0048] The data format is processed in the data format of the coco dataset.

[0049] For example, there is only one defect in the original image, so there is only one line in the label, which contains the category, the x and y coordinates of the center of the defect, and the normalized information of the length and width w and h of the defect. The normalization method is coordinates (or length and width) / image size. For example, the coordinates of the defect center are (320, 350), w=7, h=8.5. So x = 320 / 640=0.500000, y = 350 / 640 =0.547266, w = 7 / 640 =0.010937, h = 8.5 / 640 = 0.013281.

[0050] If there are multiple defects in the image, multiple lines of similar information will be included.

[0051] 3.2 Extremely large aspect ratio processing

[0052] If the label image contains defects with a large aspect ratio, a one-step interruption process is required, as follows:

[0053] Traverse all defects in the label graph and obtain the width w and height h of the defect's circumscribed rectangle. If w>2*h, the defect is considered to be a long horizontal defect, and the defect is divided into several small defects with a width of 2*h and a height of h.

[0054] If h>2*w, it is considered to be a long vertical defect, and the defect is divided into several small defects with a width of w and a height of 2*w;

[0055] The rest are left uncut.

[0056] The x, y, w, and h of all defects are normalized and written into each line of the label in txt format. Each line contains 5 data, namely category (only 0 in the detection model), x, y, w, and h.

[0057] The defects in the original image are relatively long in length and width, and after processing they can be connected one by one.

[0058] 3.3 Large physical size but normal aspect ratio

[0059] If the aspect ratio of the marked defect is normal, less than 1:2, but the actual size is larger, the defect is truncated according to the open parameters.

[0060] Finally, the post-processing of the samples is performed for inference.

[0061] For the detection network used in this embodiment, a confidence parameter of the target can be set, which can be considered as a filtering threshold in the conventional sense. In order to ensure that no detection is missed as much as possible, the recommended value is set to 0.3.

[0062] 4.1 Reasoning at a Single Scale

[0063] When the original image length and width are no greater than 640*(2~3), and most of the annotated defects are concentrated below 600, it is recommended to use single-scale reasoning. The complete reasoning process is as follows:

[0064] 1) Slice the original image into small pieces by windowing and record the coordinates of each piece for easy restoration. Windowing requires overlap, and the recommended value is half the model input size.

[0065] 2) The frames inferred from each image are merged by adding 3 pixels to the top, bottom, left and right. Then, the binary image containing the defect is obtained through the logic of binary image mapping.

[0066] 3) After the inference of an original image is completed, the inference result map of the original image is restored according to the coordinate information recorded by the slice.

[0067] 4.2 Multi-scale Reasoning

[0068] When the length and width of the original image is greater than 640*3, you can perform a supplement on the inference result of a single scale. The complete process is as follows:

[0069] 1) On the basis of a single scale, slices with a window size of 640*N are added, and these slices are resized to the model input size of 640;

[0070] 2) Reasoning is the same as reasoning on a single scale;

[0071] The restoration is the same as above. The binary images obtained after slice inference at different scales are restored and mapped to the original image to obtain the final result.

[0072] This solution is an algorithm based on convolutional neural networks. Through learning sample labels and after a certain number of iterations, a stable model can be formed, so engineers do not need to manually intervene in the model. The model has a certain degree of generalization. Even if the imaging environment changes slightly, the model can adapt well to such changes. In contrast to traditional industrial fields, once the imaging environment changes, algorithm engineers need to make corresponding adjustments to the algorithm model. In addition, the convolutional neural network algorithm, with the support of GPU, greatly improves the operating efficiency of the model, which is a significant improvement compared to traditional algorithms running on CPUs.

[0073] At the same time, compared with the defect detection using deep learning algorithms in the existing technology, they use a segmentation algorithm to extract defects at the pixel level. Although the detection capability of the defect itself is acceptable, the over-detection phenomenon is more serious for some normal pixels. This is because the segmentation model weakens the information of the defect itself as a whole. The defect detection model proposed in this scheme, by combining information of different sizes of defects, can not only detect defects better, but also greatly reduce the degree of over-detection of defects.

[0074] The above description only describes the specific implementation mode of the present invention. Various examples do not limit the essential content of the present invention. After reading the description, ordinary technicians in the relevant technical field can modify or deform the specific implementation modes described above without departing from the essence and scope of the invention.

Claims

1. A defect extraction method based on target detection, characterized in that: The following steps are included: S1. Based on the annotation tool, the defects are annotated at the pixel level and with rectangular boxes; S2. Traverse all defects, take the defect as the center, cut small image slices from the original large image according to the set window size, and save them to the corresponding category folder; If the defects are of the same category and are relatively close, save the small image slices in the same folder; If the defect is relatively far away, save the small image slices in different folders; In the sample cut image, the rectangular box annotation is first cut; the cut small rectangular box contains the pixel-level annotation, and the small rectangular box that does not contain the pixel-level annotation is discarded; After processing the rectangular box annotation, the pixel-level annotations of different categories within the rectangular box annotation are retained and enter the subsequent pixel-level annotation processing; S3, balance the number of samples in each folder; S4. Process the sample in the data format of the coco data set to obtain a txt label file; the label file includes the category of the defect, the coordinates of the center of the defect, and the normalized information of the length and width of the circumscribed rectangle of the defect; If the width w of the circumscribed rectangle of the defect is greater than the height h*2, the defect is considered to be a long horizontal defect, and the defect is divided into several small defects with a width of 2*h and a height of h; if the height h is greater than the width w*2, it is considered to be a long vertical defect, and the defect is divided into several small defects with a width of w and a height of 2*w; S5, perform post-processing reasoning on the samples; the reasoning process includes single-scale reasoning and multi-scale reasoning; The reasoning process of a single scale is: 1) Slice the original image into small pieces by window cutting and record the coordinates of each small piece; 2) The frames inferred from each image are merged by adding 3 pixels to the top, bottom, left and right. Then, the binary image containing the defect is obtained through the logic of binary image mapping. 3) After the inference of an original image is completed, the inference result map of the original image is restored according to the coordinate information recorded by the slice; The multi-scale reasoning is a supplement to the single-scale reasoning result, and the process is as follows: a) Based on the single-scale slicing, slices with a window size of 640*N are added, and the size of such slices is adjusted to the input size of the model 640; b) Reasoning is the same as reasoning on a single scale; c) Restoration: As above, the binary images obtained after slice inference at different scales are restored and mapped to the original image to obtain the final result.

2. The defect extraction method based on target detection according to claim 1, characterized in that: In the step S1, the pixel-level annotation covers all defective pixels, and non-defective areas are not annotated.

3. The defect extraction method based on target detection according to claim 1, characterized in that: In the step S1, for relatively discrete and concentrated defects, pixel-level annotation is performed first, and then rectangular frame annotation is performed.

4. The defect extraction method based on target detection according to claim 1, characterized in that: In the step S3, the balance within the sample is maintained by a replication method or the balance within the sample is achieved by a defect creation method.

Images