Defect detection method, device, system and readable storage medium

Abstract

The present invention provides a defect detection method, device, system and readable storage medium, wherein the method comprises: collecting abnormal pictures of abnormal workpieces; performing defect detection on the abnormal pictures through a deep learning model, and outputting coordinate information of a labeling box on the abnormal pictures, wherein the labeling box selects the defective area on the abnormal pictures; cutting out a small picture containing the defective area according to the coordinate information; performing convolution processing on the small picture, and extracting the defective area on the small picture; performing fitting analysis on the defective area, and obtaining the minimum circumscribed circle corresponding to the defective area; judging whether the defective area is valid according to the diameter of the minimum circumscribed circle. The present invention can improve the efficiency of defect detection.

Description

Technical Field

[0001] The present invention relates to the field of screen defect detection technology, and in particular to a defect detection method, device, system and readable storage medium. Background Art

[0002] Currently, screen inspection, especially for small screens, is done manually. However, manual inspection is not only slow but also prone to errors in detecting small defects, which affects the efficiency of defect detection. Summary of the Invention

[0003] To solve the above problems, the defect detection method, device, system and readable storage medium provided by the present invention can improve the efficiency of defect detection by processing abnormal images through deep learning models and convolution processing.

[0004] In a first aspect, the present invention provides a defect detection method, comprising:

[0005] Collect abnormal images of abnormal workpieces;

[0006] Defect detection is performed on abnormal images through a deep learning model, and the coordinate information of the annotation box on the abnormal image is output. The annotation box selects the defect area on the abnormal image;

[0007] Crop a small image containing the defect area based on the coordinate information;

[0008] Perform convolution processing on the small image and extract the defect area on the small image;

[0009] Perform fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area;

[0010] The validity of the defect area is determined based on the diameter of the minimum circumscribed circle.

[0011] Optionally, before the step of performing convolution processing on the small image, the method further includes:

[0012] Normalize small images.

[0013] Optionally, the step of performing convolution processing on the small image includes:

[0014] Perform Fourier transform on the small image to obtain a frequency domain image;

[0015] The frequency domain image is convolved with the Gaussian difference image in the preset frequency domain to obtain the frequency domain defect image;

[0016] Perform inverse Fourier transform on the frequency domain defect image to obtain the time domain defect image;

[0017] The steps for extracting defect areas on a small image include:

[0018] Extract defect regions from time-domain defect images.

[0019] Optionally, before the step of performing defect detection on the abnormal image using a deep learning model, the method further includes:

[0020] Divide the abnormal image into multiple abnormal sub-images;

[0021] The steps of detecting defects in abnormal images using a deep learning model and outputting coordinate information of the annotation boxes on the abnormal images include:

[0022] Defect detection is performed on abnormal sub-images through a deep learning model, and the coordinate information of the annotation box on the abnormal sub-image is output.

[0023] Optionally, the step of performing fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area includes:

[0024] Select any point on the contour line of the defect area and mark it as point a;

[0025] Traverse the defect area to find the point farthest from point a, record it as point b, and record the distance S1 between point a and point b;

[0026] Traverse the defect area again to find the point farthest from point b, record it as point c, and record the distance S2 between point b and point c;

[0027] Determine whether S2 is equal to S1;

[0028] When S2 is equal to S1, the center of the minimum circumscribed circle is determined based on the perpendicular bisectors on the ab segment and the bc segment. The diameter of the minimum circumscribed circle is S2.

[0029] When S2 is not equal to S1, record S2 as S1, record point b as point a, and record point c as point b. Then traverse the defect area again to find the point farthest from point b, record it as point c, and record the distance S2 between point b and point c.

[0030] Optionally, the deep learning model is obtained by adding a detection head layer to the YOLO model;

[0031] Before performing defect detection on abnormal images using a deep learning model, the method also includes:

[0032] Use image annotation tools to select and annotate the defective area on the sample image;

[0033] The sample images after selection and annotation are input into the deep learning module to train the deep learning model.

[0034] Optionally, before the step of selecting and marking the defect area on the sample image using an image annotation tool, the method further includes:

[0035] Performing data enhancement processing on the sample image, where the data enhancement processing includes at least one of random cropping, random scaling, random flipping, random rotation, contrast adjustment, and brightness adjustment;

[0036] Divide the sample image into multiple sample sub-images.

[0037] In a second aspect, the present invention provides a defect detection device, comprising:

[0038] a collection module configured to collect abnormal images of abnormal workpieces;

[0039] The defect detection module is configured to detect defects in abnormal images using a deep learning model and output coordinate information of a labeling box on the abnormal image, where the labeling box selects the defect area on the abnormal image;

[0040] a cropping module configured to crop a small image containing a defect area according to the coordinate information;

[0041] A convolution processing module is configured to perform convolution processing on the small image and extract defect areas on the small image;

[0042] A fitting analysis module is configured to perform fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area;

[0043] The judging module is configured to judge whether the defect area is valid according to the diameter of the minimum circumscribed circle.

[0044] Optionally, the device further comprises:

[0045] The normalization module is configured to perform normalization processing on the small image before executing the convolution processing module.

[0046] Optionally, the convolution processing module includes:

[0047] A transform submodule is configured to perform Fourier transform on the small image to obtain a frequency domain image;

[0048] The operator module is configured to perform a convolution operation on the frequency domain image through a preset Gaussian difference image in the frequency domain to obtain a frequency domain defect image;

[0049] an inverse transform submodule, configured to perform an inverse Fourier transform on the frequency domain defect image to obtain a time domain defect image;

[0050] The extraction submodule is configured to extract the defect area from the time domain defect image.

[0051] Optionally, the device further comprises:

[0052] a segmentation module configured to divide the abnormal image into a plurality of abnormal sub-images before executing the defect detection module;

[0053] The defect detection module is also configured to perform defect detection on abnormal sub-images through a deep learning model and output coordinate information of the annotation box on the abnormal sub-image.

[0054] Optionally, the fitting analysis module includes:

[0055] The point selection submodule is configured to select any point on the contour line of the defect area, which is recorded as point a;

[0056] The first traversal submodule is configured to traverse the defect area to find the point farthest from point a, recorded as point b, and record the distance S1 between point a and point b;

[0057] The second traversal submodule is configured to traverse the defect area again to find the point farthest from point b, recorded as point c, and record the distance S2 between point b and point c;

[0058] A judgment submodule is configured to judge whether S2 is equal to S1;

[0059] The determination submodule is configured to determine the center of the minimum circumscribed circle according to the perpendicular bisector on the ab segment and the perpendicular bisector on the bc segment when S2 is equal to S1, and the diameter of the minimum circumscribed circle is S2;

[0060] The recording submodule is configured to record S2 as S1, point b as point a, point c as point b, and execute the second traversal submodule when S2 is not equal to S1.

[0061] Optionally, the deep learning model is obtained by adding a detection head layer to the YOLO model;

[0062] The device also includes:

[0063] A frame selection and annotation module is configured to use an image annotation tool to frame and annotate defect areas on a sample image before executing the defect detection module;

[0064] The input module is configured to input the sample image after box selection and annotation into the deep learning module to train the deep learning model.

[0065] Optionally, the device further comprises:

[0066] a data enhancement module configured to perform data enhancement processing on the sample image before executing the box selection and annotation module, wherein the data enhancement processing includes at least one of random cropping, random scaling, random flipping, random rotation, contrast adjustment, and brightness adjustment;

[0067] The dividing module is configured to divide the sample image into multiple sample sub-images.

[0068] In a third aspect, the present invention provides a defect detection system comprising a transparent conveyor belt, a photoelectric sensor, a back light source, a front ring light source, a lens, and a defect detection device; the defect detection device is used to perform any one of the defect detection methods in the first aspect;

[0069] The lens is electrically connected to the photoelectric sensor and the defect detection equipment respectively. The photoelectric sensor is fixedly arranged on one side of the transparent conveyor belt. The back light source is located below the transparent conveyor belt. The front ring light source and the lens are both located above the transparent conveyor belt. The optical axis of the lens coincides with the geometric center of the front ring light source and the back light source and is perpendicular to the transparent conveyor belt.

[0070] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any one of the methods in the first aspect when the computer program is executed by a processor.

[0071] The defect detection method, device, system and readable storage medium provided by the embodiments of the present invention can automatically identify and detect defects on abnormal images through deep learning models and convolution processing, and can quickly identify whether the size of the defect area meets the defect requirements based on the size of the minimum circumscribed circle corresponding to the defect area, thereby reducing the intensity of manual input and improving the efficiency of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0073] Figure 1 A schematic diagram of a partial structure of a defect detection device according to an embodiment of the present application;

[0074] Figure 2 A schematic diagram of a partial structure of a defect detection device according to an embodiment of the present application;

[0075] Figure 3 A schematic flow chart of a defect detection method according to an embodiment of the present application;

[0076] Figure 4 This is a schematic structural diagram of a defect detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0077] To facilitate understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The accompanying drawings provide embodiments of the present application. However, the present application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to make the disclosure of the present application more thorough and comprehensive.

[0078] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0079] When used herein, the singular forms "a", "an", and "the" may also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" and the like specify the presence of stated features, integers, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

[0080] Example 1

[0081] This embodiment provides a defect detection method, which is applied to a defect detection device, which may be a computer device, etc. Figure 1 and Figure 2 The defect detection system includes: a defect detection device (not shown in the figure), a transparent conveyor belt 1, a photoelectric sensor 2, a back light source 3, a front ring light source 4 and a lens 5, and the lens 5 is electrically connected to the photoelectric sensor 2 and the defect detection device respectively.

[0082] Among them, the photoelectric sensor 2 is fixedly set on one side of the transparent conveyor belt 1, the back light source 3 is located below the transparent conveyor belt 1, the front ring light source 4 and the lens 5 are both located above the transparent conveyor belt 1, and the optical axis of the lens 5 coincides with the geometric center of the front ring light source 4 and the back light source 3 and is perpendicular to the transparent conveyor belt 1.

[0083] When the transparent conveyor belt 1 moves the workpiece 6 horizontally to the inspection station, the workpiece 6 triggers the photoelectric sensor 2, which in turn triggers the lens 5 to take a picture, thereby obtaining an image of the workpiece. The lens 5 then transmits the workpiece image to the defect detection equipment for defect detection. It will be understood that the workpiece image includes abnormal images and normal images. An abnormal image indicates that the workpiece 6 captured by the lens 5 is abnormal, i.e., a defect exists on the surface of the workpiece 6. A normal image indicates that the workpiece 6 captured by the lens 5 does not have any defect. The specific structure of the defect detection system will not be elaborated in this embodiment.

[0084] It should be noted that after obtaining the workpiece image, the defect detection equipment will perform a preliminary screening of the workpiece image to filter out abnormal images. However, the size of the defect corresponding to the abnormal image still needs to be determined to determine whether the defect is valid, that is, whether it will affect the normal use of the workpiece 6. In this way, the workpiece 6 corresponding to the valid defect will enter the subsequent scrapping or maintenance process.

[0085] In this embodiment, the workpiece is a screen, preferably a small screen of less than 1 inch. Figure 3 , the defect detection method includes steps S101 to S106.

[0086] Step S101: Collect abnormal pictures of abnormal workpieces.

[0087] Step S102: perform defect detection on the abnormal image through a deep learning model, and output the coordinate information of the annotation box on the abnormal image.

[0088] The annotation box selects the defective area on the abnormal image. The deep learning module then selects the defective area on the output image with the annotation box to complete the labeling of the defective area. This facilitates subsequent tracing of the inspection results.

[0089] In a further optional embodiment of this embodiment, before the step of performing defect detection on the abnormal image through a deep learning model, the method further includes: dividing the abnormal image into multiple abnormal sub-images.

[0090] In this embodiment, the abnormal image is divided into 16 equally divided abnormal sub-images. Because the foreign body point defect to be detected can be as small as 5 pixels in diameter, it accounts for a very small proportion of the original 5-megapixel image, with a length-to-width ratio of 5 / 2500 = 0.2%. Currently, defects less than 3% are referred to as small target detection in the industry. Directly using the abnormal image for identification and detection cannot effectively extract foreign body features. Therefore, dividing the abnormal image into multiple abnormal sub-images improves the characteristics of small target defects and facilitates the identification and extraction of defect areas.

[0091] The steps of detecting defects in abnormal images using a deep learning model and outputting coordinate information of the annotation boxes on the abnormal images include:

[0092] Defect detection is performed on abnormal sub-images through a deep learning model, and the coordinate information of the annotation box on the abnormal sub-image is output.

[0093] It is understandable that the deep learning model will perform defect detection on each abnormal sub-image, but not every abnormal sub-image is marked with a labeling box. Abnormal sub-images without defective areas will not be marked with a labeling box.

[0094] In a further optional embodiment of this embodiment, a deep learning model is obtained by adding a layer of detection heads to the YOLO model. The YOLO model can be a model of YOLOv5 or higher. In this embodiment, the YOLO model v5 version is used as an example. The newly added detection head is derived from the P2 layer features on the YOLOv5 model. The resolution of the newly added detection head is 160x160 pixels, which is equivalent to only performing two downsampling operations on the backbone network of the YOLOv5 model. In this way, more abundant underlying feature information containing the target is obtained. The two P2 layer features obtained from the top-down and bottom-up in the neck network of the YOLOv5 model are fused with the same-scale features in the backbone network through the concat form. The output feature is the fusion result of the three input features. This enables the P2 layer detection head to quickly and effectively detect small targets. In this way, the newly added detection head on the P2 layer plus the original three detection heads can effectively alleviate the negative impact of scale variance caused by the YOLOv5 model during the calculation process.

[0095] Before performing defect detection on abnormal images through a deep learning model, the method also includes: using an image annotation tool to select and annotate defective areas on sample images; and inputting the selected and annotated sample images into a deep learning module to train the deep learning model.

[0096] Among them, image annotation tools include but are not limited to Labelme (label me) software.

[0097] In this embodiment, the yolov5s.pt pre-trained model weight file is selected, and the learning rate is set to 0.01, the batch size of each training is 64, the number of training rounds is 1000, and the size of the sample image is 640x640.

[0098] In a further optional embodiment of this embodiment, before the step of using an image annotation tool to select and annotate the defect area on the sample image, the method further includes:

[0099] The sample image is subjected to data enhancement processing, where the data enhancement processing includes at least one of random cropping, random scaling, random flipping, random rotation, contrast adjustment, and brightness adjustment.

[0100] By performing data augmentation on sample images, the size and diversity of the training dataset can be expanded, improving the generalization and robustness of the model. Because the screen is made of glass and is reflective, in this embodiment, data augmentation includes random cropping, random scaling, random flipping, random rotation, contrast adjustment, and brightness adjustment.

[0101] Specifically, random cropping: Randomly cropping an area from the sample image as a new image can change the image size, aspect ratio, and content. Among them, random cropping mainly selects the proportion of the sample image to be cropped. For example, if the random cropping proportion is L%, assuming that the sample image size is A = W * H, the final generated image size A1 is shown in formula (1):

[0102] A1=W*(1-L%)*H(1-L%) Formula (1)

[0103] In formula (1), W is the width of the image and H is the height of the image.

[0104] Random Scaling: Randomly scaling the sample image can change the size and content of the sample image. Taking the example of scaling the center point of the sample image, the image scaling algorithm based on equal-interval sampling actually involves taking pixels at a certain interval in the sample image and placing them on the output image. Assuming the size of the sample image is W*H, and the reduction factors for width and length are k1 and k2 respectively, the sampling interval is: W / k1, H / k2. In other words, a pixel is taken every W / k1 in the horizontal direction and every H / k2 in the vertical direction of the sample image. When the reduction factors k1 and k2 for length and width are equal, the image is scaled down in equal proportions. When they are not equal, the image is scaled down in unequal proportions, and the ratio of the length and width of the scaled image will change.

[0105] The implementation steps of the algorithm involved in random scaling include the following:

[0106] 1) Calculate the sampling interval: Assume that the size of the sample image is W*H, and enlarge (reduce) it to (k1*W)*(K2*H). Then the sampling interval in the width direction is ii=1 / k1; the sampling interval in the height direction is jj=1 / k2;

[0107] When k1 == k2, the image is scaled down; when k1 == k2, the image is scaled up (or down); when k1 < 1 && k2 < 1, the image is scaled down; when k1 > 1 && k2 > 1, the image is scaled up.

[0108] 2) Find the magnified (reduced) image: Let the sample image be F(x, y) (i = 1, 2, ... W; j = 1, 2, ... H), and the magnified (reduced) image be G(x, y) (x = 1, 2, ... M; y = 1, 2, ... N, where M = W*k1, N = H*k2), then G(x, y) = f(ii*x, jj*y)

[0109] Random flipping: Randomly flipping sample images horizontally or vertically can increase the diversity of the dataset.

[0110] Random rotation: Rotating sample images at random angles can increase the diversity of the dataset.

[0111] Contrast adjustment: The histogram corresponding to the contrast of the sample image is adjusted through contrast stretching, thereby expanding the difference in foreground and background grayscale to achieve the purpose of enhancing the contrast.

[0112] Brightness Adjustment: By adjusting the brightness of an image, you can make the sample image brighter or darker. This improves the diversity of the model's training samples. This can eliminate the impact of light source attenuation and brightness variations, as well as the effects of ambient light on imaging.

[0113] In a further optional embodiment of this embodiment, before using the image annotation tool to select and annotate the defect area on the sample image, the method further includes: dividing the sample image into a plurality of sample sub-images. The method for dividing the sample image into equal parts is the same as the method for dividing the abnormal image, and will not be further described here.

[0114] Step S103: cropping a small image containing the defective area according to the coordinate information.

[0115] Specifically, the position of the annotation box is determined according to the coordinate information, and then the annotation box is used as the cropping edge for cropping, thereby facilitating subsequent rapid processing of the defective area.

[0116] Step S104: performing convolution processing on the small image and extracting the defective area on the small image.

[0117] In a further optional embodiment of this embodiment, before the step of performing convolution processing on the small image, the method further includes: performing normalization processing on the small image.

[0118] By normalizing the small images, the grayscale values ​​of the small images containing the defective areas can be normalized, thereby increasing the difference between the defective areas and the background, making it easier to extract the area of ​​the defective areas.

[0119] Specifically, the small image is normalized and recorded as G(x, y). The normalization formula is shown in formula (2).

[0120]

[0121] Among them, g(x, y) is the grayscale value of the pixel in the xth row and yth column of the small image area; g max is the maximum grayscale value in the small image, g min It is the minimum grayscale value within the small image.

[0122] In a further optional embodiment of this embodiment, the step of performing convolution processing on the small image includes:

[0123] Perform Fourier transform on the small image to obtain a frequency domain image;

[0124] The frequency domain image is convolved with the Gaussian difference image in the preset frequency domain to obtain the frequency domain defect image;

[0125] The frequency domain defect image is subjected to inverse Fourier transform to obtain the time domain defect image.

[0126] Among them, performing Fourier transform on the small image is essentially the process of converting the small image from the time domain to the frequency domain. The Fourier formula is shown in formula (3).

[0127]

[0128] In formula (3), M is the total number of horizontal pixels of the small image, N is the total number of vertical pixels of the small image, j is an imaginary number, and the square of j is negative 1, F(u, v) is the grayscale value corresponding to the pixel point in the uth row and vth column in the frequency domain image after Fourier transformation, and f(x, y) is the grayscale value corresponding to the pixel point in the xth row and yth column in the small image.

[0129] Based on formula (3), the calculation formula of the Gaussian difference map in the preset frequency domain is shown in formula (4).

[0130]

[0131] Wherein, D(u, v) is the grayscale value corresponding to the pixel point in the uth row and vth column in the Gaussian difference image, θ1 and θ2 are both Gaussian parameters, generally θ2 is 10 to 15, and θ1 is 1 to 5, which is not specifically limited in this embodiment.

[0132] The Gaussian difference image D(u, v) is convolved with the frequency domain image F(u, v), that is, the frequency domain image F(u, v) is multiplied point by point with the Gaussian difference image D(u, v) to obtain the frequency domain defect image G(u, v). This can highlight the defect features in the image and achieve separation of defects from the background. The convolution operation formula is shown in formula (5).

[0133] G(u,v)=F(u,v)·D(u,v) Formula (5)

[0134] Where G(u,v) is the grayscale value corresponding to the pixel in the u-th row and v-th column in the frequency domain defect image.

[0135] Perform inverse Fourier transform on the frequency domain defect image G(u,v) to obtain the time domain image after frequency domain processing, that is, the time domain defect image K(x,y), where the inverse Fourier transform formula is shown in formula (6).

[0136]

[0137] In formula (6), M is the total number of horizontal pixels of the time domain defect image, N is the total number of vertical pixels of the time domain defect image, G(u, v) is the frequency domain defect image after the convolution operation, and K(x, y) is the grayscale value corresponding to the xth row and yth column in the time domain defect image.

[0138] The step of extracting the defect area from the small image includes extracting the defect area from the time-domain defect image. In this embodiment, a threshold segmentation method is used to extract the defect area from the time-domain defect image. Specifically, the threshold is selected as 0.5 times the maximum grayscale value of the Gaussian difference image D(x, y), but this is not limited to this. In other words, the portion of the time-domain defect image with a grayscale value greater than 0.5*Max(D(x, y)) is the defect area.

[0139] Step S105: performing fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area.

[0140] In a further optional embodiment of this embodiment, the step of performing fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area includes:

[0141] Select any point on the contour line of the defect area and mark it as point a;

[0142] Traverse the defect area to find the point farthest from point a, record it as point b, and record the distance S1 between point a and point b;

[0143] Traverse the defect area again to find the point farthest from point b, record it as point c, and record the distance S2 between point b and point c;

[0144] Determine whether S2 is equal to S1;

[0145] When S2 is equal to S1, the center of the minimum circumscribed circle is determined according to the perpendicular bisector on the ab segment and the perpendicular bisector on the bc segment, and the diameter of the minimum circumscribed circle is S2; among them, the ab segment is the segment formed by points a and b, and the bc segment is the segment formed by points b and c.

[0146] When S2 is not equal to S1, record S2 as S1, record point b as point a, and record point c as point b. Then traverse the defect area again to find the point farthest from point b, record it as point c, and record the distance S2 between point b and point c.

[0147] It can be understood that the contour line of the defect area is the boundary line of the defect area; when S2 is not equal to S1, recording S2 as S1 indicates that the size of S1 is updated, recording point b as point a indicates that the position of point a is updated, and recording point c as point b indicates that the position of point b is updated.

[0148] When S2 is equal to S1, the center of the minimum circumscribed circle is determined based on the perpendicular bisectors on the ab segment and the bc segment, specifically including:

[0149] There is an angle between the perpendicular bisector on the ab segment and the perpendicular bisector on the bc segment. That is, when point c does not coincide with point a, the intersection of the perpendicular bisector on the ab segment and the perpendicular bisector on the bc segment is the center of the minimum circumscribed circle.

[0150] When the perpendicular bisector on the ab segment completely coincides with the perpendicular bisector on the bc segment, that is, when point c coincides with point a, the midpoint on the ab segment or the bc segment is the center of the minimum circumscribed circle.

[0151] Step S106: judging whether the defect area is valid according to the diameter of the minimum circumscribed circle.

[0152] Specifically, the diameter threshold is set to U. When the minimum circumscribed circle diameter of the defect area is greater than U, the defect area is valid and is a foreign body area. Otherwise, it is not considered a foreign body defect. In this way, accurate defect extraction is achieved and the over-inspection problem caused by deep learning model detection is reduced.

[0153] The defect detection method provided in this embodiment can automatically identify and detect defects on abnormal images through deep learning models and convolution processing, and can quickly identify whether the size of the defect area meets the defect requirements based on the size of the minimum circumscribed circle corresponding to the defect area, thereby reducing the intensity of manual input, saving labor costs, reducing the occurrence of over-inspection and false detection, improving the efficiency of defect detection, reducing the rework rate of workpieces, and making the detection results traceable, effectively improving the stability and reliability of the detection results, and helping to unify defect detection standards and accurately measure the degree of defects.

[0154] Example 2

[0155] Based on Example 1, this embodiment provides a defect detection method, which includes steps S201 to S211.

[0156] Step S201: Collect abnormal pictures of abnormal workpieces.

[0157] Step S202: Divide the abnormal image into multiple abnormal sub-images.

[0158] Step S203: perform defect detection on the abnormal sub-image through a deep learning model, and output the coordinate information of the annotation box on the abnormal sub-image.

[0159] Step S204: cropping a small image containing the defective area according to the coordinate information.

[0160] Step S205: normalize the small image.

[0161] Step S206: Perform Fourier transform on the normalized small image to obtain a frequency domain image.

[0162] Step S207: performing a convolution operation on the frequency domain image using a preset Gaussian difference image in the frequency domain to obtain a frequency domain defect image.

[0163] Step S208: performing inverse Fourier transform on the frequency domain defect image to obtain a time domain defect image.

[0164] Step S209: extracting the defect area from the time-domain defect image.

[0165] Step S210: performing fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area.

[0166] Step S211: Determine whether the defect area is valid based on the diameter of the minimum circumscribed circle.

[0167] In addition, the method also includes the training method of the deep learning model, as follows:

[0168] Perform data augmentation on sample images.

[0169] The sample image after data enhancement processing is equally divided into multiple sample sub-images.

[0170] The deep learning model is trained using sample sub-images.

[0171] Example 3

[0172] Based on embodiments 1 and 2, this embodiment provides a defect detection device 300, see Figure 4, the defect detection device 300 includes:

[0173] The acquisition module 301 is configured to acquire abnormal images of abnormal workpieces;

[0174] The defect detection module 302 is configured to detect defects in abnormal images using a deep learning model and output coordinate information of a labeling box on the abnormal image, where the labeling box selects the defect area on the abnormal image;

[0175] The cropping module 303 is configured to crop a small image containing a defective area according to the coordinate information;

[0176] The convolution processing module 304 is configured to perform convolution processing on the small image and extract the defect area on the small image;

[0177] The fitting analysis module 305 is configured to perform fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area;

[0178] The judgment module 306 is configured to judge whether the defect area is valid according to the diameter of the minimum circumscribed circle.

[0179] In a further optional embodiment of this embodiment, the defect detection device 300 further includes:

[0180] The normalization module is configured to perform normalization processing on the small image before executing the convolution processing module 304.

[0181] In a further optional embodiment of this embodiment, the convolution processing module 304 includes:

[0182] A transform submodule is configured to perform Fourier transform on the small image to obtain a frequency domain image;

[0183] The operator module is configured to perform a convolution operation on the frequency domain image through a preset Gaussian difference image in the frequency domain to obtain a frequency domain defect image;

[0184] an inverse transform submodule, configured to perform an inverse Fourier transform on the frequency domain defect image to obtain a time domain defect image;

[0185] The extraction submodule is configured to extract the defect area from the time domain defect image.

[0186] In a further optional embodiment of this embodiment, the defect detection device 300 further includes:

[0187] a segmentation module configured to divide the abnormal image into a plurality of abnormal sub-images before executing the defect detection module 302;

[0188] The defect detection module 302 is further configured to perform defect detection on abnormal sub-images through a deep learning model and output coordinate information of the annotation box on the abnormal sub-image.

[0189] In a further optional embodiment of this embodiment, the fitting analysis module 305 includes:

[0190] The point selection submodule is configured to select any point on the contour line of the defect area, which is recorded as point a;

[0191] The first traversal submodule is configured to traverse the defect area to find the point farthest from point a, recorded as point b, and record the distance S1 between point a and point b;

[0192] The second traversal submodule is configured to traverse the defect area again to find the point farthest from point b, recorded as point c, and record the distance S2 between point b and point c;

[0193] A judgment submodule is configured to judge whether S2 is equal to S1;

[0194] The determination submodule is configured to determine the center of the minimum circumscribed circle according to the perpendicular bisector on the ab segment and the perpendicular bisector on the bc segment when S2 is equal to S1, and the diameter of the minimum circumscribed circle is S2;

[0195] The recording submodule is configured to record S2 as S1, point b as point a, point c as point b, and execute the second traversal submodule when S2 is not equal to S1.

[0196] In a further optional embodiment of this embodiment, the deep learning model is obtained by adding a detection head layer to the YOLO model;

[0197] The defect detection device 300 further includes:

[0198] A frame selection and annotation module is configured to use an image annotation tool to frame and annotate defect areas on the sample image before executing the defect detection module 302;

[0199] The input module is configured to input the sample image after box selection and annotation into the deep learning module to train the deep learning model.

[0200] In a further optional embodiment of this embodiment, the defect detection device 300 further includes:

[0201] a data enhancement module configured to perform data enhancement processing on the sample image before executing the box selection and annotation module, wherein the data enhancement processing includes at least one of random cropping, random scaling, random flipping, random rotation, contrast adjustment, and brightness adjustment;

[0202] The dividing module is configured to divide the sample image into multiple sample sub-images.

[0203] Example 4

[0204] This embodiment provides a defect detection system, combined with Figure 1 and Figure 2 The defect detection system includes a transparent conveyor belt 1, a photoelectric sensor 2, a back light source 3, a front ring light source 4, a lens 5 and a defect detection device; the lens 5 is electrically connected to the photoelectric sensor 2 and the defect detection device respectively, and the defect detection device is used to execute the defect detection method in Example 1 or Example 2.

[0205] Among them, the photoelectric sensor 2 is fixedly set on one side of the transparent conveyor belt 1, the back light source 3 is located below the transparent conveyor belt 1, the front ring light source 4 and the lens 5 are both located above the transparent conveyor belt 1, and the optical axis of the lens 5 coincides with the geometric center of the front ring light source 4 and the back light source 3 and is perpendicular to the transparent conveyor belt 1.

[0206] Example 5

[0207] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method of embodiment 1 or embodiment 2 are implemented.

[0208] Throughout this specification, references to terms such as "some embodiments," "other embodiments," and "desired embodiments" indicate that a particular feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present application. The schematic descriptions of these terms throughout this specification do not necessarily refer to the same embodiment or example.

[0209] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Collect abnormal pictures of abnormal workpieces; Perform defect detection on the abnormal image through a deep learning model, and output coordinate information of a marked box on the abnormal image, wherein the marked box selects a defect area on the abnormal image; Cut out a small picture containing the defect area according to the coordinate information; Performing convolution processing on the small image and extracting the defect area on the small image; Performing fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area; Whether the defect area is valid is determined according to the diameter of the minimum circumscribed circle.

2. The method according to claim 1, characterized in that Before the step of performing convolution processing on the small image, the method further includes: The small image is normalized.

3. The method according to claim 1, characterized in that The step of performing convolution processing on the small image comprises: Performing Fourier transform on the small picture to obtain a frequency domain image; Performing a convolution operation on the frequency domain image through a preset Gaussian difference image in the frequency domain to obtain a frequency domain defect image; Performing an inverse Fourier transform on the frequency domain defect image to obtain a time domain defect image; The step of extracting the defective area on the small picture comprises: The defect area is extracted from the time domain defect image.

4. The method according to claim 1, characterized in that Before the step of performing defect detection on the abnormal image by using a deep learning model, the method further includes: Divide the abnormal picture into a plurality of abnormal sub-pictures; The step of performing defect detection on the abnormal image through a deep learning model and outputting coordinate information of a marked box on the abnormal image comprises: Defect detection is performed on the abnormal sub-image through a deep learning model, and coordinate information of the annotation box on the abnormal sub-image is output.

5. The method according to claim 1, characterized in that The step of performing fitting analysis on the defect area to obtain the minimum circumscribed circle corresponding to the defect area comprises: Take any point on the contour line of the defect area and record it as point a; Traverse the defect area to find the point farthest from point a, record it as point b, and record the distance S1 between point a and point b; Traverse the defect area again to find the point farthest from point b, record it as point c, and record the distance S2 between point b and point c; Determine whether S2 is equal to S1; When S2 is equal to S1, the center of the minimum circumscribed circle is determined according to the perpendicular bisector on the ab segment and the perpendicular bisector on the bc segment, and the diameter of the minimum circumscribed circle is S2; When S2 is not equal to S1, S2 is recorded as S1, point b is recorded as point a, and point c is recorded as point b, and the step of traversing the defect area again is performed to find the point farthest from point b, recorded as point c, and recording the distance S2 between point b and point c.

6. The method according to claim 1, characterized in that The deep learning model is obtained by adding a layer of detection head to the YOLO model; Before performing defect detection on the abnormal image by using a deep learning model, the method further includes: Use image annotation tools to select and annotate defective areas on sample images; The sample image after box selection and annotation is input into the deep learning module to train the deep learning model.

7. The method according to claim 6, characterized in that Before the step of using an image annotation tool to select and annotate the defect area on the sample image, the method further includes: Performing data enhancement processing on the sample image, wherein the data enhancement processing includes at least one of random cropping, random scaling, random flipping, random rotation, contrast adjustment, and brightness adjustment; The sample image is equally divided into a plurality of sample sub-images.

8. A defect detection device, characterized in that: The device comprises: A collection module, configured to collect abnormal images of abnormal workpieces; A defect detection module is configured to perform defect detection on the abnormal image through a deep learning model, and output coordinate information of a marked box on the abnormal image, wherein the marked box selects a defect area on the abnormal image; A cropping module, configured to crop a small image containing the defect area according to the coordinate information; A convolution processing module, configured to perform convolution processing on the small image and extract the defect area on the small image; A fitting analysis module is configured to perform a fitting analysis on the defect area to obtain a minimum circumscribed circle corresponding to the defect area; The judging module is configured to judge whether the defect area is valid according to the diameter of the minimum circumscribed circle.

9. A defect detection system, characterized in that: It comprises a transparent conveyor belt, a photoelectric sensor, a back light source, a front ring light source, a lens and a defect detection device; the defect detection device is used to perform the defect detection method according to any one of claims 1 to 7; The lens is electrically connected to the photoelectric sensor and the defect detection equipment respectively. The photoelectric sensor is fixedly arranged on one side of the transparent conveyor belt. The back light source is located below the transparent conveyor belt. The front ring light source and the lens are both located above the transparent conveyor belt. The optical axis of the lens coincides with the geometric centers of the front ring light source and the back light source and is perpendicular to the transparent conveyor belt.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images