Tiny defect detection method

Abstract

The invention belongs to the technical field of intelligent manufacturing, and discloses a tiny defect detection method which comprises the following steps: acquiring a picture of a standard workpiece, finding a micropore position of the standard workpiece, intercepting a micropore picture for binarization processing, and generating a standard micropore binarization template; obtaining a picture of a to-be-detected micropore workpiece, and carrying out binarization processing to generate a to-be-detected micropore binarization picture; dividing the to-be-detected micropore binary image into a plurality of image blocks, cutting off unnecessary information of the to-be-detected micropore binary image, and predicting the cut image by using a neural network model to obtain defect category and position information; and according to the defect category and the position information, identifying whether the to-be-detected workpiece meets the production standard. The problem that the detection accuracy and the detection speed of the tiny defects are difficult to consider at the same time is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent manufacturing, and in particular relates to a method for detecting tiny defects. Background Art

[0002] With the continuous advancement of industrial intelligence, the application of visual inspection technology in manufacturing has become increasingly common. Smart factory construction involves a large number of visual inspection tasks, and industrial inspection targets are often complex and diverse. This is particularly true when inspecting for minor defects such as earphone and speaker jacks in mobile phones. Due to design, process, and equipment limitations, mobile phone midframes are prone to subtle burrs and other defects during processing. These defects are difficult for manual inspection systems to accurately detect, leading to misjudgments.

[0003] Computer vision and image processing technologies have been widely used in defect detection in recent years, particularly in automated production lines and quality control. Image binarization is a common method in image preprocessing. By converting an image into black and white, it simplifies subsequent processing and highlights key features. Traditional binarization algorithms convert images into binary images using a fixed threshold. However, this method is sensitive to image quality, lighting, and noise, and cannot handle image transformations in complex environments.

[0004] To address this issue, deep learning-based image processing methods have emerged in recent years, particularly convolutional neural networks (CNNs) and object detection algorithms (such as YOLO). These methods can automatically extract image features and perform efficient defect recognition at different scales. By training deep neural network models, various defects, including tiny ones, can be automatically identified and located in images.

[0005] Although existing inspection methods based on image processing and deep learning have improved detection accuracy to a certain extent, they still face the following challenges in detecting small defects:

[0006] (1) Stability problem of binarization algorithm: The traditional binarization method cannot adapt to the complex surface conditions of the workpiece and the image changes under different lighting conditions through simple threshold processing, which easily leads to the neglect of minor defects in the image.

[0007] (2) Image noise and interference: In the actual production process, the workpiece surface is often disturbed by factors such as environmental noise, lighting changes, dust and reflections. These factors may affect the image quality and cause defects to be unable to be accurately identified.

[0008] (3) Difficulty in identifying tiny defects: Tiny defects are often small in size and irregular in shape, and are easily masked by surface texture, noise, and other factors. Existing technologies often have high recognition accuracy for larger defects (such as cracks and holes), but the detection accuracy for tiny defects is still low.

[0009] (4) Computational complexity and real-time issues: Although deep learning-based detection methods can provide higher accuracy, processing speed and real-time performance are still important issues in large-scale production environments. Especially in production lines that need to process a large number of workpieces quickly, deep learning models with large computational loads may affect the overall detection efficiency.

[0010] (5) Difficulty in obtaining labeled data: The training of deep learning models requires a large amount of labeled data, but the labeling of minor defects usually needs to be done manually, and the diversity and comprehensiveness of the samples are insufficient, resulting in insufficient training data and affecting the generalization ability of the model. Summary of the Invention

[0011] Based on this, the present invention provides a method for detecting tiny defects to overcome the problem that it is difficult to strike a balance between the accuracy and speed of tiny defect detection.

[0012] The method for detecting micro defects provided by the present invention comprises the following steps:

[0013] S1 obtains a picture of a standard workpiece, finds the position of the microholes in the standard workpiece, and intercepts the microhole picture and records it as picture M;

[0014] S2 performs binarization processing on the image M to generate a standard microwell binarization template, which is recorded as template M ′ , and further calculate the template M ′ The average value of the pixel value PM ′ ;

[0015] S3 obtains a picture of the micro-hole workpiece to be inspected, which is recorded as picture A;

[0016] S4 performs binarization processing on the image A to generate a binary image A of the micropore to be inspected ′ ;

[0017] S5 will take picture A ′ Divided into several image blocks A ′ i,j , and further calculate each image block A ′ i,j The average value of the pixel value PA ′ i,j ; Where i, j represents the image block in row i and column j;

[0018] S6 will obtain PA ′ i,j Respectively with PM ′ Subtraction, that is, SA ′ n =PA ′ i,j ―PM′ , find SA ′ n Find the three image patches with the smallest values in it, denoted as SA ′ 1. SA ′ 2. SA ′ 3;

[0019] S7 will subtract the three found image patches SA ′ 1. SA ′ 2. SA ′ 3 from the standard micropore binarization template M ′ by subtracting and summing pixel by pixel, that is S n = sum(SA ′ n − M ′ ), and record the calculation results as S1, S2, S3;

[0020] S8 selects the position information of the calculation block with the smallest S n value among S1, S2, S3 as the cropping basis of picture A, crops all the pixels around the cropping basis, and enlarges the resolution of the cropped image;

[0021] S9 uses a neural network model to predict the cropped and enlarged image to obtain the defect category and position information;

[0022] S10 identifies whether the workpiece to be inspected meets the production standard according to the defect category and position information.

[0023] Furthermore, in step S1, obtain the picture of the standard workpiece, find the micropore position of the standard workpiece, and intercept the micropore picture, denoted as picture M; specifically:

[0024] S11 obtains the picture of the standard workpiece, and the resolution size of the picture of the standard workpiece is H×W, where H is the number of pixels in the row direction of the picture and W is the number of pixels in the column direction of the picture;

[0025] S12 finds the micropore position of the standard workpiece,

[0026] When H < W, intercept the micropore picture with the resolution size of and denote it as picture M;

[0027] When H > W, intercept the micropore picture with the resolution size of and denote it as picture M;

[0028] where α is a positive integer and is the resolution adjustment coefficient of picture M.

[0029] Furthermore, in step S2, binarize picture M to generate a standard micropore binarization template, denoted as template M′ ; Specifically:

[0030] Use the binarization algorithm, take β% of the average pixel value of picture M as the threshold, and binarize picture M to generate the standard micropore binarization template M ′ ; where β is a positive integer and is the threshold adjustment coefficient for binarization processing.

[0031] Furthermore, in step S4, binarize picture A to generate the to-be-inspected micropore binarization picture A ′ ; Specifically:

[0032] Use the binarization algorithm, take β% of the average pixel value of picture A as the threshold, and binarize picture A to generate the to-be-inspected micropore binarization picture A ′ ; where β is a positive integer and is the threshold adjustment coefficient for binarization processing.

[0033] Furthermore, in step S5, divide picture A ′ into several image blocks A ′ i,j ; Specifically:

[0034] When H < W, divide picture A ′ in accordance with the size of, with a step size of (8·γ) pixel points in both the row and column directions, and use the convolutional sliding window method to divide the image into several image blocks A ′ i,j ;

[0035] When H > W, divide picture A ′ in accordance with the size of, with a step size of (8·γ) pixel points in both the row and column directions, and use the convolutional sliding window method to divide the image into several image blocks A ′ i,j ;

[0036] where γ is a positive integer and is the step size adjustment coefficient of the sliding window.

[0037] Furthermore, in step S8, expand the resolution of the cropped image; specifically:

[0038] Use the linear interpolation method to expand the resolution of the cropped image by λ times according to the actual production requirements to obtain picture P; where λ is a positive integer and is the resolution adjustment coefficient of picture P.

[0039] Furthermore, in step S9, the neural network model is a micropore processing defect detection model based on YOLOv8, including a backbone unit, a neck unit, and a detection head unit connected in sequence;

[0040] The backbone unit includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module, and an SPPF module, which are connected in sequence.

[0041] The neck unit includes a first upsampling layer, a first splicing operation layer, a first GCVIT module, a second upsampling layer, a second splicing operation layer, a second GCVIT module, a sixth convolutional layer, a third splicing operation layer, a third GCVIT module, a seventh convolutional layer, a fourth splicing operation layer, and a fifth C2f module, wherein the input of the third splicing operation layer is also connected to the output of the first GCVIT module;

[0042] The trunk unit is connected to the second splicing operation layer, the first splicing operation layer and the fourth splicing operation layer of the neck unit through the output of the second C2f module, the output of the third C2f module and the output of the SPPF module respectively; the neck unit is connected to the detection head unit through the output of the second GCVIT module, the output of the third GCVIT module and the output of the C2f module.

[0043] Furthermore, the structures of the first GCVIT module, the second GCVIT module, and the third GCVIT module are the same. Each GCVIT module includes an eleventh convolutional layer, a twelfth convolutional layer, an improved multi-head self-attention mechanism layer, a thirteenth convolutional layer, an eleventh splicing operation layer, and a fourteenth convolutional layer connected in sequence. External data are respectively input into the eleventh convolutional layer and the eleventh splicing operation layer of the GCVIT module, and the GCVIT module outputs the result of the fourteenth convolutional layer.

[0044] Furthermore, the improved multi-head self-attention mechanism layer includes a full convolution part and a self-attention score calculation part connected in sequence, specifically:

[0045] Use a 3×3 convolution kernel to split the feature map of the input data into three feature maps of equal size: Q', K', and V';

[0046] According to the channel arrangement rule of the feature map, the three feature maps Q', K', and V' are expanded into three feature matrices Q, K, and V, and the self-attention score is calculated according to the following formula:

[0047]

[0048] Among them, X in is the input data, p self is the output data of the self-attention score calculation part, g 3―i(x) represents the operation process of full group convolution with a convolution kernel size of 3×3 and expanded into the feature matrix i by channel, D OUT is the scaling factor;

[0049] Fold the calculated feature matrix back into the feature map and output it:

[0050] p A =g 3―V (X in )×p self

[0051] Among them, p A Represents the output data of the improved multi-head self-attention mechanism layer.

[0052] Furthermore, in step S9, the method for constructing the training data set of the neural network model is:

[0053] Collect a collection of images of minor defects;

[0054] Use a rectangular box to mark the defect location of each image, obtain the defect category, rectangular box width and height, and the coordinates of the rectangular box center point in the image, and save them as a label file;

[0055] Obtain the label file corresponding to each image and establish a database of minor defects.

[0056] The beneficial effects of the present invention are:

[0057] (1) The present invention utilizes a cropping algorithm based on rapid binary template matching to remove redundant background elements from the captured image, thereby enabling the deep learning model to focus more closely on the location of product defects. Unlike traditional segmentation algorithms, the cropping algorithm based on rapid binary template matching is faster and has more accurate boundaries. Compared to traditional template matching algorithms, the rapid binary template matching cropping algorithm of the present invention uses only a small number of multiplication operations to complete the micropore positioning and cropping of defective products, boasting extremely high computational efficiency and better suited for industrial inspection environments.

[0058] (2) The present invention uses the traditional convolution module to improve the Q, K, and V matrix generation process of the multi-head attention mechanism in the Transform feature extraction module. While retaining the feature extraction capability of the original Transform module, it greatly reduces the amount of its calculation parameters and speeds up the detection system's recognition rate of micro-hole processing defect targets.

[0059] (3) The YOLOv8n-GCViT model and the YOLOv8n-Transform model of the present invention are both stronger than the original YOLOv8n model in the ability to extract features of small random targets. In addition, the YOLOv8n-GCViT model of the present invention has only half the number of parameters of the YOLOv8n-Transform model, and can complete the defect recognition task more quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Flowchart of a method for detecting minute defects provided by an embodiment of the present invention.

[0061] Figure 2 A schematic diagram of the cropping algorithm flow provided by an embodiment of the present invention.

[0062] Figure 3 This is a structural diagram of the micro-hole machining defect detection model based on YOLOv8 provided in an embodiment of the present invention.

[0063] Figure 4 This is a structural diagram of the GCViT module provided in an embodiment of the present invention.

[0064] Figure 5 Schematic diagram of the improved multi-head self-attention mechanism layer provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the accompanying drawings. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present disclosure.

[0066] The method for detecting minor defects provided by this invention comprises two major steps: image cropping and model testing. Image cropping is used to remove unnecessary information. Model testing utilizes the YOLOv8-based micro-hole machining defect detection model (YOLOv8n-GCViT model) proposed in this invention to detect and determine whether defects exist at the micro-hole machining locations on the product and output the corresponding detection results.

[0067] The principle of the image cropping method is as follows: By taking advantage of the low reflectivity at the micropore positions, the binarization algorithm is used to cut out the positions of the micropores to be inspected. The main steps include binarization template generation, generation of the binarized feature map of the defective product, sliding window operation, and positive-negative sample difference matching operation. Among them, the sliding window operation uses the average value of small-size pixel blocks and the average value of the pixel template on the binarized feature map of the defective product to quickly locate the approximate positions of the micropores in the defective product image and select the three most likely slider positions. The positive-negative sample difference matching operation uses the difference between the positive and negative samples to determine the position most likely to be a micropore.

[0068] The principle of the model detection method is as follows: According to the working principle of the Transform feature extraction module, the redundant structures in the module are removed, and the convolution operation (Conv) is used to replace the linear operation (Linear) in the multi-head attention structure. The GCVIT feature extraction module is designed for use in the neural network, creating a micropore processing defect detection model based on YOLOv8 (YOLOv8n-GCViT model). Without significantly affecting the performance of the feature extraction module, the network computational parameter quantity is greatly reduced. Finally, the cropped image is used as the input of the neural network for prediction, thus achieving rapid positioning of the defect position.

[0069] It should be noted that the product of the present invention refers to the workpiece. In the embodiment, the detection of the micropore processing defects of the product is taken as an example, but the concept of the present invention is applicable to the detection of various product micro defects.

[0070] As Figure 1 shown, the method for detecting micro defects provided by the embodiment of the present invention includes the following steps:

[0071] S1 Obtain the picture of the standard workpiece, find the micropore position of the standard workpiece, and intercept the micropore picture and record it as picture M. Specifically:

[0072] S11 Obtain the picture of the standard workpiece, and the resolution size of the picture of the standard workpiece is H×W, where H is the number of pixels in the row direction of the picture and W is the number of pixels in the column direction of the picture.

[0073] In this embodiment, a grayscale picture of a standard workpiece with a size of 3072×2048 is obtained from the vision detection platform. S12 Find the micropore position of the standard workpiece.

[0074] When H < W, intercept the micropore picture with a size of and record it as picture M;

[0075] When H > W, intercept the micropore picture with a size of and record it as picture M;

[0076] Where α is a positive integer, which is the resolution adjustment coefficient of the image M.

[0077] That is, the smaller value of H and W is selected to capture the micropore image. In this embodiment, 2048 is selected.

[0078] S2 performs binarization processing on the image M to generate a standard microwell binarization template, which is recorded as template M ′ , and further calculate the template M ′ The average value of the pixel value PM ′ .

[0079] Specifically, the binarization algorithm is used to take the β% of the average value of the pixel value of the image M as the threshold, and the image M is binarized to generate a standard micropore binarization template M. ′ ; Where β is a positive integer, which is the threshold adjustment coefficient of the binarization processing.

[0080] In this embodiment, β is selected as 50, that is, based on 50% of the average brightness of the image pixels, a standard micro-hole binary template of size 128×128 containing only two pixel values ​​of 0 and 255 is drawn (where 0 represents the position where the pixel value in the original image is lower than 50% of the average brightness).

[0081] S3 obtains a picture of the micro-hole workpiece to be inspected, which is recorded as picture A.

[0082] S4 performs binarization processing on the image A to generate a binary image A of the micropore to be inspected ′ .

[0083] Specifically, use the binarization algorithm, take β% of the average value of the pixel value of image A as the threshold, binarize image A, and generate the binary image A of the micropore to be inspected. ′ ; Where β is a positive integer, which is the threshold adjustment coefficient of the binarization processing.

[0084] In this embodiment, β is selected as 50, that is, based on 50% of the average brightness of the image pixels, the pixel values ​​in the image that are 50% lower than the average value are recorded, and the recorded pixel positions are drawn in a 3072×2048 binary image to form a binary information image channel.

[0085] S5 will take picture A ′ Divided into several image blocks A ′ i,j , and further calculate each image block A ′ i,j The average value of the pixel value PA ′ i,j ; where i, j represents the image block in row i and column j. Specifically:

[0086] When H < W, the picture A ′ According to the size, in the row and column directions respectively with the number of pixel points of (8·γ) as the step length, and adopting the convolutional sliding window method, the image is divided into several image blocks A ′ i,j ;

[0087] When H > W, the picture A ′ According to the size, in the row and column directions respectively with the number of pixel points of (8·γ) as the step length, and adopting the convolutional sliding window method, the image is divided into several image blocks A ′ i,j ;

[0088] where γ is a positive integer and is the step length adjustment coefficient of the sliding window.

[0089] In this embodiment, γ is selected as 4, that is: with 128×128 as the calculation block size and 32 pixel values as the step length, calculate the average value of each calculation block in the binarized information image channel of the size of 3072×2048 in the previous step in the form of a sliding window.

[0090] S6 Subtract the obtained PA ′ i,j from PM ′ respectively, that is SA ′ n = PA ′ i,j - PM ′ , find the three image blocks with the smallest values in SA ′ n , denoted as SA ′ 1, SA ′ 2, SA ′ 3.

[0091] S7 Subtract and sum the three found image blocks SA ′ 1, SA ′ 2, SA ′ 3 from the standard micro-hole binarization template M ′ pixel by pixel, that is S n = sum(SA ′ n - M ′ ), and record the calculation results as S1, S2, S3.

[0092] S8 Select the position information of the calculation block with the smallest S n value among S1, S2, S3 as the cropping basis for picture A, crop all the surrounding pixels of the cropping basis, and enlarge the resolution of the cropped image.

[0093] Specifically: use linear interpolation to expand the resolution of the cropped image by λ times according to actual production requirements to obtain image P; where λ is a positive integer, which is the resolution adjustment coefficient of image P.

[0094] In this embodiment, the image is enlarged to 256×256 size using linear interpolation method, and the cropping process is as follows: Figure 2 shown.

[0095] S9 uses a neural network model to predict the cropped and enlarged image to obtain the defect category m and location information n.

[0096] Defect category m represents the total number of defects identified in image P. Location information n represents the location of the identified defects in image P, typically expressed as coordinates. If no defects are detected in image P, location information n is empty. That is, if m = 0, then n = null.

[0097] It should be noted that the neural network model here can be a YOLOv8 model, a YOLOv8n-Transform model, or other models. This embodiment is described using the YOLOv8n-GCViT model proposed in the present invention as an example.

[0098] In this embodiment, the neural network model adopts a micro-hole processing defect detection model based on YOLOv8, namely the YOLOv8n-GCViT model, which includes a trunk unit, a neck unit and a detection head unit connected in sequence, such as Figure 3 As shown, the trunk unit is used for image feature extraction, the neck unit is used for image feature fusion, and the detection head unit is used for regression calculation of defect category and defect location.

[0099] The YOLOv8n-GCViT model can be expressed as described in formula (1):

[0100] F=F 检测头单元 (F 颈部单元 (F 主干单元 (F))) (1)

[0101] Among them, F is the model output, F 检测头单元 Indicates the calculation process of the detection head unit, F 颈部单元 Indicates the calculation process of the neck unit, F 主干单元 Represents the calculation process of the backbone unit, and X is the input image.

[0102] Please refer to Figure 3 ,The backbone unit and the detection head unit are the contents of the original YOLOv8 algorithm, and the neck unit is the improved algorithm content of the present invention.

[0103] The backbone unit includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module, and an SPPF module, which are connected in sequence.

[0104] The neck unit includes a first upsampling layer, a first splicing operation layer, a first GCVIT module, a second upsampling layer, a second splicing operation layer, a second GCVIT module, a sixth convolutional layer, a third splicing operation layer, a third GCVIT module, a seventh convolutional layer, a fourth splicing operation layer, and a fifth C2f module, wherein the input of the third splicing operation layer is also connected to the output of the first GCVIT module;

[0105] The trunk unit is connected to the second splicing operation layer, the first splicing operation layer and the fourth splicing operation layer of the neck unit through the output of the second C2f module, the output of the third C2f module and the output of the SPPF module respectively; the neck unit is connected to the detection head unit through the output of the second GCVIT module, the output of the third GCVIT module and the output of the C2f module.

[0106] The calculation process of the backbone unit can be expressed by formula group (2):

[0107]

[0108] in:

[0109] f 主干―i is the i-th output of the neck unit and f 主干―1 、f 主干―2 、f 主干―3 The three results are connected to the neck unit as the output of the trunk unit;

[0110] X is the input image;

[0111] f3 represents the convolution operation with a convolution kernel size of 3×3;

[0112] f C2f Indicates the calculation process of the C2f module. The C2f module has the function of extracting image features. It is composed of multiple convolution operations combined according to certain rules. It is one of the components of the YOLO algorithm and is an existing calculation method.

[0113] f SPPF The SPPF module represents the calculation process of the SPPF module. The SPPF module has the function of enhancing the feature information of the image. It is a spatial pyramid pooling operation composed of multiple convolution operations and pooling operations according to certain rules. It is an existing algorithm.

[0114] The calculation process of the neck unit can be expressed by formula group (3):

[0115]

[0116] in:

[0117] f 颈部―i is the i-th output of the neck unit, and f 颈部―2 、f 颈部―3 、f 颈部―4 The three results are connected to the detection head unit as the output of the neck unit;

[0118] f GCVIT Represents the calculation process of the GCVIT module; f3 represents the convolution operation with a convolution kernel size of 3×3;

[0119] Concat represents the concatenation operation;

[0120] UpSample indicates an upsampling operation.

[0121] The first splicing operation layer, the second splicing operation layer, the third splicing operation layer, and the fourth splicing operation layer in the neck unit have the same form, and are all "Concat" splicing operations, which can combine the two inputs input to the splicing layer in the channel direction and then output them.

[0122] Concat stitching is an existing stitching method: it can stitch images of the same resolution along the channel direction. Explanation: Assume that the normal color image format size is 3×640×640, where 640×640 represents the resolution size and 3 represents the three color channels R, G, and B. After two images are input into Concat stitching, the output image format size is 6×640×640.

[0123] The first upsampling operation layer and the second upsampling operation layer in the neck unit are both "UpSample" operations, which can adjust the resolution size of the input information according to the output requirements. UpSample is a prior art.

[0124] The first GCVIT module, the second GCVIT module, and the third GCVIT module in the neck unit have the same structure. Figure 4 As shown in the figure, each GCVIT module includes the eleventh convolutional layer, the twelfth convolutional layer, the improved multi-head self-attention mechanism layer, the thirteenth convolutional layer, the eleventh splicing operation layer, and the fourteenth convolutional layer connected in sequence. External data are input into the eleventh convolutional layer and the eleventh splicing operation layer of the GCVIT module respectively, and the GCVIT module outputs the result of the fourteenth convolutional layer.

[0125] The mathematical expression of the GCVIT module is shown in formula (4):

[0126] f GCVIT =X+f3(f1(pA (f1(f3(X))))) (4)

[0127] in:

[0128] f GCVIT To output the picture;

[0129] X is the input image;

[0130] f1 represents the convolution operation with a convolution kernel size of 1×1;

[0131] f3 represents the convolution operation with a convolution kernel size of 3×3;

[0132] p A Represents a GCVIT operation.

[0133] The splicing operation in the GCVIT module is the "ADD" splicing operation, which can combine two inputs of the same format to the splicing layer in the resolution direction and output them to ensure that the output format maintains the original input format.

[0134] ADD stitching is an existing stitching method: images with the same resolution can be stitched together in an additive manner along the resolution direction. Explanation: Assuming a normal color image format size of 3×640×640, where 640×640 represents the resolution size and 3 represents the three color channels R, G, and B, after two images are input into ADD stitching, the output image format size is still 3×640×640, but the value of each pixel in the output image is the sum of the pixel values ​​at the corresponding position in the original two input images.

[0135] like Figure 5 As shown, Figure 4 The improved multi-head self-attention mechanism layer includes a fully convolutional part and a self-attention score calculation part connected in sequence, specifically:

[0136] Use a 3×3 convolution kernel to split the feature map of the input data into three feature maps of equal size: Q', K', and V';

[0137] According to the channel arrangement rule of the feature map, the three feature maps Q', K', and V' are expanded into three feature matrices Q, K, and V, and the self-attention score is calculated according to the following formula:

[0138]

[0139] Among them, X in is the input data, p self is the output data of the self-attention score calculation part, g 3―i (x) represents the operation process of full group convolution with a convolution kernel size of 3×3 and expanded into the feature matrix i by channel, D OUTis the scaling factor;

[0140] Fold the calculated feature matrix back into the feature map and output it:

[0141] p A =g 3―V (X in )×p self

[0142] Among them, p A Represents the output data of the improved multi-head self-attention mechanism layer.

[0143] The training data set of the neural network model is constructed as follows:

[0144] Collect a collection of images of minor defects;

[0145] Use a rectangular box to mark the defect location of each image, obtain the defect category, rectangular box width and height, and the coordinates of the rectangular box center point in the image, and save them as a label file in ".txt" format;

[0146] Obtain the label file corresponding to each image and establish a database of minor defects.

[0147] In this embodiment, the feature maps of 20×20, 40×40 and 80×80 dimensions generated by the trunk unit are used as the input of the neck unit. After two upsampling (Upsample) and GCVIT modules, feature maps of 80×80, 40×40 and 20×20 dimensions are obtained. The obtained 80×80 feature map is then subjected to convolution downsampling (Conv), GCVIT module, convolution downsampling (Conv), C2f module and concatenation and fusion operation (Concat) with the previously obtained feature map, and then input into the feature regression calculation module (Detect), thus ending the target detection link.

[0148] S10 identifies whether the workpiece to be inspected meets the production standards based on the defect category and location information.

[0149] Usually the production inspection standard is: m 标准 =0 and n 标准 =null.

[0150] In steps S3 to S10, this embodiment also uses data enhancement methods (including operations such as rotation, scaling, flipping, and color jittering) to enhance the diversity of training data, thereby improving the robustness and detection accuracy of the detection model.

[0151] Experimental design:

[0152] The algorithm dataset used 1,400 images of defects in the micro-frame micro-hole processing. The dataset was divided into a training set, a test set, and a validation set in an 8:1:1 ratio. The training dataset included images of micro-hole processing burrs, micro-hole processing cap defects, and micro-hole processing chip defects. The images in the training set were trained using the YOLOv8n model (original model), the YOLOv8n-Transform model (original model with the Transform module added), and the YOLOv8n-GCViT model (the model provided by the present invention). The training weights were imported into the network structure to test the test set.

[0153] The performance comparison of each network model is shown in the experimental results in the following table:

[0154]

[0155]

[0156] As can be seen from the above table, compared with the YOLOv8n model (original model), the YOLOv8n-GCViT model (the model provided by the present invention) has a significant improvement in the recognition accuracy of tiny burr defects, and the number of parameters has a great advantage over the YOLOv8n-Transform model.

[0157] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The above embodiments merely represent preferred embodiments of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for detecting micro defects, characterized in that: The following steps are involved: S1 obtains a picture of a standard workpiece, finds the position of the microholes of the standard workpiece, and captures the microhole picture as picture M; S2 performs binarization processing on the image M to generate a standard microwell binarization template, which is recorded as template M ′ , and further calculate the template M ′ The average value of the pixel value PM ′ ; S3 obtains a picture of the micro-hole workpiece to be inspected, which is recorded as picture A; S4 performs binarization processing on the image A to generate a binarized image A of the micropore to be inspected ′ ; S5 will picture A ′ Divided into several image blocks A ′ i,j , and further calculate each image block A ′ i,j The average value of the pixel value PA ′ i,j ; Where i, j represents the image block in the i-th row and j-th column; S6 will obtain PA ′ i,j Respectively with PM ′ Subtraction, that is, SA ′ n =PA ′ i,j ―PM ′ , find SA ′ n The three image blocks with the smallest values ​​are denoted as SA ′ 1. SA ′ 2. SA ′ 3; S7 will find the three image blocks SA ′ 1. SA ′ 2. SA ′ 3. Subtract the pixel values ​​one by one from the standard micropore binary template M ′ Subtract and sum, that is, S n =sum(SA ′ n ―M ′ ), and record the calculation results as S1, S2, and S3; S8 selects S from S1, S2, S3 n The calculated block position information with the smallest value is used as the cropping basis of the image A, all pixels around the cropping basis are cropped, and the resolution of the cropped image is enlarged; S9 uses a neural network model to predict the cropped and enlarged image to obtain defect category and location information; S10 identifies whether the workpiece to be inspected meets the production standard based on the defect category and location information.

2. The method for detecting micro defects according to claim 1, characterized in that: Step S1 obtains a picture of a standard workpiece, finds the position of the microholes of the standard workpiece, and captures the microhole picture as picture M; specifically: S11 acquires a picture of a standard workpiece, wherein the picture resolution of the standard workpiece is H×W, where H is the number of pixels in the row direction of the picture, and W is the number of pixels in the column direction of the picture; S12 finds the micro-hole position of the standard workpiece, When H < W, the microporous image is intercepted in the size with the resolution size of and denoted as image M; When H>W, the resolution is The micropore image is intercepted with the size of , and recorded as image M; Where α is a positive integer, which is the resolution adjustment coefficient of the image M.

3. The method for detecting micro defects according to claim 1, characterized in that: In step S2, the image M is binarized to generate a standard microwell binarization template, which is denoted as template M. ′ ; Specifically: Using the binarization algorithm, take β% of the average value of the pixel value of the image M as the threshold, binarize the image M, and generate a standard micropore binarization template M ′ ; Where β is a positive integer, which is the threshold adjustment coefficient for binarization processing.

4. The method for detecting micro defects according to claim 1, characterized in that: In step S4, the image A is binarized to generate a binary image A of the micropores to be inspected. ′ ; Specifically: Use the binarization algorithm, take β% of the average pixel value of image A as the threshold, binarize image A, and generate the binary image A of the micropore to be inspected. ′ ; Where β is a positive integer, which is the threshold adjustment coefficient for binarization processing.

5. The method for detecting micro defects according to claim 2, characterized in that: In step S5, picture A ′ Divided into several image blocks A ′ i,j ; Specifically: When H < W, picture A ′ is divided into a number of image patches A in both the row and column directions with a step size of (8·γ) pixel points according to the size of by means of a convolutional sliding window ′ i,j ; When H>W, image A ′ according to The size of the image is , and the number of pixels in the row and column directions is (8·γ) as the step size. The convolution sliding window method is used to divide the image into several image blocks A ′ i,j ; Where γ is a positive integer, which is the step size adjustment coefficient of the sliding window.

6. The method for detecting micro defects according to claim 1, characterized in that: In step S8, the resolution of the cropped image is enlarged; specifically: Use linear interpolation to enlarge the resolution of the cropped image by λ times according to actual production requirements to obtain image P; Where λ is a positive integer, which is the resolution adjustment coefficient of the image P.

7. The method for detecting micro defects according to claim 1, characterized in that: In step S9, the neural network model is a micro-hole processing defect detection model based on YOLOv8, including a trunk unit, a neck unit and a detection head unit connected in sequence; The backbone unit includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module, and an SPPF module, which are connected in sequence. The neck unit includes a first upsampling layer, a first splicing operation layer, a first GCVIT module, a second upsampling layer, a second splicing operation layer, a second GCVIT module, a sixth convolutional layer, a third splicing operation layer, a third GCVIT module, a seventh convolutional layer, a fourth splicing operation layer, and a fifth C2f module, wherein the input of the third splicing operation layer is also connected to the output of the first GCVIT module; The trunk unit is connected to the second splicing operation layer, the first splicing operation layer and the fourth splicing operation layer of the neck unit through the output of the second C2f module, the output of the third C2f module and the output of the SPPF module respectively; the neck unit is connected to the detection head unit through the output of the second GCVIT module, the output of the third GCVIT module and the output of the C2f module.

8. The method for detecting micro defects according to claim 7, characterized in that: The structures of the first GCVIT module, the second GCVIT module and the third GCVIT module are the same. Each GCVIT module includes an eleventh convolutional layer, a twelfth convolutional layer, an improved multi-head self-attention mechanism layer, a thirteenth convolutional layer, an eleventh splicing operation layer and a fourteenth convolutional layer connected in sequence. External data are respectively input into the eleventh convolutional layer and the eleventh splicing operation layer of the GCVIT module, and the GCVIT module outputs the result of the fourteenth convolutional layer.

9. The method for detecting micro defects according to claim 8, characterized in that: The improved multi-head self-attention mechanism layer includes a full convolution part and a self-attention score calculation part connected in sequence, specifically: Use a 3×3 convolution kernel to split the feature map of the input data into three feature maps of equal size: Q', K', and V'; According to the channel arrangement rule of the feature map, the three feature maps Q', K', and V' are expanded into three feature matrices Q, K, and V, and the self-attention score is calculated according to the following formula: Among them, X in is the input data, p self is the output data of the self-attention score calculation part, g 3―i (x) represents the operation process of full group convolution with a convolution kernel size of 3×3 and expanding it into feature matrix i by channel, D OUT is the scaling factor; Fold the calculated feature matrix back into the feature map and output it: p A =g 3―V (X in )×p self Among them, p A Represents the output data of the improved multi-head self-attention mechanism layer.

10. The method for detecting micro defects according to claim 1, characterized in that: In step S9, the method for constructing the training data set of the neural network model is: Collect a collection of images of minor defects; Use a rectangular frame to mark the defect position of each image, obtain the defect category, rectangular frame width and height, and the position coordinates of the center point of the rectangular frame in the image, and save them as a label file; Obtain the label file corresponding to each image and establish a database of minor defects.

Images