Method for detecting minute defects

Abstract

The application belongs to the technical field of intelligent manufacturing, and discloses a micro defect detection method, which comprises the following steps: obtaining a picture of a standard workpiece, finding the position of a micro hole of the standard workpiece, intercepting the micro hole picture for binaryzation processing, and generating a standard micro hole binaryzation template; obtaining a picture of a workpiece to be detected, performing binaryzation processing, and generating a binaryzation picture of the micro hole to be detected; dividing the binaryzation picture of the micro hole to be detected into a plurality of image blocks, cutting off unnecessary information of the binaryzation picture of the micro hole to be detected, using a neural network model to predict the cut image, and obtaining defect categories and position information; and identifying whether the workpiece to be detected meets the production standard according to the defect categories and position information. The application overcomes the problem that the micro defect detection accuracy and detection speed are difficult to be considered simultaneously.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent manufacturing, and particularly relates to a micro defect detection method. BACKGROUND

[0002] With the continuous progress of industrial intelligence, the application of visual detection technology in manufacturing industry becomes increasingly common. There are a large number of visual detection tasks in the construction of intelligent factories, and the industrial detection targets are often complex and diverse. In particular, in the detection process of micro defects such as mobile phone earphone holes and loudspeaker holes, due to the limitations of design, process and equipment, fine burrs and other defects are easily generated in the processing of mobile phone middle frames. These defects are difficult to be accurately captured by manual detection systems, which can easily lead to misjudgment.

[0003] In recent years, computer vision and image processing technology has been widely applied in the field of defect detection, especially in automated production lines and quality control. Image binarization processing is a common method in image preprocessing. By converting the image into a black and white image, the subsequent processing process can be simplified, and the key features in the image can be highlighted. Traditional binarization algorithms convert images into binary images by using fixed thresholds, but this method is sensitive to image quality, lighting and noise, and cannot handle image transformation in complex environments.

[0004] In order to solve this problem, in recent years, image processing methods based on deep learning have emerged, especially convolutional neural networks (CNN) and object detection algorithms (such as YOLO). These methods can automatically extract image features and perform efficient defect recognition at different scales. By training a deep neural network model, various defects, including micro defects, can be automatically recognized and located in images.

[0005] Although existing image processing and deep learning-based detection methods have improved detection accuracy to some extent, they still face the following challenges in micro defect detection:

[0006] (1) Stability problem of binarization algorithm: Traditional binarization methods use simple thresholding processing, which cannot adapt to complex workpiece surface conditions and image changes under different lighting conditions, and can easily cause micro defects in the image to be ignored.

[0007] (2) Image noise and interference: In actual production process, workpiece surface is often disturbed by environmental noise, lighting changes, dust and reflection, etc. These factors can affect image quality and lead to inaccurate recognition of defects.

[0008] (3) Difficulty in recognizing micro defects: Micro defects are often small in size, irregular in shape, and easily covered by surface texture, noise and other factors. Existing technologies often have high recognition accuracy for larger defects (such as cracks, holes, etc.), but the detection accuracy for micro defects is still low.

[0009] (4)Computational complexity and real-time issues: Although deep learning-based detection methods can provide higher accuracy, in large-scale production environments, processing speed and real-time performance remain an important issue, especially on production lines that require rapid processing of large quantities of workpieces, the large computational load of deep learning models may affect overall detection efficiency.

[0010] (5)Difficulty in obtaining labeled data: The training of deep learning models requires a large amount of labeled data, but the labeling of micro-defects usually requires manual work, and the diversity and comprehensiveness of samples are insufficient, leading to insufficient training data and affecting the generalization ability of the model. SUMMARY

[0011] Therefore, the present application provides a micro-defect detection method to overcome the problem of difficult trade-off between micro-defect detection accuracy and detection speed.

[0012] The micro-defect detection method provided by the present application comprises the following steps:

[0013] S1acquire the picture of the standard workpiece, find the micro-hole position of the standard workpiece, and cut the micro-hole picture as picture M;

[0014] S2perform binaryzation processing on picture M to generate a standard micro-hole binaryzation template, denoted as template M ′ , and further calculate the average value PM ′ of the pixel value of template M ′ ;

[0015] S3acquire the picture of the micro-hole workpiece to be detected, denoted as picture A;

[0016] S4perform binaryzation processing on picture A to generate a binaryzation picture A ′ ;

[0017] S5divide picture A ′ into several image blocks A ′ i,j , and further calculate the average value PA ′ i,j of the pixel value of each image block A ′ i,j ; wherein i, j represent the i-th row and j-th column image block;

[0018] S6obtain PA ′ i,j and PM ′ respectively, and then subtract PM ′ n from PA ′ i,j , i.e. SA′ , find SA ′ n The three image blocks with the smallest values are found, denoted as SA ′ 1, SA ′ 2, SA ′ 3

[0019] S7 subtracts the three image blocks SA ′ 1, SA ′ 2, SA ′ 3 from the standard micropore binary template M ′ by pixel value by subtraction, and sums, that is, S n = sum(SA ′ n -M ′ ), and the calculation results are denoted as S1, S2, and S3.

[0020] S8 selects the calculation block position information with the smallest S n value among S1, S2, and S3 as the basis for cropping the picture A, crops all the pixels around the basis, and expands the resolution of the cropped image.

[0021] S9 uses a neural network model to predict the cropped and expanded image to obtain defect category and position information.

[0022] S10 identifies whether the workpiece to be inspected meets the production standard according to the defect category and position information.

[0023] Further, step S1 acquires a picture of a standard workpiece, finds the micropore position of the standard workpiece, and cuts the micropore picture, denoted as picture M. Specifically:

[0024] S11 acquires a picture of a standard workpiece, and the resolution size of the picture of the standard workpiece is HxW, where H is the number of pixels in the row direction of the picture, and W is the number of pixels in the column direction of the picture.

[0025] S12 finds the micropore position of the standard workpiece,

[0026] H W, the micropore picture is cut down to a size with a resolution of

[0027] H W, the micropore picture is cut down to a size with a resolution of

[0028] where α is a positive integer, and is the resolution adjustment coefficient of the picture M.

[0029] Further, in step S2, the picture M is subjected to binaryzation processing to generate a standard micropore binary template, denoted as template M′ ; specifically:

[0030] The binaryzation algorithm is used to take β% of the average value of the pixel number of the picture M as the threshold value, and the picture M is binaryzated to generate a standard micro-hole binaryzation template M ′ ; wherein β is a positive integer, which is a binaryzation processing threshold adjustment coefficient.

[0031] Further, in step S4, the picture A is binaryzated to generate a binaryzation picture A of the micro-hole to be detected ′ ; specifically:

[0032] The binaryzation algorithm is used to take β% of the average value of the pixel number of the picture A as the threshold value, and the picture A is binaryzated to generate a binaryzation picture A of the micro-hole to be detected ′ ; wherein β is a positive integer, which is a binaryzation processing threshold adjustment coefficient.

[0033] Further, in step S5, the picture A ′ is divided into a plurality of image blocks A ′ i,j ; specifically:

[0034] When H ′ is less than W, the picture A is divided into a plurality of image blocks A ′ i,j in the row and column directions respectively with a step size of (8·γ) pixel points according to the size of ;

[0035] When H ′ is greater than W, the picture A is divided into a plurality of image blocks A ′ i,j in the row and column directions respectively with a step size of (8·γ) pixel points according to the size of ;

[0036] ; wherein γ is a positive integer, which is a step size adjustment coefficient of the sliding window.

[0037] Further, in step S8, the resolution of the cropped image is expanded; specifically:

[0038] The resolution of the cropped image is expanded by λ times according to actual production requirements using the linear interpolation method to obtain a picture P; wherein λ is a positive integer, which is a resolution adjustment coefficient of the picture P.

[0039] Further, in step S9, the neural network model is a micro-hole processing defect detection model based on YOLOv8, which includes a backbone unit, a neck unit and a detection head unit connected in sequence;

[0040] The trunk unit comprises, in sequence, a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module, an SPPF module,

[0041] The neck unit comprises, in sequence, a first up-sampling layer, a first concatenation operation layer, a first GCVIT module, a second up-sampling layer, a second concatenation operation layer, a second GCVIT module, a sixth convolutional layer, a third concatenation operation layer, a third GCVIT module, a seventh convolutional layer, a fourth concatenation operation layer, a fifth C2f module, and an input of the third concatenation operation layer is further connected to an output of the first GCVIT module.

[0042] The trunk unit is connected to the second concatenation operation layer, the first concatenation operation layer and the fourth concatenation operation layer of the neck unit through outputs of the second C2f module, the third C2f module and the SPPF module respectively, and the neck unit is connected to the detection head unit through outputs of the second GCVIT module, the third GCVIT module and the C2f module.

[0043] Further, the first GCVIT module, the second GCVIT module and the third GCVIT module have the same structure, each GCVIT module comprising, in sequence, an eleventh convolutional layer, a twelfth convolutional layer, an improved multi-head self-attention mechanism layer, a thirteenth convolutional layer, an eleventh concatenation operation layer and a fourteenth convolutional layer, external data being input to the eleventh convolutional layer and the eleventh concatenation operation layer of the GCVIT module, and the GCVIT module outputting a result of the fourteenth convolutional layer.

[0044] Further, the improved multi-head self-attention mechanism layer comprises, in sequence, a full convolutional part and a self-attention score calculation part, and specifically comprises:

[0045] The full-group convolution with a 3x3 convolution kernel divides the feature map of the input data into three feature maps of equal size Q', K' and V';

[0046] According to the channel arrangement rule of the feature map, the three feature maps Q', K' and V' are expanded into three feature matrices Q, K and V, and the self-attention score calculation is completed according to the following formula:

[0047]

[0048] wherein X in is input data, p self is output data of the self-attention score calculation part, and g 3―i(x) represents the operation process of full-group convolution with a kernel size of 3*3 and unfolding into a feature matrix i by channel, D OUT is a scaling coefficient;

[0049] Fold the calculated feature matrix back to the feature map and output:

[0050] p A =g 3―V (X in )×p self

[0051] wherein p A represents the output data of the improved multi-head self-attention mechanism layer.

[0052] Further, in step S9, the construction method of the training data set of the neural network model is:

[0053] Collect a set of micro-defect pictures;

[0054] Use a rectangular frame to label the defect position of each picture, obtain the defect category, rectangular frame width and height, and the position coordinates of the rectangular frame center point in the picture of each picture, and save it as a label file;

[0055] Obtain the corresponding label file of each picture, and establish a micro-defect database.

[0056] The beneficial effects of the present application are:

[0057] (1) The present application uses a cutting algorithm based on binary template fast matching to remove the redundant background elements in the collected pictures, so that the deep learning model can focus more on the product defect position. Unlike traditional segmentation algorithms, the cutting algorithm based on binary template fast matching is faster and has more accurate boundaries. Compared with the traditional template matching algorithm, the binary template fast matching cutting algorithm of the present application can complete the micro-hole positioning and cutting of the defective product with only a small number of multiplication operations, has extremely high computing efficiency, and can be better used in industrial detection environment.

[0058] (2) The present application uses a traditional convolution module to improve the Q, K and V matrix generation process of the multi-head attention mechanism in the Transform feature extraction module, while retaining the feature extraction capability of the original Transform module, greatly reducing the calculation parameter amount, and speeding up the recognition rate of the detection system for micro-hole processing defect targets.

[0059] (3) The YOLOv8n-GCViT model and the YOLOv8n-Transform model of the present application are stronger than the original YOLOv8n model in feature extraction ability for micro random targets, and the parameter amount of the YOLOv8n-GCViT model of the present application is only half of that of the YOLOv8n-Transform model, which can complete the defect recognition task more quickly. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The micro defect detection method flowchart provided for the embodiments of the present application.

[0061] Figure 2 The cutting algorithm flowchart provided for the embodiments of the present application.

[0062] Figure 3 The micro-hole processing defect detection model structure diagram based on YOLOv8 provided for the embodiments of the present application.

[0063] Figure 4 The GCViT module structure diagram provided for the embodiments of the present application.

[0064] Figure 5 The improved multi-head self-attention mechanism layer schematic diagram provided for the embodiments of the present application. DETAILED DESCRIPTION

[0065] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the related drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be realized in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0066] The micro defect detection method provided by the present application includes two parts of picture cutting and model detection. The picture cutting is to remove unnecessary information. The model detection is to apply the micro-hole processing defect detection model based on YOLOv8 (YOLOv8n-GCViT model) proposed by the present application to detect and judge whether there is a defect in the product micro-hole processing position, and output the corresponding detection result.

[0067] The principle of the picture cropping method is: using the low reflectivity of the micro-hole position, using a binary algorithm to cut out the micro-hole position to be detected, the main steps including binary template generation, defect product binary feature map generation, sliding window operation, positive and negative sample difference matching operation. The sliding window operation is to use the average value of the small size pixel block and the average value of the pixel template on the defect product binary feature map to quickly locate the approximate position of the micro-hole in the defect product image, and select the most possible three sliding block positions. The positive and negative sample difference matching operation is to use the positive and negative sample difference to determine the position most likely to be a micro-hole.

[0068] The principle of the model detection method is: according to the working principle of the Transform feature extraction module, removing the redundant structure in the module, using convolution operation (Conv) to replace the linear operation (Linear) in the multi-head attention structure, designing a GCVIT feature extraction module for neural network, creating a micro-hole processing defect detection model based on YOLOv8 (YOLOv8n-GCViT model), greatly reducing the network calculation parameter amount without affecting the performance of the feature extraction module. Finally, the cropped image is used as the input of the neural network for prediction, so as to realize the rapid positioning of the defect position.

[0069] It should be noted that the product of the present application refers to a workpiece. In the embodiments, the detection of micro-hole processing defects of the product is exemplified, but the concept of the present application is applicable to the detection of various micro-defects of the product.

[0070] As shown in Figure 1 , the micro-defect detection method provided by the embodiments of the present application comprises the following steps:

[0071] S1 obtains a picture of a standard workpiece, finds the micro-hole position of the standard workpiece, and cuts out a micro-hole picture, which is denoted as picture M. Specifically:

[0072] S11 obtains a picture of a standard workpiece, and the resolution size of the picture of the standard workpiece is HxW, wherein H is the number of pixels in the row direction of the picture, and W is the number of pixels in the column direction of the picture.

[0073] In this embodiment, a standard workpiece gray-scale picture with a size of 3072x2048 is obtained from a visual detection platform. S12 finds the micro-hole position of the standard workpiece,

[0074] When H , the micro-hole picture is cut down to a size with a resolution size of ;

[0075] When H >W, the micro-hole picture is cut down to a size with a resolution size of

[0076] ;Wherein a is a positive integer, and is a resolution adjustment coefficient of the picture M.

[0077] That is, the smaller value of H and W is selected to intercept the micropore picture. The present embodiment selects 2048.

[0078] S2 binarizes the picture M to generate a standard micropore binary template, denoted as template M ′ , and further calculates the average value PM ′ of the pixel values of the template M ′ .

[0079] Specifically, a binarization algorithm is used, and β% of the average value of the pixel values of the picture M is taken as a threshold to binarize the picture M to generate a standard micropore binary template M ′ ; wherein β is a positive integer, and is a binarization threshold adjustment coefficient.

[0080] In the present embodiment, β is selected as 50, that is, 50% of the average value of the pixel brightness of the picture is taken as a reference to draw a standard micropore binary template with a size of 128x128, which only contains two kinds of pixel values of 0 and 255 (wherein 0 represents a position in the original picture with a pixel value lower than 50% of the average brightness).

[0081] S3 obtains a picture of a micropore workpiece to be inspected, denoted as picture A.

[0082] S4 binarizes the picture A to generate a binary picture A ′ to be inspected.

[0083] Specifically, a binarization algorithm is used, and β% of the average value of the pixel values of the picture A is taken as a threshold to binarize the picture A to generate a binary picture A ′ to be inspected; wherein β is a positive integer, and is a binarization threshold adjustment coefficient.

[0084] In the present embodiment, β is selected as 50, that is, 50% of the average value of the pixel brightness of the picture is taken as a reference to record pixel values lower than the average value by 50% in the image, and the recorded pixel positions are drawn in a binary image with a size of 3072x2048 to form a binary information image channel.

[0085] S5 divides the picture A ′ into a plurality of image blocks A ′ i,j , and further calculates the average value PA ′ i,j of the pixel values of each image block A ′ i,j ; wherein i, j represent the i-th row and j-th column of the image block. Specifically,

[0086] When H ′ According to the size of 128x128, the image is divided into several image blocks A in the form of convolution sliding window in the row and column directions respectively with (8·γ) pixel points as the step size. ′ i,j ;

[0087] When H ′ According to the size of 128x128, the image is divided into several image blocks A in the form of convolution sliding window in the row and column directions respectively with (8·γ) pixel points as the step size. ′ i,j ;

[0088] Wherein γ is a positive integer, which is the step adjustment coefficient of the sliding window.

[0089] In this embodiment, γ is selected as 4, that is, the calculation block size is 128x128, and 32 pixel values are taken as the step size. The average value of each calculation block in the 3072x2048 size binary information image channel in the last step is calculated in the form of sliding window.

[0090] S6 subtracts the obtained PA ′ i,j from PM ′ , that is, SA ′ n = PA ′ i,j - PM ′ , finds the three image blocks with the smallest values in SA ′ n , and records them as SA ′ 1, SA ′ 2, and SA ′ 3.

[0091] S7 subtracts the three image blocks SA ′ 1, SA ′ 2, and SA ′ 3 from the standard micropore binary template M ′ in the form of pixel value by pixel value, and sums them, that is, S n = sum(SA ′ n - M ′ ), and records the calculation result as S1, S2, and S3.

[0092] S8 selects the calculation block position information with the smallest S n value among S1, S2, and S3 as the cropping basis of the picture A, crops all the pixels around the cropping basis, and expands the resolution of the cropped image.

[0093] Specifically, the resolution of the cropped image is expanded by λ times according to actual production requirements using a linear interpolation method to obtain a picture P; wherein λ is a positive integer, which is a resolution adjustment coefficient of the picture P.

[0094] In this embodiment, the image is enlarged to 256x256 size using a linear interpolation method, and the cropping process is as shown in Figure 2 .

[0095] S9 uses a neural network model to predict the cropped and expanded image to obtain defect category m and position information n.

[0096] The defect category m represents the total number of defects identified in the picture P; the position information n represents the position information of the identified defects in the picture P, which is usually represented by coordinate information. If no defects are detected in the picture P, the position information n is empty, that is, if m = 0, then n = null.

[0097] It should be noted that the neural network model here can be a YOLOv8 model, a YOLOv8n-Transform model, or other models. This embodiment takes the YOLOv8n-GCViT model proposed in the present application as an example for illustration.

[0098] In this embodiment, the neural network model adopts a micropore processing defect detection model based on YOLOv8, i.e., a YOLOv8n-GCViT model, which includes a backbone unit, a neck unit and a detection head unit connected in sequence, as shown in Figure 3 , the backbone unit is used for picture feature extraction, the neck unit is used for picture feature fusion, and the detection head unit is used for regression operation of defect category and defect position.

[0099] The YOLOv8n-GCViT model can be expressed in the form of formula (1):

[0100] F = F 检测头单元 (F 颈部单元 (F 主干单元 (F))) (1)

[0101] Wherein, F is the model output, F 检测头单元 represents the detection head unit calculation process, F 颈部单元 represents the neck unit calculation process, F 主干单元 represents the backbone unit calculation process, and X is the input picture.

[0102] Please refer to Figure 3 , the backbone unit and the detection head unit are both the contents of the original YOLOv8 algorithm, and the neck unit is the improved algorithm content of the present application.

[0103] The backbone unit comprises, in sequence, a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module, an SPPF module,

[0104] The neck unit comprises, in sequence, a first up-sampling layer, a first concatenation operation layer, a first GCVIT module, a second up-sampling layer, a second concatenation operation layer, a second GCVIT module, a sixth convolutional layer, a third concatenation operation layer, a third GCVIT module, a seventh convolutional layer, a fourth concatenation operation layer, and a fifth C2f module, wherein the input of the third concatenation operation layer is also connected to the output of the first GCVIT module.

[0105] The backbone unit is connected to the second concatenation operation layer, the first concatenation operation layer, and the fourth concatenation operation layer of the neck unit through the output of the second C2f module, the output of the third C2f module, and the output of the SPPF module, respectively; and the neck unit is connected to the detection head unit through the output of the second GCVIT module, the output of the third GCVIT module, and the output of the C2f module.

[0106] The calculation process of the backbone unit can be represented by formula group (2):

[0107]

[0108] Wherein:

[0109] f 主干―i is the i-th output of the neck unit and f 主干―1 , f 主干―2 , f 主干―3 The three results are connected to the neck unit as the outputs of the backbone unit.

[0110] X is an input picture.

[0111] f3 represents a convolution operation with a convolution kernel size of 3x3.

[0112] f C2f represents the calculation process of the C2f module, the C2f module has a picture feature extraction function, is composed of multiple convolution operations according to a certain rule, is one of the components in the YOLO algorithm, and is a prior calculation method.

[0113] f SPPF represents the calculation process of the SPPF module, the SPPF module has a function of enhancing picture feature information, is a spatial pyramid pooling operation, is composed of multiple convolution operations and pooling operations according to a certain rule, and is a prior algorithm.

[0114] The calculation process of the neck unit can be represented by formula group (3):

[0115]

[0116] wherein:

[0117] f 颈部―i is the i-th output of the neck unit, and f 颈部―2 , f 颈部―3 , f 颈部―4 The three results are connected to the detection head unit as the outputs of the neck unit.

[0118] f GCVIT represents the calculation process of the GCVIT module; f3 represents a convolution operation with a convolution kernel size of 3x3;

[0119] Concat represents a concatenation operation.

[0120] UpSample represents an up-sampling operation.

[0121] The first concatenation operation layer, the second concatenation operation layer, the third concatenation operation layer, and the fourth concatenation operation layer in the neck unit are all in the same form of "Concat" concatenation operation, which can combine two inputs into the concatenation layer in the channel direction and then output.

[0122] Concat concatenation is a conventional concatenation method: two pictures with the same resolution size can be concatenated in the channel direction; explanation: assuming that the normal color picture format size is 3x640x640, 640x640 represents the resolution size, and 3 represents the R, G, and B color channels respectively, the output picture format size after two pictures are input into the Concat concatenation is 6x640x640.

[0123] The first up-sampling operation layer and the second up-sampling operation layer in the neck unit are both "UpSample" operations, which can adjust the resolution size of the input information according to the output requirement. UpSample is a conventional technology.

[0124] The first GCVIT module, the second GCVIT module, and the third GCVIT module in the neck unit have the same structure. As shown in Figure 4 each GCVIT module includes an eleventh convolution layer, a twelfth convolution layer, an improved multi-head self-attention mechanism layer, a thirteenth convolution layer, an eleventh concatenation operation layer, and a fourteenth convolution layer connected in sequence, the external data is input into the eleventh convolution layer and the eleventh concatenation operation layer of the GCVIT module, and the GCVIT module outputs the result of the fourteenth convolution layer.

[0125] The mathematical expression of the GCVIT module is shown in formula (4):

[0126] f GCVIT = X + f3(f1(pA (f1(f3(X))))) (4)

[0127] wherein:

[0128] f GCVIT is an output picture;

[0129] X is an input picture;

[0130] f1 represents a convolution operation with a kernel size of 1x1;

[0131] f3 represents a convolution operation with a kernel size of 3x3;

[0132] p A represents a GCVIT operation.

[0133] The splicing operation in the GCVIT module is an "ADD" splicing operation, which can combine two inputs with the same format into the splicing layer in the resolution direction and output after the resolution direction combination, so as to ensure that the output format maintains the original input format.

[0134] The ADD splicing is a conventional splicing method: two pictures with the same resolution size can be spliced in the form of addition in the resolution direction; explanation: assuming that the normal color picture format size is 3x640x640, 640x640 represents the resolution size, and 3 represents the R, G, and B color channels respectively, after the two pictures are input into the ADD splicing, the output picture format size is still 3x640x640, but the value of each pixel of the output picture is the sum of the pixel values of the corresponding positions of the original two input pictures.

[0135] As shown in Figure 5 , the improved multi-head self-attention mechanism layer includes a full convolution part and a self-attention score calculation part connected in sequence, specifically: Figure 4 The input data feature map is divided into three feature maps with the same size Q', K', and V' by using a full group convolution with a 3x3 kernel;

[0136] According to the channel arrangement rule of the feature map, Q', K', and V' are expanded into Q, K, and V three feature matrices, and the self-attention score calculation is completed according to the following formula:

[0137]

[0138]

[0139] wherein, X in is input data, p self is output data of the self-attention score calculation part, g 3―i (x) represents an operation process of expanding into a feature matrix i after full group convolution with a 3x3 kernel, D OUT ​to scale the coefficients;

[0140] Fold the calculated feature matrix back to the feature map and output:

[0141] p A = g 3―V (X in ) x p self

[0142] wherein p A represents the output data of the improved multi-head self-attention mechanism layer.

[0143] The construction method of the training data set of the neural network model is:

[0144] Collect a set of micro-defect pictures;

[0145] Use a rectangular frame to label the defect position of each picture, obtain the defect category, rectangular frame width and height, and position coordinates of the rectangular frame center point in the picture of each picture, and save it as a label file in ".txt" format;

[0146] Obtain the label file corresponding to each picture, and establish a micro-defect database.

[0147] In this embodiment, the 20x20, 40x40 and 80x80 dimensional feature maps generated by the backbone unit are used as the input of the neck unit, and after two times of upsampling (Upsample), GCVIT module, 80x80, 40x40 and 20x20 dimensional feature maps are obtained. The 80x80 feature map obtained is subjected to convolution downsampling (Conv), GCVIT module, convolution downsampling (Conv), C2f module, and then splicing and fusion operation (Concat) with the previously obtained feature map, and then input into the feature regression calculation module (Detect). Thus, the target detection link is completed.

[0148] S10 identifies whether the workpiece to be inspected meets the production standard according to the defect category and position information.

[0149] The general production detection standard is: m 标准 = 0 and n 标准 = null.

[0150] In steps S3 to S10, the embodiment also uses a data enhancement method (including rotation, scaling, flipping, color jittering, etc.) to enhance the diversity of the training data, thereby improving the robustness and detection accuracy of the detection model.

[0151] Experimental design:

[0152] The machining defects of the middle frame microphone hole in the mobile phone are used as the algorithm data set. 1400 pictures of the machining defects of the middle frame microphone hole are divided into a training set, a test set and a validation set according to a ratio of 8:1:1. The training data set includes pictures of the machining burr defects, the machining cap defects and the machining chip breaking defects. The pictures in the training set are trained by using a YOLOv8n model (an original model), a YOLOv8n-Transform model (an original model added with a Transform module) and a YOLOv8n-GCViT model (a model provided by the application), and the training weights are imported into the network structure to test the test set.

[0153] The performance comparison of each network model is shown in the following table of experimental results:

[0154]

[0155]

[0156] As can be seen from the above table, compared with the YOLOv8n model (the original model), the YOLOv8n-GCViT model (the model provided by the application) has a great improvement in the recognition accuracy of the tiny burr defects, and the parameter amount has a great advantage compared with the YOLOv8n-Transform model.

[0157] The technical features of the above embodiments can be combined arbitrarily, and to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0158] The above embodiments only express the preferred implementation of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting minute defects, characterized in that, Includes the following steps: S1 acquires an image of a standard workpiece, locates the micro-holes in the standard workpiece, and extracts an image of the micro-hole, denoted as image M. S2 performs binarization on image M to generate a standard micro-hole binarization template, denoted as template M′, and further calculates the average pixel value PM′ of template M′. S3 acquires an image of the micro-hole workpiece to be inspected, denoted as image A; S4 performs binarization on image A to generate a binarized image A′ of the micropore to be inspected. S5 divides image A′ into several image blocks A. ′ i,j And further calculate each image block A ′ i,j The average pixel value PA ′ i,j Where i,j represent the image patch in the i-th row and j-th column; S6 will obtain PA ′ i,j Subtract each from PM′, i.e., SA ′ n =PA ′ i,j -PM', find SA ′ n The three image patches with the smallest median values ​​are denoted as SA. ′ 1. SA ′ 2. SA ′ 3; S7 will find three image patches SA ′ 1. SA ′ 2. SA ′ 3. Subtract the values ​​of each pixel from the standard micro-hole binarized template M′ and sum the results, i.e., S. n =sum(SA) ′ n -M′), and record the calculation results as S1, S2, S3; S8 selects S1, S2, and S3. n The location information of the calculation block with the smallest value is used as the cropping basis for image A. All pixels surrounding the cropping basis are cropped, and the resolution of the cropped image is increased. S9 uses a neural network model to predict the cropped and enlarged image to obtain defect category and location information; S10 identifies whether the workpiece to be inspected meets production standards based on the defect category and location information.

2. The method for detecting minute defects according to claim 1, characterized in that, Step S1: Obtain an image of the standard workpiece, locate the micro-holes in the standard workpiece, and crop an image of the micro-holes, denoted as image M; specifically: S11 Acquire an image of a standard workpiece, wherein the image resolution of the standard workpiece is H×W, where H is the number of pixels in the row direction of the image and W is the number of pixels in the column direction of the image. S12 locates the micro-hole position on the standard workpiece. When H < W, the microporous image is intercepted in the size of and denoted as image M; When H>W, the resolution size is... The size of the micropore image is cropped and denoted as image M; Where α is a positive integer and is the resolution adjustment factor for image M.

3. The method for detecting minute defects according to claim 1, characterized in that: In step S2, the image M is binarized to generate a standard micropore binarized template, denoted as template M′; Specifically: Using a binarization algorithm, the average value of the pixel values ​​in image M is taken as β% as the threshold to binarize image M and generate a standard micropore binarization template M′. Where β is a positive integer and is the threshold adjustment coefficient for binarization processing.

4. The method for detecting minute defects according to claim 1, characterized in that: In step S4, image A is binarized to generate a binarized image A′ of the micropore to be inspected; specifically: Using a binarization algorithm, the average value of the pixel values ​​in image A is taken as β% as the threshold to binarize image A, generating a binarized image A′ of the micropores to be detected; where β is a positive integer and is the threshold adjustment coefficient for binarization processing.

5. The method for detecting minute defects according to claim 2, characterized in that, In step S5, image A′ is divided into several image blocks A. ′ i,j Specifically: When H < W, the picture A' is divided into a number of image blocks A in the form of image size, with a step size of (8·γ) pixel points in both the row and column directions, using a convolutional sliding window method. ′ i,j ; When H>W, adjust image A′ according to... The image is divided into several image blocks A by using a convolution sliding window with a stride of (8·γ) pixels in both the row and column directions. ′ i,j ; Where γ is a positive integer and is the step size adjustment coefficient of the sliding window.

6. The method for detecting minute defects according to claim 1, characterized in that: In step S8, the resolution of the cropped image is increased; specifically: The resolution of the cropped image is increased by a factor of λ according to actual production needs using linear interpolation to obtain image P; Where λ is a positive integer and is the resolution adjustment factor for image P.

7. The method for detecting minute defects according to claim 1, characterized in that, In step S9, the neural network model is a micro-hole processing defect detection model based on YOLOv8, which includes a trunk unit, a neck unit, and a detection head unit connected in sequence. The backbone unit comprises a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module, and an SPPF module, all connected in sequence. The neck unit includes a first upsampling layer, a first splicing operation layer, a first GCVIT module, a second upsampling layer, a second splicing operation layer, a second GCVIT module, a sixth convolutional layer, a third splicing operation layer, a third GCVIT module, a seventh convolutional layer, a fourth splicing operation layer, and a fifth C2f module connected in sequence. The input of the third splicing operation layer is also connected to the output of the first GCVIT module. The backbone unit is connected to the second splicing operation layer, the first splicing operation layer, and the fourth splicing operation layer of the neck unit through the outputs of the second C2f module, the third C2f module, and the SPPF module, respectively; the neck unit is connected to the detection head unit through the outputs of the second GCVIT module, the third GCVIT module, and the fifth C2f module.

8. The method for detecting minute defects according to claim 7, characterized in that: The first GCVIT module, the second GCVIT module, and the third GCVIT module have the same structure. Each GCVIT module includes an eleventh convolutional layer, a twelfth convolutional layer, an improved multi-head self-attention mechanism layer, a thirteenth convolutional layer, an eleventh concatenation operation layer, and a fourteenth convolutional layer connected in sequence. External data is input to the eleventh convolutional layer and the eleventh concatenation operation layer of each GCVIT module, and each GCVIT module outputs the result of the fourteenth convolutional layer.

9. The method for detecting minute defects according to claim 8, characterized in that, The improved multi-head self-attention mechanism layer includes a fully convolutional part and a self-attention score calculation part connected in sequence, specifically: The input data feature map is divided into three equal-sized feature maps Q', K', and V' using a fully grouped convolution with a 3×3 kernel. Based on the channel arrangement pattern of the feature maps, the three feature maps Q', K', and V' are expanded into three feature matrices Q, K, and V, and the self-attention score is calculated according to the following formula: Among them, X in For input data, (X) in ) T For X in transpose, p self For the output data of the self-attention score calculation part, g 3-i (x) represents the operation process of performing a fully grouped convolution with a kernel size of 3×3 and expanding it into a feature matrix i by channel, where i is Q, K, V, and D. OUT This is the scaling factor; Fold the calculated feature matrix back into the feature map and output it: p A =g 3-V (X in )×p self Where, p A This represents the output data of the improved multi-head self-attention mechanism layer.

10. The method for detecting minute defects according to claim 1, characterized in that, In step S9, the method for constructing the training dataset of the neural network model is as follows: Collect images of minor defects; Use rectangles to mark the defect locations in each image, obtain the defect category, rectangle width and height, and the coordinates of the rectangle center point in the image for each image, and save them as a label file; Obtain the tag file corresponding to each image and build a database of minor defects.

Images