Transparent element micro-defect double-stage detection method and transparent element micro-defect double-stage detection system

Abstract

The invention discloses a transparent element tiny defect double-stage detection method and system. A double-stage defect detection network framework generated based on pseudo labels is designed. In the first stage, pseudo-label generation of transparent element data is carried out, an unsupervised detection method and a pseudo-label extraction strategy are adopted, and a spatial pyramid balance feature fusion-adaptive dynamic modulation convolutional network is provided in the aspect of the unsupervised detection method. The network is combined with pyramid balance feature fusion and adaptive dynamic modulation convolution, so that the feature extraction capability of an unsupervised detection network on a multi-scale target and the effectiveness and flexibility of coping with more complex data features are enhanced. In the aspect of a pseudo label extraction strategy, a pseudo label dynamic mining strategy based on fusion of threshold segmentation and RPN proposal is provided, and pseudo labels of various defects can be directly generated from an unsupervised network detection result. And the second stage is defect detection based on a pseudo label. According to the network, the performance of the supervised defect detection network is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of transparent component detection, and in particular relates to a dual-stage detection method and system for tiny defects of transparent components. Background Art

[0002] Transparent components refer to materials or devices with high light transmittance that can maintain the directionality and intensity of light to a certain extent. Such components can be used to transmit light, protect optical systems, or achieve visual visibility, etc. due to their transparent properties. However, due to their high light transmittance, surface or internal defects (such as cracks, scratches, stains, etc.) of transparent components may significantly affect their performance and reliability. At present, the detection of tiny defects in transparent components mainly relies on manual inspection, which is time-consuming and labor-intensive, and the detection accuracy is not high. With the rapid development of artificial intelligence and deep learning technology, defect detection based on deep learning has gradually become an effective way to replace traditional machine vision methods. Deep learning models have strong generalization capabilities and can cope with transparent component defect detection in complex environments. Although deep learning technology has achieved remarkable results in the field of defect detection, it still faces many challenges in transparent component defect detection scenarios.

[0003] In actual industrial scenarios, defects of tiny transparent components are rare and difficult to obtain, resulting in insufficient defect samples in the training data set, affecting the accuracy and reliability of the detection model. Traditional data expansion methods are difficult to solve this problem, so data enhancement methods have broad application prospects in this context. Since the defects of transparent components are tiny and diverse, manual labeling is difficult and easily affected by subjective factors, resulting in inconsistent and inaccurate labeling. It is particularly critical to develop efficient and accurate defect labeling methods. Based on the unsupervised pseudo-label generation method, labels can be automatically generated in the absence of large-scale labeled data, reducing manual intervention and improving labeling efficiency and quality. Transparent components have various types of defects, including scratches, bubbles, impurities, etc. Different defects vary greatly in shape, size, and color characteristics, and some defects have characteristics such as transparency and edge fuzziness, making it difficult for traditional algorithms to identify and distinguish these complex defect types.

[0004] The present application is different from the prior art as follows:

[0005] CN 117197550A VR lens defect detection method and system based on image cube and deep learning

[0006] The invention provides a VR lens defect detection method and system based on image cube and deep learning, which collects a batch of offline defect sample libraries, optimizes the network based on the YOLOV8 network, builds a lightweight convolutional neural network, and trains the samples to generate efficient algorithms for image classification, Anchor-Free defect detection and instance segmentation. Each time a detection is made, the detection and classification results can be output by giving the extracted defect slice image to the trained deep learning classification model. The present invention can meet the requirements of subtle defect detection and classification of VR lens modules with different thicknesses with high accuracy.

[0007] However, in the actual production process, defective samples are very rare. At the same time, due to the characteristics of transparent components that defects and backgrounds often interfere with each other, the trained detection model has poor generalization, is prone to overfitting, and cannot identify unknown defects. Summary of the invention

[0008] The present invention takes transparent components as the research object, and proposes a dual-stage detection method for tiny defects in transparent components to address the problems of small sample size, high annotation difficulty, and multi-scale defects in transparent components. The present invention designs a tiny transparent component image acquisition system and obtains a dataset of tiny defects in transparent components.

[0009] To achieve the above object, the technical solution adopted by the present invention is:

[0010] A dual-stage detection system for minute defects of transparent components comprises an LED light source, a wide-field CCD camera, a bracket, an integrated cable and a computer. The LED light source illuminates the transparent component by backlighting, the wide-field CCD camera is mounted on the bracket, and the wide-field CCD camera is connected to the computer by an integrated cable.

[0011] As a further improvement of the system of the present invention, the wide-field CCD camera has a resolution of 2448×2048 and a field of view of 10 mm.

[0012] The present invention provides a two-stage detection method for tiny defects of transparent components, and the specific process is as follows:

[0013] (1) Design a tiny transparent component image acquisition system and create a tiny transparent component defect dataset;

[0014] (2) A multi-scale step-by-step generation model for transparent component defects is proposed. The first step is to separate defects and background. The second step is to use the generation network to generate high-quality defect images. The third step is to synthesize the generated high-quality defect images with the background images.

[0015] (3) A spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network (PBA-Net) is proposed. PBA-Net enhances the feature extraction capability of the PatchCore network for multi-scale defects and its adaptability to complex data through pyramid balanced feature fusion, adaptive dynamic convolution, and convolution instead of average pooling, thereby improving the accuracy and flexibility of unsupervised detection.

[0016] (4) Generate pseudo labels for various defects for the unsupervised detection network results. The input image is first classified by hierarchical clustering to classify the defects detected by the unsupervised detection network. The classified image is then segmented by adaptive threshold to preliminarily screen out the defective parts. Then, the pseudo labels based on adaptive threshold segmentation are corrected by candidate boxes based on RPN proposal fusion.

[0017] (5) Using data annotated with pseudo labels, the improved YOLOv11 defect detection network is trained and a method combining multi-frequency wavelet convolution with multi-scale variable convolution kernel attention mechanism is proposed to enhance the accuracy of feature expression.

[0018] As a further improvement of the method of the present invention, the specific process of step (1) is as follows:

[0019] A transparent component micro-defect image acquisition system is designed. The system consists of three parts: a wide-field CCD camera (3), an LED light source, and a backlighting method. The wide-field CCD camera (3) converts a light signal into a charge signal through the photoelectric effect, and then obtains a digital image signal through charge transfer, signal amplification, and analog-to-digital conversion steps. The LED light source is used to provide lighting. An observable image is generated through the interaction between light and the object being photographed, which directly affects the quality, brightness, contrast, and detail of the image. The LED light source meets the conditions that the light brightness is always adjustable, the color is relatively uniform, and it can be illuminated at multiple angles. The LED light source is used as the light source of the system. The backlighting method illuminates the object being photographed by controlling the distribution, intensity, angle, and direction of the light source. The light source is located behind the object and is used to highlight the outline of the object. Images of micro-defects of transparent components are acquired through the transparent component micro-defect image acquisition system, and a transparent component micro-defect data set is established based on these images.

[0020] As a further improvement of the method of the present invention, the specific process of step (2) is as follows:

[0021] The transparent component micro defect multi-scale step-by-step generation model is used to expand the transparent component micro defect dataset. In the first step, the defect foreground and platform background are separated using the background foreground separation method based on the Otsu threshold method. First, the grayscale histogram of the image is calculated, that is, the number of pixels at each grayscale level, and then the global grayscale mean is calculated, as shown in formula (1):

[0022]

[0023] In formula (1), p(i) is the probability of a pixel with gray value i, and N is the total number of pixels in the image;

[0024] Using the improved background average gray value calculation method, the image is divided into three parts. The first part is the background adjacent to the defect part, the second part is the background not adjacent to the defect part, and the third part is the defect part. The weight calculation formulas of the three parts are shown in equations (2), (3), and (4):

[0025]

[0026]

[0027] ω2(T)=1-ω0(T)-ω1(T) (4)

[0028] In the formula, ω0(T) represents the weight of the background adjacent to the defect, ω1(T) represents the weight of the background not adjacent to the defect, and ω2(T) represents the weight of the defect. The means of the three parts are shown in formulas (5), (6), and (7):

[0029]

[0030] Among them, μ0(T) represents the mean of the background adjacent to the defect part, μ1(T) represents the mean of the background not adjacent to the defect part, and μ2(T) represents the mean of the defect part. Combining the weights of the three parts for each possible threshold T, the inter-class variance of the foreground and background is calculated, as shown in formula (8):

[0031]

[0032] Where α represents the variance weight of the grayscale value of the background pixels adjacent to the defective part, and β represents the variance weight of the grayscale value of the background pixels not adjacent to the defective part. The value of α should be greater than the value of β to ensure that the contrast between the defective part and the background part is relatively large. The weight, mean, and intra-class variance of the background and foreground under different thresholds T are calculated, and the threshold T that maximizes the inter-class variance is selected. Then, the threshold is used to perform image binarization to extract the background foreground.

[0033] The second step of the multi-scale step-by-step generation method for transparent component defects is defect image generation based on the GAN network. The GAN network used significantly expands the network size and utilizes more complex training techniques and conditional generation methods.

[0034] The second step of the multi-scale step-by-step generation model of transparent component defects is an image synthesis method based on smooth transparency transition. First, the defect image and background image and their transparency information are prepared, and then a transparency gradient mask is created. Considering the influence of lighting and environment, the lighting model is introduced to calculate the smooth transparency transition. For each pixel, the synthesized color value C final The calculation formula is as follows:

[0035] C final =α source ·(C source I light )+(1-α source )·(C target I background ) (9)

[0036] Where C source and C target are the color values ​​of the source image and the background image, respectively, light is the illumination intensity calculated by the illumination model, I background is the background illumination or background light intensity, which is used to adjust the brightness of the background part in the composite image. source is the transparency value of the source image.

[0037] As a further improvement of the method of the present invention, the specific process of step (3) is as follows:

[0038] Step 1: Divide the image into multiple P×P patches;

[0039] The input image is divided into multiple small patches, each patch contains a part of the image area, and multiple patches are put into a list, called a patch library, as shown in formula (10):

[0040] P={p1,p2,...,p n} (10)

[0041] Step 2: For each patchp i, use the pre-trained convolutional neural network CNN to extract its features. The convolutional neural network CNN used is ResNet. The features of each patch are extracted by the pre-trained ResNet. The features extracted by ResNet are divided into three parts: large-scale features, medium-scale features, and small-scale features. After the three different scale features are further extracted by convolution operation, the three features are embedded and spliced ​​for feature fusion. First, the convolution expansion operation is used, that is, the image is divided according to the specified convolution kernel size. This operation converts the input image into multiple local small blocks of feature representation. Assuming that the sizes of the input features x and y are B×C1×H1×W1 and B×C2×H2×W2 respectively, the embedding splicing formula is as follows:

[0042]

[0043] Where i, j represent the position in the expanded feature map, m, n represent the rows and columns of the small block, K(p, q) is the convolution kernel, and the expansion operation divides the image into small blocks according to the convolution kernel, and then the image feature x of the convolution expansion operation is unfolded Concatenate y with the beginning and the end, as shown in formula (12):

[0044]

[0045] Where x unfolded (b,c1,h,w,n) is the feature of the (c1,h,w)th position of the nth small block of the expanded image x, y(b,c2,h,w) is the feature of the image y at (c2,h,w), and z(b,c,h,w) is the concatenated feature with a size of B×(C1+C2)×H2×W2, where (C1+C2) is the number of channels after concatenation;

[0046] Step 3: Use the deconvolution operation. After the splicing operation, the spliced ​​result z is folded to restore its original image size H1×W1. The calculation formula is as follows:

[0047]

[0048] The image size after deconvolution becomes the size of the original image x, B×(C1+C2)×H1×W1. Here, the features of the three scales are spliced, and the size of the splicing result is B×(C1+C2+C3)×H1×W1, where C1+C2+C3 is the number of channels of the three scale features, and H1×W1 is the height and width of the medium-scale feature. Each patch is mapped to a high-dimensional feature space through the feature extraction network, and these patches are regarded as a feature. The features of all normal image blocks are saved in the feature library;

[0049] F={f(xj )|x j ∈normalimagepatches} (14)

[0050] Where F is the feature library, f(x j ) is from the normal image block x j The extracted features,the feature library is a reference library for the subsequent detection stage,,and each element in the feature library is a feature vector of a,trained normal image patch;

[0051] Step 4: Match the features extracted by the feature extraction module with the features in the feature library;

[0052] To measure the abnormality of the test image, we need to let the test image pass through the same feature extraction module to extract its patch features, and then compare the extracted patch features with the patch features in the feature library to obtain the abnormality score. test , get each f test And the feature f in the feature library * , as shown in formula (15):

[0053]

[0054] Next, use f test The maximum distance to each neighbor in the feature library is used to estimate the anomaly score. The distance here uses the Euclidean distance, as shown in formula (16):

[0055] D(f test ,f * )=||f test -f * ||2 (16)

[0056] In the formula, D(f test ,f * ) is used to estimate the feature f of the test image x test The anomaly score of ||·||2 represents the L2 norm. In order to highlight the difference with the test image f test Adjacent feature library features f * The contribution of test ,f * ) is used to calculate the anomaly score corresponding to the test image, as shown in formula (17):

[0057]

[0058] Where s represents the abnormality score of the corresponding feature of the test image, and F b Substitute the closest test image feature f testThe feature set consists of b patches;

[0059] Step 5: Draw the abnormal image heat map by testing the abnormal score of each feature of the image.

[0060] As a further improvement of the method of the present invention, the specific process of step (4) is as follows:

[0061] First, the high-score areas are classified by hierarchical clustering to obtain the category pseudo-labels of the pre-selected defects. The similarity or distance between all defect data is calculated. Then, the two most similar clusters are selected to merge or the least similar cluster is split according to the inter-cluster distance. The inter-cluster distance is recalculated based on the new clusters. The above steps are repeated until the stopping condition is met, where the average distance of all point pairs between the two clusters is used as the distance between the two clusters, as shown in formula (18):

[0062]

[0063] Among them C i and C j Represents two different clusters. The three defects obtained after hierarchical clustering are classified into cracks, scratches, and stains according to the defect characteristics and size;

[0064] Then, the data image is segmented by adaptive threshold to preliminarily screen out the defective parts. Each defect corresponds to a different threshold. The entropy theory based on information theory is used to maximize the sum of the entropy of the foreground and background. The threshold is selected according to the information entropy of the foreground and background in the image grayscale histogram.

[0065] Assume the grayscale range of the image is [0, L-1], the threshold is T, P i is the probability of gray level i, then the information entropy H is defined as:

[0066] H(T)=H f (T)+H b (T) (19)

[0067] In the formula, H f (T) is the foreground entropy, and its calculation formula is:

[0068]

[0069] H b (T) is the background entropy, and its calculation formula is:

[0070]

[0071] The threshold T is selected to maximize H(T). After calculation, the threshold of the fracture defect is 225, the threshold of the scratch defect is 213, and the threshold of the spot defect is 156. The position of the defect is initially obtained through the adaptive entropy threshold method, and then the position of the defect is corrected through the image edge features extracted by the pre-selected box based on the RPN proposal.

[0072] Then, the pseudo-label is corrected through the RPN proposal network. First, RPN accepts the feature map output by the backbone network as input, which is usually a high-dimensional feature representation of the input image after several convolutional layers. Then, a 3×3 sliding window is applied to the input feature map for convolution. Each sliding window is responsible for generating a candidate region at a position. A set of anchor boxes is generated at each sliding window position in the feature map. These anchor boxes are a series of rectangular boxes with predefined sizes and aspect ratios. The sizes and aspect ratios are different for different types of defects. The size of the fracture defect is set to [32, 64, 128]. , its aspect ratio is (1,1), (2,1), (1,2), the size of the scratch defect setting is [16,32,64], its aspect ratio is (1,1), (2,1), (1,2), the size of the scratch defect setting is [8,16,32], its aspect ratio is (1,1), (2,1), (1,2). Secondly, the defect positioning box obtained based on the adaptive entropy threshold will be combined with the candidate box of the RPN proposal to filter out the candidate box with correct position and high confidence through the IOU threshold. The calculation formula is as follows:

[0073]

[0074] In the formula, B gt represents the real box, B pred represents the prediction box, T iou is the intersection-over-union ratio threshold, M keep Represents the set of pre-selected boxes that meet the conditions, i.e., those with a value greater than the intersection-over-union ratio threshold;

[0075] Then, the coordinate average of these candidate boxes is taken to obtain the candidate box based on RPN proposal fusion, as shown in formula (24):

[0076]

[0077] in, Represents the average value of the horizontal coordinate of the set of points in the upper left corner of the candidate box. Represents the average value of the ordinate of the set of upper left corner points of the candidate box, N represents the number of pre-selected boxes in the set, and the lower right corner point can be obtained similarly;

[0078] The pseudo-label based on adaptive threshold segmentation is corrected by the candidate box based on RPN proposal fusion. The calculation formula is as follows:

[0079]

[0080] where x fianl is the horizontal coordinate of the upper left corner of the corrected positioning frame, y fianl is the vertical coordinate of the upper left corner of the corrected positioning frame, x * is the horizontal coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, y * is the ordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation. α and β are preset values ​​to ensure that the correction result is close to reality. The lower right corner can be obtained similarly. Finally, the anchor box and category obtained from the defect are converted into YOLO format and pseudo-labeled.

[0081] As a further improvement of the method of the present invention, the specific process of step (5) is as follows:

[0082] The prepared pseudo-label data is input into the supervised defect detection network for training. The real-time performance of the detection network based on RCNN and transformer cannot meet the requirements. The defect detection algorithm based on YOLOv11 is adopted, and ML-Net is proposed, which consists of two parts: multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism. The input image is decomposed into different frequency components through wavelet transform, and small-size convolution is performed on each frequency layer. Finally, the results are recombined through inverse wavelet transform, thereby realizing multi-scale analysis of the image. The MDKA attention mechanism highlights important areas through weighted fusion of feature maps;

[0083] a) Multi-frequency wavelet convolution;

[0084] First, MFWTConv uses two-dimensional Haar wavelet transform to perform multi-level decomposition of the input image. Assume that the image is a two-dimensional function f(x, y). Haar wavelet transform uses four filters to decompose the image into four sub-bands. The calculation formula is as follows:

[0085]

[0086] Low-frequency component LL: captures low-frequency information of the image, such as overall shape or contour;

[0087]

[0088] Horizontal high frequency component LH: captures horizontal edge information in the image;

[0089]

[0090] Vertical high frequency component HL: captures vertical edge information in the image;

[0091]

[0092] Diagonal high-frequency components HH: capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled, that is, the spatial resolution is halved, but the frequency information is more finely decomposed. Recursively performing wavelet transform is called multi-level decomposition to obtain frequency components at different scales. Use a small convolution kernel of 3x3 or 5x5 to perform convolution operations on each decomposed sub-band. Since the wavelet transform reduces the spatial resolution of each sub-band, a small-sized convolution kernel can cover a larger area of ​​the original image, that is, the receptive field is increased. After completing the convolution, use the inverse wavelet transform to re-synthesize the convolution results of each sub-band into a complete output;

[0093]

[0094] By using the above operations, the low-frequency and high-frequency components of the original image are restored to a complete image, and the features of different frequency levels are fused together;

[0095] b) Multi-scale variable convolution kernel attention mechanism;

[0096] Deformable convolution enables the sampling grid to be freely deformed with integer offsets. An additional convolutional layer learns the deformation from the feature map to create an offset field. The deformation is learned based on the features themselves to obtain the adaptive convolution kernel. The convolution layer responsible for calculating the offset follows the kernel size and expansion of its corresponding convolution layer. Bilinear interpolation is used to calculate the pixel value of the offset that is not found on the image grid. The MDKA module can be expressed as:

[0097] Attention=Conv1×1(DDW-D-Conv(DDW-Conv(F′))) (36)

[0098]

[0099] The input feature F∈R C×H×W ,Attention∈R C×H×W Represents the attention matrix, each value represents the relative importance of the corresponding feature, and the operator Represents the convolution operation, and the multi-scale variable convolution kernel attention mechanism allows the shape and size of the kernel to be adaptively adjusted according to the input features in the attention calculation;

[0100] After using MFWTConv to replace the traditional convolution module of the C3K2 module in YOLOv11 and introducing the MDKA module into the C2PSA module in YOLOv11, we first perform initialization training, setting epochs=300, imgsz=640, batch=16, and then input the image to extract features through the backbone network, and the decoder layer outputs the anchor box prediction. Then, we use the label to calculate the loss, and finally update the weights. We use the optimizer SGD to adjust the model parameters. The positioning loss is used to measure the deviation between the predicted box and the real box. The positioning loss used is as follows:

[0101]

[0102] Where IOU represents the intersection-over-union ratio between the predicted box and the real box. represents the Euclidean distance between the center points of the boxes, c represents the minimum box diagonal length of the predicted box and the true box, and v represents the aspect ratio consistency term, as shown in formula (39);

[0103]

[0104] α is the weight balancing term, as shown in formula (40)

[0105]

[0106] The classification loss is used to measure the difference between the predicted category distribution and the true category. Assuming that the category probability predicted by the model is The true category is p i : The classification loss formula used is as follows:

[0107]

[0108] Confidence loss is used to measure the accuracy of whether the prediction box contains the target. Let the confidence of the prediction box be The confidence of the true box is c, the target box is 1, and no target is 0. N represents the total number of predicted boxes, and its calculation formula is as follows:

[0109]

[0110] The total loss of YOLO is the weighted sum of the above three losses, as shown in formula (43):

[0111]

[0112] where λ loc , cls , conf They are the weight hyperparameters of the positioning loss, classification loss, and confidence loss of the YOLOv11 network respectively;

[0113] After the model training is completed, the optimal weights are loaded and inference is performed on the image or dataset.

[0114] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0115] (1) The present invention designs a multi-scale step-by-step generation model of transparent component defects based on a GAN network to separate defects through a foreground-background separation method, and then generates a realistic defect image through a high-resolution GAN image generation network, and then synthesizes the generated defect image with the background image through a smooth transparency transition.

[0116] (2) The present invention proposes a two-stage defect detection network framework based on pseudo labels. The framework consists of a pseudo label generation network in the first stage and a supervised network trained with pseudo label data in the second stage. The framework exhibits stronger adaptability and high generalization ability when facing unknown defects, and realizes accurate detection of tiny defects in transparent components.

[0117] (3) The present invention constructs a spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network (PBA-Net), which combines pyramid balanced feature fusion and adaptive dynamic modulation convolution to enhance PatchCore's feature extraction capability for multi-scale targets and its effectiveness and flexibility in dealing with more complex data features.

[0118] (4) The present invention proposes a pseudo-label dynamic mining strategy based on threshold segmentation and RPN proposal fusion, which can directly generate pseudo-labels of various defects from unsupervised network detection results.

[0119] (5) The present invention introduces a new network based on wavelet transform and multi-scale attention, which combines multi-frequency wavelet convolution and variable large kernel attention mechanism to significantly improve the performance of supervised defect detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] Figure 1 This is a diagram of a system for detecting minute defects of a transparent element according to the present invention;

[0121] In the figure, 1. transparent element; 2. LED light source; 3. wide-field CCD camera; 4. bracket; 5. integrated cable; 6. computer;

[0122] Figure 2 It is a flow chart of the dual-stage detection of tiny defects of transparent components based on pseudo-label generation of the present invention;

[0123] Figure 3 It is the original image of the tiny defects of the transparent element of the present invention;

[0124] Figure 4It is an image generated by the multi-scale step-by-step generation model of transparent element defects of the present invention;

[0125] Figure 5 The present invention generates pseudo labels of various defects by using a pseudo label dynamic mining strategy;

[0126] Figure 6 It is a PR curve of the two-stage detection result of the transparent element micro defect generated by the present invention based on the pseudo label;

[0127] Figure 7 The present invention visualizes the results of the two-stage detection of tiny defects of transparent components generated based on pseudo labels. DETAILED DESCRIPTION

[0128] The following is a detailed description of the technical solution of the application in conjunction with the accompanying drawings. The described embodiments are only part of the embodiments involved in this patent. All non-innovative embodiments of other researchers in this field based on this embodiment are within the scope of protection of this patent.

[0129] This paper proposes a two-stage detection method and system for tiny defects of transparent components based on pseudo-label generation. Figure 1 The invention is a transparent element micro defect system, which includes an LED light source 2, a wide-field CCD camera 3, a bracket 4, an integrated cable 5 and a computer 6. The LED light source 2 illuminates the transparent element 1 by backlighting, the wide-field CCD camera 3 is installed on the bracket 4, and the wide-field CCD camera 3 is connected to the computer 6 by an integrated cable 5.

[0130] Figure 2 The present invention proposes a two-stage detection method for tiny defects of transparent components based on pseudo-label generation. Figure 3 This is the original image of the tiny defects of the transparent element of the present invention. The specific implementation scheme is as follows:

[0131] (1) Design of a system for capturing images of tiny defects in transparent components

[0132] The research object of the present invention is the tiny defects of transparent components. Since transparent components have the characteristics of reflectivity, tiny defects and complex environment, it is difficult to obtain images of tiny defects of transparent components. Therefore, in order to obtain images of tiny defects of transparent components to construct a data set of tiny defects of transparent components, the present invention designs a transparent component tiny defect image acquisition system, which consists of three parts: a wide-field CCD camera, an LED light source, and a backlighting method. Among them, the wide-field CCD camera is the core of the system, which converts light signals into charge signals through the photoelectric effect, and then obtains digital image signals through charge transfer, signal amplification, analog-to-digital conversion and other steps. The selected CCD camera has a resolution of 2448×2048 and a field of view of 10mm, which can meet the requirement that the viewing range needs to cover the entire component. The second part is the LED light source. The main function of the light source is to provide lighting. Through the interaction between light and the object being photographed, an observable image is generated, which directly affects the quality, brightness, contrast and detail performance of the image. The LED light source meets the conditions of constant, adjustable light brightness, relatively uniform color, and multi-angle irradiation. Therefore, the LED light source is used as the light source of the system. The third part is the backlighting method. Lighting refers to illuminating the object being photographed by controlling the distribution, intensity, angle and direction of the light source to ensure that the imaging effect reaches the expected goal. Different lighting methods can greatly affect the quality, detail presentation, contrast and aesthetic effect of the image. Backlighting refers to the light source located behind the object, which is mainly used to highlight the outline of the object. It is suitable for extracting the edge of the object being measured and detecting various impurities in the object being measured. Therefore, the backlighting method is selected as the lighting method. Images of tiny defects of transparent components are collected through the transparent component tiny defect image acquisition system, and a transparent component tiny defect data set is established with these images.

[0133] (2) Multi-scale step-by-step generation model for transparent component defects

[0134] In actual industrial scenarios, defective components often have problems such as small number and difficulty in obtaining. Transparent component defect images lack large-scale data, and the small sample size may lack representativeness of the whole, which makes the estimate calculated based on the sample lack effectiveness and low accuracy. Therefore, the present invention adopts a multi-scale step-by-step generation model of transparent component micro-defects to expand the transparent component micro-defect data set. Since transparent component micro-defect images have low defect-background contrast and a small proportion of defective parts, directly generating data through transparent component micro-defects will cause edge distortion and background and foreground fusion problems, and will also cause serious waste of computing resources. Therefore, the first step of the multi-scale step-by-step generation model of transparent component defects is to use the background and foreground separation method based on the Otsu threshold method (OTSU) to separate the defect foreground and the platform background. First, the grayscale histogram of the image is calculated, that is, the number of pixels at each grayscale level, and then the global grayscale mean is calculated, as shown in formula (1):

[0135]

[0136] In formula (1), p(i) is the probability of a pixel with gray value i, and N is the total number of pixels in the image.

[0137] Since our goal is to increase the contrast between the defect part and the background part, especially the grayscale value of the background pixel adjacent to the defect part should be greatly different from the grayscale value of the defect part, we use the improved background average grayscale value calculation method to divide the image into three parts. The first part is the background adjacent to the defect part, the second part is the background not adjacent to the defect part, and the third part is the defect part. The weight calculation formulas of the three parts are shown in equations (2), (3), and (4):

[0138]

[0139] ω2(T)=1-ω0(T)-ω1(T) (4)

[0140] In the formula, ω0(T) represents the weight of the background adjacent to the defect, ω1(T) represents the weight of the background not adjacent to the defect, and ω2(T) represents the weight of the defect. The means of the three parts are shown in formulas (5), (6), and (7):

[0141]

[0142] Among them, μ0(T) represents the mean of the background adjacent to the defect part, μ1(T) represents the mean of the background not adjacent to the defect part, and μ2(T) represents the mean of the defect part. Combining the weights of the three parts for each possible threshold T, the inter-class variance of the foreground and background is calculated, as shown in formula (8):

[0143]

[0144] Where α represents the variance weight of the grayscale value of the background pixels adjacent to the defective part, and β represents the variance weight of the grayscale value of the background pixels not adjacent to the defective part. Generally speaking, the value of α should be greater than the value of β to ensure that the contrast between the defective part and the background part is relatively large. The weight, mean, and intra-class variance of the background and foreground are calculated under different thresholds T, and the threshold T that maximizes the inter-class variance is selected. Then, the threshold is used to binarize the image to extract the background foreground.

[0145] The second step of the multi-scale step-by-step generation method for transparent element defects is defect image generation based on the GAN network. The GAN network used in the present invention significantly improves the quality of the generated images by significantly expanding the network scale and utilizing more complex training techniques and conditional generation methods, and can generate higher resolution and more detailed images.

[0146] The second step of the multi-scale step-by-step generation model of transparent component defects is an image synthesis method based on smooth transparency transition. First, the defect image and background image and their transparency information are prepared, and then a transparency gradient mask is created. Considering the influence of lighting and environment, the lighting model is introduced to calculate the smooth transparency transition. For each pixel, the synthesized color value C final The calculation formula is as follows:

[0147] C final =α source ·(C source I light )+(1-α source )·(C target I background ) (9)

[0148] Where C source and C target are the color values ​​of the source image and the background image respectively. light is the illumination intensity calculated by the illumination model (including ambient light, diffuse light, specular light, etc.), I background is the background illumination or background light intensity, which is used to adjust the brightness of the background part in the composite image. source is the transparency value of the source image. The above method can be used to smoothly synthesize the defect image and the background image. The result of the synthesis is as follows: Figure 4 shown.

[0149] (3) Use PatchCore improved based on PBA-Net to identify abnormalities in normal samples.

[0150] Step 1: Divide the image into multiple P×P patches

[0151] The input image is divided into multiple small blocks (patches), each patch contains a part of the image area. This operation can help the model focus on the local information of the image. We put multiple patches into a list, called a patch library, as shown in formula (10):

[0152] P={p1,p2,...,p n} (10)

[0153] Step 2: For each patchp i, we use a pre-trained convolutional neural network (CNN) to extract its features. The convolutional neural network (CNN) used in the present invention is ResNet. The features of each patch are extracted through the pre-trained ResNet. To solve the problem of difficulty in extracting multi-scale defects, the present invention proposes pyramid balanced feature fusion, which divides the features extracted by ResNet into three parts: large-scale features, medium-scale features, and small-scale features. After further feature extraction through convolution operation, the three features are embedded and spliced ​​for feature fusion. First, the convolution expansion operation is used to segment the image according to the specified convolution kernel size. This operation converts the input image into feature representations of multiple local small blocks. Assuming that the sizes of the input features x and y are B×C1×H1×W1 and B×C2×H2×W2, respectively, the embedding and splicing formula is as follows:

[0154]

[0155] Where i, j represent the position in the expanded feature map, m, n represent the rows and columns of the small block, K(p, q) is the convolution kernel, and the expansion operation divides the image into small blocks according to the convolution kernel. unfolded Concatenate y with the beginning and the end, as shown in formula (12):

[0156]

[0157] Where x unfolded (b,c1,h,w,n) is the feature of the (c1,h,w)th position of the nth block of the expanded image x, y(b,c2,h,w) is the feature of the image y at (c2,h,w), and z(b,c,h,w) is the concatenated feature with a size of B×(C1+C2)×H2×W2, where (C1+C2) is the number of channels after concatenation.

[0158] Step 3: Use the deconvolution operation. After the splicing operation, we fold the spliced ​​result z to restore its original image size H1×W1. The calculation formula is as follows:

[0159]

[0160] The image size after deconvolution becomes the size of the original image x, B×(C1+C2)×H1×W1. Here, the features of the three scales are concatenated, and the size of the concatenated result is B×(C1+C2+C3)×H1×W1, where C1+C2+C3 is the number of channels of the three scale features, and H1×W1 is the height and width of the medium-scale feature. Each patch is mapped to a high-dimensional feature space through the feature extraction network. These patches are regarded as a feature, and the features of all normal image blocks are saved in the feature library.

[0161] F={f(x j )|x j ∈normalimagepatches} (14)

[0162] Where F is the feature library, f(x j ) is from the normal image block x j Extracted features. The feature library is a reference library for the subsequent detection stage. Each element in the feature library is a feature vector of a trained normal image block.

[0163] Step 4: Match the features extracted by the feature extraction module with the features in the feature library. To measure the abnormality of the test image, the test image needs to be passed through the same feature extraction module to extract its patch features, and then the extracted patch features are compared with the patch features in the feature library to obtain the abnormality score. test , we obtain each f test And the feature f in the feature library * , as shown in formula (15):

[0164]

[0165] Next we use f test The maximum distance to each neighbor in the feature library is used to estimate the anomaly score. Here we use the Euclidean distance, as shown in formula (16):

[0166] D(f test ,f * )=||f test -f * ||2 (16)

[0167] In the formula, D(f test ,f * ) is used to estimate the feature f of the test image x test The anomaly score of ||·||2 represents the L2 norm. In order to highlight the difference with the test image f test Adjacent feature library features f *The contribution of test ,f * ) is used to calculate the anomaly score corresponding to the test image, as shown in formula (17):

[0168]

[0169] Where s represents the abnormality score of the corresponding feature of the test image, and F b Substitute the closest test image feature f test The feature set consists of b patches.

[0170] Step 5: Draw the abnormal image heat map by testing the abnormal score of each feature of the image.

[0171] (4) Generate pseudo labels for various defects through a pseudo-label dynamic mining strategy based on adaptive entropy threshold and RPN proposal fusion.

[0172] First, the high-score areas are classified by hierarchical clustering to obtain the category pseudo-labels of the pre-selected defects. The similarity or distance between all defect data is calculated, and then the two most similar clusters are selected to merge (or the least similar cluster is split) according to the inter-cluster distance. The inter-cluster distance is recalculated based on the new clusters, and the above steps are repeated until the stop condition is reached, where the average distance of all point pairs between the two clusters is used as the distance between the two clusters, as shown in formula (18):

[0173]

[0174] Among them C i and C j Represents two different clusters. The three defects obtained after hierarchical clustering are classified into fracture, scratch, and spot according to the defect characteristics and size. Then the data image is segmented by adaptive threshold to preliminarily screen out the defective part (each defect corresponds to a different threshold). Here we use the entropy theory based on information theory to maximize the sum of the entropy of the foreground and background. The threshold is selected according to the information entropy of the foreground and background in the image grayscale histogram.

[0175] Assume the grayscale range of the image is [0, L-1], the threshold is T, P i is the probability of gray level i, then the information entropy H is defined as:

[0176] H(T)=H f (T)+H b (T) (19)

[0177] In the formula, H f (T) is the foreground entropy, and its calculation formula is:

[0178]

[0179] H b (T) is the background entropy, and its calculation formula is:

[0180]

[0181] The threshold T is selected to maximize H(T). After calculation, the threshold of the fracture defect is 225, the threshold of the scratch defect is 213, and the threshold of the spot defect is 156. The position of the defect is initially obtained through the adaptive entropy threshold method, and then the position of the defect is corrected through the image edge features extracted by the pre-selected box based on the RPN proposal.

[0182] The pseudo-labels are then corrected through the RPN proposal network. First, RPN accepts the feature map output by the backbone network (such as ResNet, VGG) as input, which is usually a high-dimensional feature representation of the input image after several convolutional layers. Then a 3×3 sliding window is applied to the input feature map for convolution, and each sliding window is responsible for generating a candidate region at a position. A set of anchors are generated at each sliding window position in the feature map. These anchor boxes are a series of rectangular boxes with predefined sizes and aspect ratios. Different types of defects have different sizes and aspect ratios. In the present invention, the size of the fracture defect is set to [32, 64, 128], and its aspect ratio is (1, 1), (2, 1), (1, 2); the size of the scratch defect is set to [16, 32, 64], and its aspect ratio is (1, 1), (2, 1), (1, 2); the size of the scratch defect is set to [8, 16, 32], and its aspect ratio is (1, 1), (2, 1), (1, 2).

[0183] Secondly, the defect positioning box obtained based on the adaptive entropy threshold will be combined with the candidate box of the RPN proposal and the candidate box with a high confidence level will be selected through the IOU threshold. The calculation formula is as follows:

[0184]

[0185] In the formula, B gt represents the real box, B pred represents the prediction box, T iou is the intersection-over-union ratio threshold, M keep Represents a set of pre-selected boxes that meet the conditions (greater than the intersection-over-union ratio threshold).

[0186] Then, the coordinate average of these candidate boxes is taken to obtain the candidate box based on RPN proposal fusion, as shown in formula (24):

[0187]

[0188] in, Represents the average value of the horizontal coordinate of the set of points in the upper left corner of the candidate box. Represents the average value of the ordinate of the set of upper left corner points of the candidate box, N represents the number of pre-selected boxes in the set, and the lower right corner point can be obtained similarly.

[0189] The pseudo-label based on adaptive threshold segmentation is corrected by the candidate box based on RPN proposal fusion. The calculation formula is as follows:

[0190]

[0191] where x fianl is the horizontal coordinate of the upper left corner of the corrected positioning frame, y fianl is the vertical coordinate of the upper left corner of the corrected positioning frame, x * is the horizontal coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, y * is the ordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation. α and β are preset values ​​to ensure that the correction result is close to reality. The lower right corner can be obtained in the same way. Finally, the anchor box and category obtained by the defect are converted into YOLO format and pseudo-labeled. The result is shown in Figure 5 shown.

[0192] (5) The pseudo-labeled data is trained through the ultralytics defect detection network improved based on ML-Net;

[0193] The prepared pseudo-label data is input into the supervised defect detection network for training. The real-time performance of the detection network based on RCNN and transformer cannot meet the requirements. Therefore, the defect detection algorithm based on YOLOv11 is currently used. The defect detection algorithm based on YOLOv11 is currently insensitive to multi-scale defects and has weak perception of small defect features. Therefore, the present invention proposes ML-Net, which consists of two parts: multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism. The input image is decomposed into different frequency components by wavelet transform, and small-size convolution is performed on each frequency layer. Finally, the results are recombined by inverse wavelet transform, thereby realizing multi-scale analysis of the image. The MDKA attention mechanism highlights important areas by weighted fusion of feature maps. This weighted method is based on the output of the deformable kernel, and can focus on key areas (such as small target features) more accurately.

[0194] b) Multi-frequency wavelet convolution

[0195] First, MFWTConv uses two-dimensional Haar wavelet transform to perform multi-level decomposition on the input image. Assume that the image is a two-dimensional function f(x,y). Haar wavelet transform uses four filters to decompose the image into four sub-bands. The calculation formula is as follows:

[0196]

[0197] Low-frequency components (LL): capture low-frequency information of an image, such as overall shape or outline.

[0198]

[0199] Horizontal high frequency component (LH): captures horizontal edge information in the image.

[0200]

[0201] Vertical high frequency component (HL): captures vertical edge information in the image.

[0202]

[0203] Diagonal high frequency components (HH): capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled (the spatial resolution is halved), but the frequency information is decomposed more finely. Recursively performing wavelet transform (called multi-level decomposition) can obtain frequency components at different scales. The present invention uses a small convolution kernel of 3x3 or 5x5 to perform convolution operations on each decomposed sub-band. Since the wavelet transform reduces the spatial resolution of each sub-band, a small-sized convolution kernel can cover a larger area of ​​the original image, that is, the receptive field is increased. After the convolution is completed, the inverse wavelet transform is used to re-synthesize the convolution results of each sub-band into a complete output.

[0204]

[0205] Using the above operations, we can restore the low-frequency and high-frequency components of the original image back to the complete image. This process is similar to fusing features at different frequency levels together. It is worth noting that the inverse wavelet transform operation is linear, so the convolution result can be reconstructed to the original space without loss.

[0206] b) Multi-scale variable convolution kernel attention mechanism

[0207] Deformable convolutions enable the free deformation of the resized sampling grid with integer offsets. An additional convolutional layer learns the deformation from the feature map, creating an offset field. The deformation is learned based on the features themselves, resulting in an adaptive convolution kernel. This flexible kernel shape allows for an improved representation of lesions or organ deformations, thereby enhancing the definition of object boundaries. The convolutional layers responsible for computing the offsets follow the kernel size and spread of their corresponding convolutional layers. Bilinear interpolation is used to compute pixel values ​​for offsets not found on the image grid. The MDKA module can be formulated as:

[0208] Attention=Conv1×1(DDW-D-Conv(DDW-Conv(F′))) (36)

[0209]

[0210] The input feature F∈R C×H×W ,Attention∈R C×H×W Represents the attention matrix, each value indicates the relative importance of the corresponding feature. Represents a convolution operation. The multi-scale variable convolution kernel attention mechanism allows the shape and size of the kernel to be adaptively adjusted according to the input features in the attention calculation. This flexibility enables the model to better capture the features of objects of different shapes and scales without the need for additional normalization functions, which tend to ignore high-frequency information, thereby reducing the performance of self-attention-based methods.

[0211] Use MFWTConv to replace the traditional convolution module of the C3K2 module in YOLOv11, and introduce the MDKA module after the C2PSA module in YOLOv11. First, perform initialization training, set epochs=300, imgsz=640, batch=16, then input the image to extract features through the backbone network, and the decoding layer outputs the anchor box prediction. Then use the label to calculate the loss (positioning loss, classification loss, confidence loss), and finally update the weights and adjust the model parameters through the optimizer (SGD). The positioning loss is used to measure the deviation between the predicted box and the real box. The positioning loss used in this invention is as follows:

[0212]

[0213] Where IOU represents the intersection-over-union ratio between the predicted box and the real box. represents the Euclidean distance between the center points of the box, c represents the minimum box diagonal length of the predicted box and the real box, and v represents the aspect ratio consistency term, as shown in formula (39)

[0214]

[0215] α is the weight balancing term, as shown in formula (40)

[0216]

[0217] The classification loss is used to measure the difference between the predicted category distribution and the true category. Assuming that the category probability predicted by the model is The true category is p i : The classification loss formula used in the present invention is as follows:

[0218]

[0219] Confidence loss is used to measure the accuracy of whether the prediction box contains the target. Let the confidence of the prediction box be The confidence of the true box is c (the target box is 1 and the no-target box is 0), and N represents the total number of predicted boxes. The calculation formula is as follows:

[0220]

[0221] The total loss of YOLOv11 is the weighted sum of the above three losses, as shown in formula (43):

[0222]

[0223] where λ loc , cls , conf They are the weight hyperparameters of the positioning loss, classification loss, and confidence loss of the YOLOv11 network respectively.

[0224] After the model training is completed, the optimal weights are loaded and the image or data set is inferred. The detection results are as follows Figure 6 , Figure 7 . Figure 6 The PR curve of the two-stage detection results of tiny defects of transparent components generated based on pseudo labels, Figure 7 Visualization of the two-stage inspection results for tiny defects on transparent components based on pseudo-label generation.

[0225] Example 1

[0226] This embodiment uses transparent lenses as a test platform, and applies the invented two-stage detection method and system for transparent component micro-defects based on pseudo-label generation to the online detection of transparent lenses. A transparent component micro-defect data acquisition system is composed of a wide-field CCD camera, an LED light source, and a backlighting method. Then, our transparent component micro-defect data set is preliminarily acquired through the transparent component micro-defect data acquisition system. Then, a sufficient number of high-resolution transparent component defect images are generated through a multi-scale step-by-step generation framework for transparent component defects to solve the problem of insufficient samples when constructing a training data set. Finally, the constructed data set is trained using a two-stage defect detection network framework based on pseudo-labels, and a supervised defect detection network is trained using pseudo-labels generated by an unsupervised defect detection network, and finally a transparent component micro-defect detection model is obtained. The detected results are as follows. Figure 6 , Figure 7 As shown, Figure 6 The PR curve (precision-recall curve) of the two-stage detection results of transparent component tiny defects generated based on pseudo labels is shown in Figure 1. Figure 7 Visualization of the two-stage inspection results for tiny defects on transparent components based on pseudo-label generation.

[0227] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A two-stage detection system for minute defects of transparent components, comprising an LED light source (2), a wide-field CCD camera (3), a bracket (4), an integrated cable (5) and a computer (6), characterized in that: The LED light source (2) illuminates the transparent element (1) by backlighting, the wide-field CCD camera (3) is mounted on a bracket (4), and the wide-field CCD camera (3) is connected to a computer (6) via an integrated cable (5).

2. A two-stage detection method for tiny defects of transparent components according to claim 1, characterized in that: The specific process is as follows: (1) Design a tiny transparent component image acquisition system and create a tiny transparent component defect dataset; (2) A multi-scale step-by-step generation model for transparent component defects is proposed. The first step is to separate defects and background. The second step is to use the generation network to generate high-quality defect images. The third step is to synthesize the generated high-quality defect images with the background images. (3) A spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network (PBA-Net) is proposed. PBA-Net enhances the feature extraction capability of the PatchCore network for multi-scale defects and its adaptability to complex data through pyramid balanced feature fusion, adaptive dynamic convolution, and convolution instead of average pooling, thereby improving the accuracy and flexibility of unsupervised detection. (4) Generate pseudo labels for various defects for the unsupervised detection network results. The input image is first classified by hierarchical clustering to classify the defects detected by the unsupervised detection network. The classified image is then segmented by adaptive threshold to preliminarily screen out the defective parts. Then, the pseudo labels based on adaptive threshold segmentation are corrected by candidate boxes based on RPN proposal fusion. (5) Using data annotated with pseudo labels, the improved YOLOv11 defect detection network is trained and a method combining multi-frequency wavelet convolution with multi-scale variable convolution kernel attention mechanism is proposed to enhance the accuracy of feature expression.

3. A two-stage detection method for tiny defects of transparent components according to claim 2, characterized in that: The specific process of step (1) is as follows: A transparent component micro-defect image acquisition system is designed. The system consists of three parts: a wide-field CCD camera (3), an LED light source, and a backlighting method. The wide-field CCD camera (3) converts a light signal into a charge signal through the photoelectric effect, and then obtains a digital image signal through charge transfer, signal amplification, and analog-to-digital conversion steps. The LED light source is used to provide lighting. An observable image is generated through the interaction between light and the object being photographed, which directly affects the quality, brightness, contrast, and detail performance of the image. The LED light source meets the conditions that the light brightness is always adjustable, the color is relatively uniform, and it can be illuminated at multiple angles. The LED light source is used as the light source of the system. The backlighting method illuminates the object being photographed by controlling the distribution, intensity, angle, and direction of the light source. The light source is located behind the object and is used to highlight the outline of the object. Images of micro-defects of transparent components are acquired through the transparent component micro-defect image acquisition system, and a transparent component micro-defect data set is established based on these images.

4. A two-stage detection method for tiny defects of transparent components according to claim 2, characterized in that: The specific process of step (2) is as follows: The transparent component micro defect multi-scale step-by-step generation model is used to expand the transparent component micro defect dataset. In the first step, the defect foreground and platform background are separated using the background foreground separation method based on the Otsu threshold method. First, the grayscale histogram of the image is calculated, that is, the number of pixels at each grayscale level, and then the global grayscale mean is calculated, as shown in formula (1): In formula (1), p(i) is the probability of a pixel with gray value i, and N is the total number of pixels in the image; Using the improved background average gray value calculation method, the image is divided into three parts. The first part is the background adjacent to the defect part, the second part is the background not adjacent to the defect part, and the third part is the defect part. The weight calculation formulas of the three parts are shown in equations (2), (3), and (4): In the formula, ω0(T) represents the weight of the background adjacent to the defect, ω1(T) represents the weight of the background not adjacent to the defect, and ω2(T) represents the weight of the defect. The means of the three parts are shown in formulas (5), (6), and (7): Among them, μ0(T) represents the mean of the background adjacent to the defect part, μ1(T) represents the mean of the background not adjacent to the defect part, and μ2(T) represents the mean of the defect part. Combining the weights of the three parts for each possible threshold T, the inter-class variance of the foreground and background is calculated, as shown in formula (8): Where α represents the variance weight of the grayscale value of the background pixels adjacent to the defective part, and β represents the variance weight of the grayscale value of the background pixels not adjacent to the defective part. The value of α should be greater than the value of β to ensure that the contrast between the defective part and the background part is relatively large. The weight, mean, and intra-class variance of the background and foreground under different thresholds T are calculated, and the threshold T that maximizes the inter-class variance is selected. Then, the threshold is used to perform image binarization to extract the background foreground. The second step of the multi-scale step-by-step generation method for transparent component defects is defect image generation based on the GAN network. The GAN network used significantly expands the network size and utilizes more complex training techniques and conditional generation methods. The second step of the multi-scale step-by-step generation model of transparent component defects is an image synthesis method based on smooth transparency transition. First, the defect image and background image and their transparency information are prepared, and then a transparency gradient mask is created. Considering the influence of lighting and environment, the lighting model is introduced to calculate the smooth transparency transition. For each pixel, the synthesized color value C final The calculation formula is as follows: C final =α source ·(C source ·I light )+(1-α source )·(C target ·I background ) (9) Where C source and C target are the color values ​​of the source image and the background image, respectively, light is the illumination intensity calculated by the illumination model, I background is the background illumination or background light intensity, which is used to adjust the brightness of the background part in the composite image. source is the transparency value of the source image.

5. A two-stage detection method for tiny defects of transparent components according to claim 2, characterized in that: The specific process of step (3) is as follows: Step 1: Divide the image into multiple P×P patches; The input image is divided into multiple small patches, each patch contains a part of the image area, and multiple patches are put into a list, called a patch library, as shown in formula (10): P={p1,p2,...,p n } (10) Step 2: For each patchp i , use the pre-trained convolutional neural network CNN to extract its features. The convolutional neural network CNN used is ResNet. The features of each patch are extracted by the pre-trained ResNet. The features extracted by ResNet are divided into three parts: large-scale features, medium-scale features, and small-scale features. After the three different scale features are further extracted by convolution operation, the three features are embedded and spliced ​​for feature fusion. First, the convolution expansion operation is used, that is, the image is divided according to the specified convolution kernel size. This operation converts the input image into multiple local small blocks of feature representation. Assuming that the sizes of the input features x and y are B×C1×H1×W1 and B×C2×H2×W2 respectively, the embedding splicing formula is as follows: Where i, j represent the position in the expanded feature map, m, n represent the rows and columns of the small block, K(p, q) is the convolution kernel, and the expansion operation divides the image into small blocks according to the convolution kernel, and then the image feature x of the convolution expansion operation is unfolded Concatenate y with the beginning and the end, as shown in formula (12): Where x unfolded (b,c1,h,w,n) is the feature of the (c1,h,w)th position of the nth small block of the expanded image x, y(b,c2,h,w) is the feature of the image y at (c2,h,w), and z(b,c,h,w) is the concatenated feature with a size of B×(C1+C2)×H2×W2, where (C1+C2) is the number of channels after concatenation; Step 3: Use the deconvolution operation. After the splicing operation, the spliced ​​result z is folded to restore its original image size H1×W1. The calculation formula is as follows: The image size after deconvolution becomes the size of the original image x, B×(C1+C2)×H1×W1. Here, the features of the three scales are spliced, and the size of the splicing result is B×(C1+C2+C3)×H1×W1, where C1+C2+C3 is the number of channels of the three scale features, and H1×W1 is the height and width of the medium-scale feature. Each patch is mapped to a high-dimensional feature space through the feature extraction network, and these patches are regarded as a feature. The features of all normal image blocks are saved in the feature library; F={f(x j )|x j ∈normalimagepatches} (14) Where F is the feature library, f(x j ) is from the normal image block x j The extracted features,the feature library is a reference library for the subsequent detection stage,,and each element in the feature library is a feature vector of a,trained normal image patch; Step 4: Match the features extracted by the feature extraction module with the features in the feature library; To measure the abnormality of the test image, we need to let the test image pass through the same feature extraction module to extract its patch features, and then compare the extracted patch features with the patch features in the feature library to obtain the abnormality score. test , get each f test And the feature f in the feature library * , as shown in formula (15): Next, use f test The maximum distance to each neighbor in the feature library is used to estimate the anomaly score. The distance here uses the Euclidean distance, as shown in formula (16): D(f test ,f * )=||f test -f * ||2 (16) In the formula, D(f test ,f * ) is used to estimate the feature f of the test image x test The anomaly score of ||·||2 represents the L2 norm. In order to highlight the difference with the test image f test Adjacent feature library features f * The contribution of test ,f * ) is used to calculate the anomaly score corresponding to the test image, as shown in formula (17): Where s represents the abnormality score of the corresponding feature of the test image, and F b Substitute the closest test image feature f test The feature set consists of b patches; Step 5: Draw the abnormal image heat map by testing the abnormal score of each feature of the image.

6. A two-stage detection method for tiny defects of transparent components according to claim 2, characterized in that: The specific process of step (4) is as follows: First, the high-score areas are classified by hierarchical clustering to obtain the category pseudo-labels of the pre-selected defects. The similarity or distance between all defect data is calculated. Then, the two most similar clusters are selected to merge or the least similar cluster is split according to the inter-cluster distance. The inter-cluster distance is recalculated based on the new clusters. The above steps are repeated until the stopping condition is met, where the average distance of all point pairs between the two clusters is used as the distance between the two clusters, as shown in formula (18): Among them C i and C j Represents two different clusters. The three defects obtained after hierarchical clustering are classified into cracks, scratches, and stains according to the defect characteristics and size; Then, the data image is segmented by adaptive threshold to preliminarily screen out the defective parts. Each defect corresponds to a different threshold. The entropy theory based on information theory is used to maximize the sum of the entropy of the foreground and background. The threshold is selected according to the information entropy of the foreground and background in the image grayscale histogram. Assume the grayscale range of the image is [0, L-1], the threshold is T, P i is the probability of gray level i, then the information entropy H is defined as: H(T)=H f (T)+H b (T) (19) In the formula, H f (T) is the foreground entropy, and its calculation formula is: H b (T) is the background entropy, and its calculation formula is: The threshold T is selected to maximize H(T). After calculation, the threshold of the fracture defect is 225, the threshold of the scratch defect is 213, and the threshold of the spot defect is 156. The position of the defect is initially obtained through the adaptive entropy threshold method, and then the position of the defect is corrected through the image edge features extracted by the pre-selected box based on the RPN proposal. Then, the pseudo-label is corrected through the RPN proposal network. First, RPN accepts the feature map output by the backbone network as input, which is usually a high-dimensional feature representation of the input image after several convolutional layers. Then, a 3×3 sliding window is applied to the input feature map for convolution. Each sliding window is responsible for generating a candidate region at a position. A set of anchor boxes is generated at each sliding window position in the feature map. These anchor boxes are a series of rectangular boxes with predefined sizes and aspect ratios. The sizes and aspect ratios are different for different types of defects. The size of the fracture defect is set to [32, 64, 128]. , its aspect ratio is (1,1), (2,1), (1,2), the size of the scratch defect setting is [16,32,64], its aspect ratio is (1,1), (2,1), (1,2), the size of the scratch defect setting is [8,16,32], its aspect ratio is (1,1), (2,1), (1,2). Secondly, the defect positioning box obtained based on the adaptive entropy threshold will be combined with the candidate box of the RPN proposal to filter out the candidate box with correct position and high confidence through the IOU threshold. The calculation formula is as follows: M keep ={m|m∈IoU>T iou } (23) In the formula, B gt represents the real box, B pred represents the prediction box, T iou is the intersection-over-union ratio threshold, M keep Represents the set of pre-selected boxes that meet the conditions, i.e., those with a value greater than the intersection-over-union ratio threshold; Then, the average coordinates of these candidate boxes are taken to obtain the candidate box based on RPN proposal fusion, as shown in formula (24): in, Represents the average value of the horizontal coordinate of the set of points in the upper left corner of the candidate box. Represents the average value of the ordinate of the set of upper left corner points of the candidate box, N represents the number of pre-selected boxes in the set, and the lower right corner point can be obtained similarly; The pseudo-label based on adaptive threshold segmentation is corrected by the candidate box based on RPN proposal fusion. The calculation formula is as follows: where x fianl is the horizontal coordinate of the upper left corner of the corrected positioning frame, y fianl is the vertical coordinate of the upper left corner of the corrected positioning frame, x * is the horizontal coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, y * is the ordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation. α and β are preset values ​​to ensure that the correction result is close to reality. The lower right corner can be obtained similarly. Finally, the anchor box and category obtained from the defect are converted into YOLO format and pseudo-labeled.

7. A two-stage detection method for tiny defects of transparent components according to claim 2, characterized in that: The specific process of step (5) is as follows: The prepared pseudo-label data is input into the supervised defect detection network for training. The real-time performance of the detection network based on RCNN and transformer cannot meet the requirements. The defect detection algorithm based on YOLOv11 is adopted, and ML-Net is proposed, which consists of two parts: multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism. The input image is decomposed into different frequency components through wavelet transform, and small-size convolution is performed on each frequency layer. Finally, the results are recombined through inverse wavelet transform, thereby realizing multi-scale analysis of the image. The MDKA attention mechanism highlights important areas through weighted fusion of feature maps; a) Multi-frequency wavelet convolution; First, MFWTConv uses two-dimensional Haar wavelet transform to perform multi-level decomposition of the input image. Assume that the image is a two-dimensional function f(x, y). Haar wavelet transform uses four filters to decompose the image into four sub-bands. The calculation formula is as follows: Low-frequency component LL: captures low-frequency information of the image, such as overall shape or contour; Horizontal high frequency component LH: captures horizontal edge information in the image; Vertical high frequency component HL: captures vertical edge information in the image; Diagonal high-frequency components HH: capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled, that is, the spatial resolution is halved, but the frequency information is more finely decomposed. Recursively performing wavelet transform is called multi-level decomposition to obtain frequency components at different scales. Use a small convolution kernel of 3x3 or 5x5 to perform convolution operations on each decomposed sub-band. Since the wavelet transform reduces the spatial resolution of each sub-band, a small-sized convolution kernel can cover a larger area of ​​the original image, that is, the receptive field is increased. After completing the convolution, use the inverse wavelet transform to re-synthesize the convolution results of each sub-band into a complete output; By using the above operations, the low-frequency and high-frequency components of the original image are restored to a complete image, and the features of different frequency levels are fused together; b) Multi-scale variable convolution kernel attention mechanism; Deformable convolution enables the sampling grid to be freely deformed with integer offsets. An additional convolutional layer learns the deformation from the feature map to create an offset field. The deformation is learned based on the features themselves to obtain the adaptive convolution kernel. The convolution layer responsible for calculating the offset follows the kernel size and expansion of its corresponding convolution layer. Bilinear interpolation is used to calculate the pixel value of the offset that is not found on the image grid. The MDKA module can be expressed as: Attention=Conv1×1(DDW-D-Conv(DDW-Conv(F′))) (36) The input feature F∈R C×H×W ,Attention∈R C×H×W Represents the attention matrix, each value represents the relative importance of the corresponding feature, and the operator Represents the convolution operation, and the multi-scale variable convolution kernel attention mechanism allows the shape and size of the kernel to be adaptively adjusted according to the input features in the attention calculation; After using MFWTConv to replace the traditional convolution module of the C3K2 module in YOLOv11 and introducing the MDKA module into the C2PSA module in YOLOv11, we first perform initialization training, setting epochs=300, imgsz=640, batch=16, and then input the image to extract features through the backbone network, and the decoder layer outputs the anchor box prediction. Then, we use the label to calculate the loss, and finally update the weights. We use the optimizer SGD to adjust the model parameters. The positioning loss is used to measure the deviation between the predicted box and the real box. The positioning loss used is as follows: Where IOU represents the intersection-over-union ratio between the predicted box and the real box. represents the Euclidean distance between the center points of the boxes, c represents the minimum box diagonal length of the predicted box and the true box, and v represents the aspect ratio consistency term, as shown in formula (39); α is the weight balancing term, as shown in formula (40) The classification loss is used to measure the difference between the predicted category distribution and the true category. Assuming that the category probability predicted by the model is The true category is p i : The classification loss formula used is as follows: Confidence loss is used to measure the accuracy of whether the prediction box contains the target. Let the confidence of the prediction box be The confidence of the true box is c, the target box is 1, and no target is 0. N represents the total number of predicted boxes, and its calculation formula is as follows: The total loss of YOLO is the weighted sum of the above three losses, as shown in formula (43): where λ loc , cls , conf They are the weight hyperparameters of the positioning loss, classification loss, and confidence loss of the YOLOv11 network respectively; After the model training is completed, the optimal weights are loaded and inference is performed on the image or dataset.

Images