A dual-stage detection method and system for tiny defects in transparent components

Abstract

A two-stage detection method and system for tiny defects in transparent components is proposed, and a two-stage defect detection network framework based on pseudo-label generation is designed. The first stage is the pseudo-label generation of transparent component data, which consists of an unsupervised detection method and a pseudo-label extraction strategy. In terms of the unsupervised detection method, a spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network is proposed. This network combines pyramid balanced feature fusion and adaptive dynamic modulation convolution to enhance the unsupervised detection network's feature extraction capabilities for multi-scale targets and its effectiveness and flexibility in dealing with more complex data features. In terms of pseudo-label extraction strategy, a pseudo-label dynamic mining strategy based on threshold segmentation and RPN proposal fusion is proposed, which can directly generate pseudo-labels for multiple defects from the unsupervised network detection results. The second stage is defect detection based on pseudo-labels. The network of this application significantly improves the performance of supervised defect detection networks.

Description

Technical Field

[0001] The present invention belongs to the field of transparent component detection, and in particular relates to a dual-stage detection method and system for tiny defects in transparent components. Background Art

[0002] Transparent components refer to materials or devices with high light transmittance that can maintain the directionality and intensity of light to a certain extent. Due to their transparent properties, such components can be used to transmit light, protect optical systems, or achieve visual visibility, etc. However, due to their high light transmittance, surface or internal defects of transparent components (such as cracks, scratches, stains, etc.) may significantly affect their performance and reliability. Currently, the detection of tiny defects in transparent components mainly relies on manual inspection, which is time-consuming and labor-intensive, and the detection accuracy is not high. With the rapid development of artificial intelligence and deep learning technologies, defect detection based on deep learning has gradually become an effective way to replace traditional machine vision methods. Deep learning models have strong generalization capabilities and can cope with transparent component defect detection in complex environments. Although deep learning technology has achieved remarkable results in the field of defect detection, it still faces many challenges in transparent component defect detection scenarios.

[0003] In actual industrial scenarios, defects in tiny transparent components are rare and difficult to obtain, resulting in insufficient defect samples in the training data set, affecting the accuracy and reliability of the detection model. Traditional data expansion methods are difficult to solve this problem, so data enhancement methods have broad application prospects in this context. Since the defects of transparent components are small and diverse, manual labeling is difficult and easily affected by subjective factors, resulting in inconsistent and inaccurate labeling. It is particularly critical to develop efficient and accurate defect labeling methods. Based on unsupervised pseudo-label generation methods, labels can be automatically generated in the absence of large-scale labeled data, reducing manual intervention and improving labeling efficiency and quality. Transparent components have various types of defects, including scratches, bubbles, impurities, etc. Different defects vary greatly in shape, size, and color characteristics, and some defects have characteristics such as transparency and blurred edges, making it difficult for traditional algorithms to identify and distinguish these complex defect types.

[0004] The present application differs from the prior art in the following ways:

[0005] CN 117197550A VR lens defect detection method and system based on image cube and deep learning

[0006] This invention provides a VR lens defect detection method and system based on image cubes and deep learning. It collects a library of offline defect samples, optimizes the network based on the YOLOV8 network, and constructs a lightweight convolutional neural network. This network is then trained on these samples to generate efficient algorithms for image classification, anchor-free defect detection, and instance segmentation. Each time a defect is detected, the extracted defect slice image is fed into a trained deep learning classification model to output the detection and classification results. This invention can accurately detect and classify subtle defects in VR lens modules of varying thicknesses.

[0007] However, in the actual production process, defective samples are very rare. At the same time, due to the fact that defects in transparent components often interfere with the background, the trained detection model has poor generalization, is prone to overfitting, and cannot identify unknown defects. Summary of the Invention

[0008] This paper studies transparent components. Addressing the challenges of small sample sizes, difficult labeling, and multi-scale defects, this paper proposes a two-stage method for detecting minute defects in transparent components. The paper also designs an image acquisition system for minute transparent components and acquires a dataset of minute defects in these components.

[0009] To achieve the above object, the technical solution adopted by the present invention is:

[0010] A dual-stage detection system for minute defects in transparent components includes an LED light source, a wide-field CCD camera, a bracket, an integrated cable, and a computer. The LED light source illuminates the transparent component through backlighting, the wide-field CCD camera is mounted on the bracket, and the wide-field CCD camera is connected to the computer through an integrated cable.

[0011] As a further improvement to the system of the present invention, the wide-field CCD camera has a resolution of 2448×2048 and a field of view of 10 mm.

[0012] The present invention provides a two-stage detection method for tiny defects in transparent components, the specific process is as follows:

[0013] (1) Design a micro-transparent component image acquisition system and create a micro-transparent component defect dataset;

[0014] (2) A multi-scale step-by-step generation model for transparent component defects is proposed. The first step is to separate the defect and the background. The second step is to generate high-quality defect images using a generative network. The third step is to synthesize the generated high-quality defect images with the background image.

[0015] (3) A spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network (PBA-Net) is proposed. PBA-Net enhances the feature extraction capability of the PatchCore network for multi-scale defects and its adaptability to complex data through pyramid balanced feature fusion, adaptive dynamic convolution, and convolution instead of average pooling, thereby improving the accuracy and flexibility of unsupervised detection.

[0016] (4) Generate pseudo labels for various defects based on the unsupervised detection network results. The input image is first classified by hierarchical clustering to classify the defects detected by the unsupervised detection network. The classified image is then segmented by adaptive threshold to preliminarily screen out the defective parts. The pseudo labels based on adaptive threshold segmentation are then corrected by candidate boxes based on RPN proposal fusion.

[0017] (5) Using data annotated with pseudo labels, the improved YOLOv11 defect detection network is trained, and a method combining multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism is proposed to enhance the accuracy of feature expression.

[0018] As a further improvement of the method of the present invention, the specific process of step (1) is as follows:

[0019] A transparent component micro-defect image acquisition system is designed. The system consists of three parts: a wide-field CCD camera (3), an LED light source, and a backlighting method. The wide-field CCD camera (3) converts light signals into charge signals through the photoelectric effect, and then obtains a digital image signal through charge transfer, signal amplification, and analog-to-digital conversion steps. The LED light source is used to provide illumination. An observable image is generated through the interaction between light and the object being photographed, which directly affects the quality, brightness, contrast, and detail of the image. The LED light source meets the conditions of constant, adjustable light brightness, relatively uniform color, and multi-angle illumination. The LED light source is used as the light source of the system. The backlighting method illuminates the object being photographed by controlling the distribution, intensity, angle, and direction of the light source. The light source is located behind the object and is used to highlight the outline of the object. Images of micro-defects of transparent components are acquired through the transparent component micro-defect image acquisition system, and a transparent component micro-defect data set is established based on these images.

[0020] As a further improvement of the method of the present invention, the specific process of step (2) is as follows:

[0021] The transparent component micro defect multi-scale step-by-step generation model is used to expand the transparent component micro defect dataset. In the first step, the defect foreground and platform background are separated using the background foreground separation method based on the Otsu threshold method. First, the grayscale histogram of the image is calculated, that is, the number of pixels at each grayscale level, and then the global grayscale mean is calculated, as shown in formula (1):

[0022]

[0023] In formula (1), p(i) is the probability of a pixel with gray value i, and N is the total number of pixels in the image;

[0024] Using the improved background average gray value calculation method, the image is divided into three parts. The first part is the background adjacent to the defect part, the second part is the background not adjacent to the defect part, and the third part is the defect part. The weight calculation formulas of the three parts are shown in formulas (2), (3), and (4):

[0025]

[0026]

[0027] ω2(T)=1-ω0(T)-ω1(T) (4)

[0028] Where ω0(T) represents the weight of the background adjacent to the defect, ω1(T) represents the weight of the background not adjacent to the defect, and ω2(T) represents the weight of the defect. The means of the three parts are shown in Equations (5), (6), and (7):

[0029]

[0030] Among them, μ0(T) represents the mean of the background adjacent to the defect part, μ1(T) represents the mean of the background not adjacent to the defect part, and μ2(T) represents the mean of the defect part. Combining the weights of the three parts for each possible threshold T, the inter-class variance of the foreground and background is calculated, as shown in formula (8):

[0031]

[0032] Where α represents the variance weight of the grayscale values ​​of the background pixels adjacent to the defect, and β represents the variance weight of the grayscale values ​​of the background pixels not adjacent to the defect. The value of α should be greater than the value of β to ensure that the contrast between the defect and the background is relatively large. The weights, means, and intra-class variances of the background and foreground are calculated under different thresholds T. The threshold T that maximizes the inter-class variance is selected, and then the threshold is used to perform image binarization to extract the background and foreground.

[0033] The second step of the multi-scale, step-by-step generation method for transparent component defects is defect image generation based on a GAN network. The GAN network used significantly expands the network size and utilizes more complex training techniques and conditional generation methods.

[0034] The second step of the multi-scale step-by-step generation model of transparent component defects is an image synthesis method based on transparency smooth transition. First, the defect image and background image and their transparency information are prepared. Then, a transparency gradient mask is created. Considering the influence of lighting and environment, the lighting model is introduced to calculate the transparency smooth transition. For each pixel, the synthesized color value C final The calculation formula is as follows:

[0035] C final =α source ·(C source I light )+(1-α source )·(C target I background ) (9)

[0036] Where C source and C target are the color values ​​of the source image and background image, I light is the illumination intensity calculated by the illumination model, I background is the background illumination or background light intensity, which is used to adjust the brightness of the background part in the composite image. source is the transparency value of the source image.

[0037] As a further improvement of the method of the present invention, the specific process of step (3) is as follows:

[0038] Step 1: Divide the image into multiple P×P patches;

[0039] The input image is divided into multiple small patches, each patch contains a part of the image area, and multiple patches are put into a list, called a patch library, as shown in formula (10):

[0040] P={p1,p2,...,p n} (10)

[0041] Step 2: For each patchp i, use the pre-trained convolutional neural network CNN to extract its features, the convolutional neural network CNN used is ResNet, and the features of each patch are extracted by the pre-trained ResNet. The features extracted by ResNet are divided into three parts: large-scale features, medium-scale features, and small-scale features. After the three different scale features are further extracted by convolution operation, the three features are embedded and spliced ​​for feature fusion. First, the convolution expansion operation is used, that is, the image is divided according to the specified convolution kernel size. This operation converts the input image into multiple local small block feature representations. Assuming that the sizes of the input features x and y are B×C1×H1×W1 and B×C2×H2×W2 respectively, the embedding splicing formula is as follows:

[0042]

[0043] Where i, j represent the position in the expanded feature map, m, n represent the rows and columns of the small block, K(p,q) is the convolution kernel, and the expansion operation divides the image into small blocks according to the convolution kernel, and then the image feature x of the convolution expansion operation is unfolded Concatenate y with the end of y, as shown in formula (12):

[0044]

[0045] Where x unfolded (b,c1,h,w,n) is the feature of the expanded image x at the (c1,h,w)th position in the nth small block, y(b,c2,h,w) is the feature of the image y at (c2,h,w), and z(b,c,h,w) is the concatenated feature, with a size of B×(C1+C2)×H2×W2, where (C1+C2) is the number of channels after concatenation;

[0046] Step 3: Use the deconvolution operation. After the splicing operation, the spliced ​​result z is folded to restore its original image size H1×W1. The calculation formula is as follows:

[0047]

[0048] The image size after deconvolution becomes the size of the original image x, B×(C1+C2)×H1×W1. Here, the features of the three scales are spliced, and the size of the spliced ​​result is B×(C1+C2+C3)×H1×W1, where C1+C2+C3 is the number of channels of the three scale features, and H1×W1 is the height and width of the medium-scale feature. Each patch is mapped to a high-dimensional feature space through the feature extraction network, and these patches are regarded as a feature. The features of all normal image blocks are saved in the feature library;

[0049] F={f(xj )|x j ∈normalimagepatches} (14)

[0050] Where F is the feature library, f(x j ) is from the normal image block x j The extracted features,the feature library is a reference library for the subsequent detection stage,,and each element in the feature library is a feature vector of a trained,normal image block;

[0051] Step 4: Match the features extracted by the feature extraction module with the features in the feature library;

[0052] To measure the abnormality of the test image, it is necessary to let the test image pass through the same feature extraction module to extract its patch features, and then compare the extracted patch features with the patch features in the feature library to obtain the abnormality score. test , get each f test and the feature f in the feature library * , as shown in formula (15):

[0053]

[0054] Next, use f test The maximum distance to each neighbor in the feature library is used to estimate the anomaly score. The distance here uses the Euclidean distance, as shown in formula (16):

[0055] D(f test ,f * )=||f test -f * ||2 (16)

[0056] Where, D(f test ,f * ) is used to estimate the features f of the test image x test The anomaly score of ||·||2 represents the L2 norm. In order to highlight the difference between the test image f test Adjacent feature library features f * The contribution of D(f test ,f * ) to calculate the anomaly score corresponding to the test image, as shown in formula (17):

[0057]

[0058] Where s represents the abnormality score of the corresponding feature of the test image, F b The closest substitute test image feature f testThe feature set consists of b patches;

[0059] Step 5: Draw an abnormal image heat map by testing the abnormal score of each feature of the image.

[0060] As a further improvement of the method of the present invention, the specific process of step (4) is as follows:

[0061] First, the high-scoring areas are classified by hierarchical clustering to obtain pseudo-labels of pre-selected defects. The similarity or distance between all defect data is calculated. Then, the two most similar clusters are selected to merge or the least similar cluster is split according to the inter-cluster distance. The inter-cluster distance is recalculated based on the new clusters. The above steps are repeated until the stopping condition is met. The average distance of all point pairs between the two clusters is used as the distance between the two clusters, as shown in Equation (18):

[0062]

[0063] Among them C i and C j Representing two different clusters. The three defects obtained through hierarchical clustering are classified into cracks, scratches, and stains based on defect characteristics and size;

[0064] Then, the data image is segmented using adaptive thresholding to preliminarily screen out defects. Each defect corresponds to a different threshold. The entropy theory based on information theory is used to maximize the sum of the foreground and background entropy. The threshold is selected based on the information entropy of the foreground and background in the image grayscale histogram.

[0065] Assume the grayscale range of the image is [0, L-1], the threshold is T, P i is the probability of gray level i, then the information entropy H is defined as:

[0066] H(T)=H f (T)+H b (T) (19)

[0067] Where H f (T) is the foreground entropy, which is calculated as follows:

[0068]

[0069] H b (T) is the background entropy, which is calculated as follows:

[0070]

[0071] The threshold T is selected to maximize H(T). After calculation, the threshold for fracture defects is 225, the threshold for scratch defects is 213, and the threshold for spot defects is 156. The location of the defect is initially obtained through the adaptive entropy threshold method. Then, the location of the defect is corrected using the image edge features extracted from the pre-selected box based on the RPN proposal.

[0072] The pseudo-labels are then corrected using the RPN proposal network. First, the RPN accepts the feature map output by the backbone network as input, which is usually a high-dimensional feature representation of the input image after passing through several convolutional layers. A 3×3 sliding window is then applied to the input feature map for convolution. Each sliding window is responsible for generating a candidate region at a position. A set of anchor boxes is generated at each sliding window position in the feature map. These anchor boxes are a series of rectangular boxes with predefined sizes and aspect ratios. The sizes and aspect ratios are different for different types of defects. The size of the fracture defect is set to [32, 64, 128]. , its aspect ratio is (1,1), (2,1), (1,2), the size of the scratch defect setting is [16,32,64], its aspect ratio is (1,1), (2,1), (1,2), the size of the scratch defect setting is [8,16,32], its aspect ratio is (1,1), (2,1), (1,2). Secondly, the defect positioning box obtained based on the adaptive entropy threshold will be combined with the candidate box of the RPN proposal to filter out the candidate box with correct position and high confidence through the IOU threshold. The calculation formula is as follows:

[0073]

[0074] Where B gt represents the real box, B pred Represents the prediction box, T iou is the intersection-over-union ratio threshold, M keep Represents the set of pre-selected boxes that meet the conditions, that is, those with a value greater than the intersection-over-union ratio threshold;

[0075] Then, the average coordinates of these candidate boxes are taken to obtain the candidate box based on RPN proposal fusion, as shown in formula (24):

[0076]

[0077] in, Represents the average value of the horizontal coordinate of the set of points in the upper left corner of the candidate box, Represents the average value of the vertical coordinate of the set of upper left corner points of the candidate box, N represents the number of pre-selected boxes in the set, and the lower right corner point can be obtained similarly;

[0078] The pseudo-label based on adaptive threshold segmentation is corrected by the candidate box based on RPN proposal fusion. The calculation formula is as follows:

[0079]

[0080] where x fianl is the horizontal coordinate of the upper left corner of the positioning frame after correction, y fianl is the vertical coordinate of the upper left corner of the corrected positioning frame, x * is the horizontal coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, y * is the vertical coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation. α and β are preset values ​​to ensure that the correction result is close to reality. The lower right corner can be obtained similarly. Finally, the anchor box and category obtained from the defect are converted into YOLO format and pseudo-labeled.

[0081] As a further improvement of the method of the present invention, the specific process of step (5) is as follows:

[0082] The prepared pseudo-labeled data is input into the supervised defect detection network for training. The real-time performance of the detection networks based on RCNN and transformer cannot meet the requirements. Therefore, a defect detection algorithm based on YOLOv11 is adopted and ML-Net is proposed. It consists of two parts: multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism. The input image is decomposed into different frequency components through wavelet transform, and small-size convolution is performed on each frequency layer. Finally, the results are recombined through inverse wavelet transform, thus realizing multi-scale analysis of the image. The MDKA attention mechanism highlights important areas through weighted fusion of feature maps.

[0083] a) Multi-frequency wavelet convolution;

[0084] First, MFWTConv uses two-dimensional Haar wavelet transform to perform multi-level decomposition of the input image. Let the image be a two-dimensional function f(x, y). The Haar wavelet transform uses four filters to decompose the image into four sub-bands. The calculation formula is as follows:

[0085]

[0086] Low-frequency component LL: captures low-frequency information of the image, such as overall shape or outline;

[0087]

[0088] Horizontal high frequency component LH: captures horizontal edge information in the image;

[0089]

[0090] Vertical high-frequency component HL: captures vertical edge information in the image;

[0091]

[0092] Diagonal high-frequency components HH: capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled, that is, the spatial resolution is halved, but the frequency information is decomposed more finely. Recursively performing wavelet transform is called multi-level decomposition to obtain frequency components at different scales. A small convolution kernel of 3x3 or 5x5 is used to perform convolution operations on each decomposed subband. Since the wavelet transform reduces the spatial resolution of each subband, the small convolution kernel can cover a larger area of ​​the original image, that is, the receptive field is increased. After the convolution is completed, the inverse wavelet transform is used to re-synthesize the convolution results of each subband into a complete output;

[0093]

[0094] By using the above operations, the low-frequency and high-frequency components of the original image are restored to a complete image, and the features of different frequency levels are fused together;

[0095] b) Multi-scale variable convolution kernel attention mechanism;

[0096] Deformable convolution enables the sampling grid to be freely deformed with integer offsets. An additional convolutional layer learns the deformation from the feature map to create an offset field. The deformation is learned based on the features themselves to obtain the adaptive convolution kernel. The convolution layer responsible for calculating the offset follows the kernel size and expansion of its corresponding convolution layer. Bilinear interpolation is used to calculate the pixel value of the offset that is not found on the image grid. The MDKA module can be expressed as:

[0097] Attention=Conv1×1(DDW-D-Conv(DDW-Conv(F′))) (36)

[0098]

[0099] Where the input feature F∈R C×H×W , Attention∈R C×H×W Represents the attention matrix, each value represents the relative importance of the corresponding feature, the operator Represents the convolution operation, and the multi-scale variable convolution kernel attention mechanism allows the shape and size of the kernel to be adaptively adjusted according to the input features in the attention calculation;

[0100] After replacing the traditional convolutional module in YOLOv11's C3K2 module with MFWTConv and introducing the MDKA module into YOLOv11's C2PSA module, we first perform initial training with epochs=300, imgsz=640, and batch=16. The input image is then passed through the backbone network to extract features, and the decoder layer outputs anchor box predictions. The loss is then calculated using the labels, and finally the weights are updated. The model parameters are adjusted using the SGD optimizer. The localization loss is used to measure the deviation between the predicted and true boxes. The localization loss used is as follows:

[0101]

[0102] Where IOU represents the intersection-union ratio between the predicted box and the real box. represents the Euclidean distance between the center points of the boxes, c represents the minimum box diagonal length of the predicted box and the true box, and v represents the aspect ratio consistency term, as shown in formula (39);

[0103]

[0104] α is the weight balance term, as shown in formula (40)

[0105]

[0106] Classification loss is used to measure the difference between the predicted category distribution and the true category. Assume that the category probability predicted by the model is The true category is p i : The classification loss formula used is as follows:

[0107]

[0108] Confidence loss is used to measure the accuracy of whether the prediction box contains the target. Let the confidence of the prediction box be The confidence of the true box is c, the target box is 1, and no target is 0. N represents the total number of predicted boxes, and its calculation formula is as follows:

[0109]

[0110] The total loss of YOLO is the weighted sum of the above three losses, as shown in formula (43)

[0111]

[0112] where λ loc ,λ cls ,λ conf They are the weight hyperparameters of the positioning loss, classification loss, and confidence loss of the YOLOv11 network respectively;

[0113] After the model training is completed, the optimal weights are loaded and inference is performed on the image or dataset.

[0114] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0115] (1) The present invention designs a multi-scale step-by-step generation model of transparent component defects based on the GAN network to separate the defects through the foreground-background separation method, and then generates a realistic defect image through a high-resolution GAN image generation network, and then synthesizes the generated defect image and the background image through a smooth transparency transition.

[0116] (2) This paper proposes a two-stage defect detection network framework based on pseudo-labels. This framework consists of a pseudo-label generation network in the first stage and a supervised network trained with pseudo-label data in the second stage. This framework exhibits stronger adaptability and high generalization ability when facing unknown defects, achieving accurate detection of tiny defects in transparent components.

[0117] (3) The present invention constructs a spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network (PBA-Net), which combines pyramid balanced feature fusion and adaptive dynamic modulation convolution to enhance PatchCore's feature extraction capability for multi-scale targets and its effectiveness and flexibility in dealing with more complex data features.

[0118] (4) This paper proposes a pseudo-label dynamic mining strategy based on the fusion of threshold segmentation and RPN proposal, which can directly generate pseudo-labels of various defects from the unsupervised network detection results.

[0119] (5) The present invention introduces a new network based on wavelet transform and multi-scale attention, which combines multi-frequency wavelet convolution and variable large kernel attention mechanism to significantly improve the performance of supervised defect detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] Figure 1 This is a diagram of a system for detecting minute defects in transparent components according to the present invention;

[0121] In the figure, 1. transparent component; 2. LED light source; 3. wide-field CCD camera; 4. bracket; 5. integrated cable; 6. computer;

[0122] Figure 2 This is a flow chart of the dual-stage detection of tiny defects in transparent components based on pseudo-label generation according to the present invention;

[0123] Figure 3 It is an original image of a tiny defect of the transparent element of the present invention;

[0124] Figure 4It is an image generated by the multi-scale step-by-step generation model of transparent element defects of the present invention;

[0125] Figure 5 The present invention generates pseudo labels of various defects by using a dynamic mining strategy based on pseudo labels;

[0126] Figure 6 This is the PR curve of the two-stage detection result of the transparent component tiny defect generated by the present invention based on the pseudo label;

[0127] Figure 7 The present invention visualizes the results of the dual-stage detection of tiny defects in transparent components based on pseudo-label generation. DETAILED DESCRIPTION

[0128] The following is a detailed description of the technical solution of the application in conjunction with the accompanying drawings. The described embodiments are only part of the embodiments involved in this patent. All non-innovative embodiments based on this embodiment by other researchers in this field fall within the scope of protection of this patent.

[0129] This paper proposes a two-stage detection method and system for small defects in transparent components based on pseudo-label generation. Figure 1 It is a transparent component micro-defect system, which includes an LED light source 2, a wide-field-of-view CCD camera 3, a bracket 4, an integrated cable 5 and a computer 6. The LED light source 2 illuminates the transparent component 1 through backlighting, the wide-field-of-view CCD camera 3 is installed on the bracket 4, and the wide-field-of-view CCD camera 3 is connected to the computer 6 through the integrated cable 5.

[0130] Figure 2 The present invention proposes a two-stage detection method for tiny defects of transparent components based on pseudo-label generation. Figure 3 This is the original image of the tiny defects of the transparent element of the present invention. The specific implementation scheme is as follows:

[0131] (1) Design of a system for capturing images of tiny defects in transparent components

[0132] The present invention studies tiny defects in transparent components. Due to their reflective nature, tiny defects, and complex environments, capturing images of these defects is difficult. Therefore, to capture images of these defects and construct a dataset for these defects, the present invention designs a system for capturing images of these defects. The system consists of a wide-field-of-view CCD camera, an LED light source, and a backlighting system. The wide-field-of-view CCD camera is the core of the system, converting light signals into charge signals through the photoelectric effect. This system then generates digitized image signals through charge transfer, signal amplification, and analog-to-digital conversion. The CCD camera has a resolution of 2448×2048 and a field of view of 10 mm, which allows it to capture the entire component. The second part is the LED light source. The main function of the light source is to provide illumination. Through the interaction of light and the photographed object, an observable image is produced. It directly affects the image quality, brightness, contrast, and detail. The LED light source meets the conditions of consistent and adjustable light brightness, relatively uniform color, and the ability to illuminate from multiple angles. Therefore, the LED light source is used as the light source of the system. The third part is the backlighting method. Lighting refers to illuminating the photographed object by controlling the distribution, intensity, angle, and direction of the light source to ensure that the imaging effect achieves the desired goal. Different lighting methods can greatly affect the image quality, detail presentation, contrast, and aesthetic effect. Backlighting refers to the light source located behind the object. It is mainly used to highlight the outline of the object and is suitable for extracting the edge of the object being measured and detecting various impurities in the object being measured. Therefore, the backlighting method is selected as the lighting method. Images of small defects in transparent components are collected through the transparent component small defect image acquisition system, and these images are used to establish a transparent component small defect dataset.

[0133] (2) Multi-scale step-by-step generation model for transparent component defects

[0134] In actual industrial scenarios, defective components often have problems such as small number and difficulty in obtaining. Transparent component defect images lack large-scale data, and the small sample size may lack representativeness of the whole, which makes the estimate calculated based on the sample lack effectiveness and low accuracy. Therefore, the present invention adopts a multi-scale step-by-step generation model of transparent component micro-defects to expand the transparent component micro-defect data set. Since the transparent component micro-defect images have the problems of low contrast between defects and background and small proportion of defective parts, directly generating data through transparent component micro-defects will cause edge distortion and fusion of background and foreground, and will also cause serious waste of computing resources. Therefore, the first step of the multi-scale step-by-step generation model of transparent component defects is to use the background foreground separation method based on the Otsu threshold method (OTSU) to separate the defect foreground and the platform background. First, calculate the grayscale histogram of the image, that is, the number of pixels at each grayscale level, and then calculate the global grayscale mean, as shown in formula (1):

[0135]

[0136] In formula (1), p(i) is the probability of a pixel with gray value i, and N is the total number of pixels in the image.

[0137] Since our goal is to increase the contrast between the defect part and the background part, especially the grayscale value of the background pixels adjacent to the defect part should be significantly different from that of the defect part, we use the improved background average grayscale value calculation method to divide the image into three parts. The first part is the background adjacent to the defect part, the second part is the background not adjacent to the defect part, and the third part is the defect part. The weight calculation formulas of the three parts are shown in Equations (2), (3), and (4):

[0138]

[0139] ω2(T)=1-ω0(T)-ω1(T) (4)

[0140] In the formula, ω0(T) represents the weight of the background adjacent to the defect, ω1(T) represents the weight of the background not adjacent to the defect, and ω2(T) represents the weight of the defect. The means of the three parts are shown in formulas (5), (6), and (7):

[0141]

[0142] Among them, μ0(T) represents the mean of the background adjacent to the defect part, μ1(T) represents the mean of the background not adjacent to the defect part, and μ2(T) represents the mean of the defect part. Combining the weights of the three parts for each possible threshold T, the inter-class variance of the foreground and background is calculated, as shown in formula (8):

[0143]

[0144] Where α represents the variance weight of the grayscale values ​​of the background pixels adjacent to the defective part, and β represents the variance weight of the grayscale values ​​of the background pixels not adjacent to the defective part. Generally speaking, the value of α should be greater than the value of β to ensure that the contrast between the defective part and the background part is relatively large. The weights, means, and intra-class variances of the background and foreground are calculated under different thresholds T. The threshold T that maximizes the inter-class variance is selected, and then the threshold is used to binarize the image to extract the background and foreground.

[0145] The second step of the multi-scale step-by-step generation method for transparent component defects is defect image generation based on the GAN network. The GAN network used in the present invention significantly improves the quality of the generated images by significantly expanding the network scale and utilizing more complex training techniques and conditional generation methods, and can generate higher resolution and more detailed images.

[0146] The second step of the multi-scale step-by-step generation model of transparent component defects is an image synthesis method based on transparency smooth transition. First, the defect image and background image and their transparency information are prepared. Then, a transparency gradient mask is created. Considering the influence of lighting and environment, the lighting model is introduced to calculate the transparency smooth transition. For each pixel, the synthesized color value C final The calculation formula is as follows:

[0147] C final =α source ·(C source I light )+(1-α source )·(C target I background ) (9)

[0148] Where C source and C target are the color values ​​of the source image and background image respectively. light is the illumination intensity calculated by the illumination model (including ambient light, diffuse light, specular light, etc.), I background is the background illumination or background light intensity, which is used to adjust the brightness of the background part in the composite image. source is the transparency value of the source image. The above method can smoothly synthesize the defect image and the background image. The result of the synthesis is as follows: Figure 4 shown.

[0149] (3) Use PatchCore improved based on PBA-Net to identify abnormalities in normal samples.

[0150] Step 1: Divide the image into multiple P×P patches

[0151] The input image is divided into multiple small blocks (patches), each patch contains a part of the image area. This operation can help the model focus on the local information of the image. We put multiple patches into a list, called a patch library, as shown in formula (10):

[0152] P={p1,p2,...,p n} (10)

[0153] Step 2: For each patchp i, we use a pre-trained convolutional neural network (CNN) to extract its features. The convolutional neural network (CNN) used in the present invention is ResNet. The features of each patch are extracted through the pre-trained ResNet. To solve the problem of difficulty in extracting multi-scale defects, the present invention proposes a pyramid balanced feature fusion, which divides the features extracted by ResNet into three parts: large-scale features, medium-scale features, and small-scale features. After the three different scale features are further extracted through convolution operations, the three features are embedded and spliced ​​for feature fusion. First, the convolution expansion operation is used, that is, the image is divided according to the specified convolution kernel size. This operation converts the input image into multiple local small block feature representations. Assuming that the sizes of the input features x and y are B×C1×H1×W1 and B×C2×H2×W2 respectively, the embedding and splicing formulas are as follows:

[0154]

[0155] Where i, j represent the position in the expanded feature map, m, n represent the rows and columns of the small block, K(p,q) is the convolution kernel, and the expansion operation divides the image into small blocks according to the convolution kernel. unfolded Concatenate y with the end of y, as shown in formula (12):

[0156]

[0157] Where x unfolded (b,c1,h,w,n) is the feature of the (c1,h,w)th position of the nth block of the expanded image x, y(b,c2,h,w) is the feature of the image y at (c2,h,w), and z(b,c,h,w) is the concatenated feature with a size of B×(C1+C2)×H2×W2, where (C1+C2) is the number of channels after concatenation.

[0158] Step 3: Using the deconvolution operation, after the splicing operation, we fold the spliced ​​result z to restore its original image size H1×W1. The calculation formula is as follows:

[0159]

[0160] After deconvolution, the image size becomes the original image x, with dimensions B × (C1 + C2) × H1 × W1. Here, the features at the three scales are concatenated, resulting in a concatenated size of B × (C1 + C2 + C3) × H1 × W1, where C1 + C2 + C3 represents the number of channels for the three scale features, and H1 × W1 represents the height and width of the medium-scale feature. Each patch is mapped into a high-dimensional feature space through a feature extraction network. These patches are considered as features, and the features of all normal image patches are stored in a feature library.

[0161] F={f(x j )|x j ∈normalimagepatches} (14)

[0162] Where F is the feature library, f(x j ) is from the normal image block x j Extracted features. The feature library is a reference library used in the subsequent detection stage. Each element in the feature library is a feature vector of a trained normal image block.

[0163] Step 4: Match the features extracted by the feature extraction module with the features in the feature library. To measure the abnormality of the test image, the test image needs to be passed through the same feature extraction module to extract its patch features, and then the extracted patch features are compared with the patch features in the feature library to obtain the abnormality score. test , we obtain each f test and the feature f in the feature library * , as shown in formula (15):

[0164]

[0165] Next we use f test The maximum distance to each neighbor in the feature library is used to estimate the anomaly score. Here we use the Euclidean distance, as shown in formula (16):

[0166] D(f test ,f * )=||f test -f * ||2 (16)

[0167] Where, D(f test ,f * ) is used to estimate the features f of the test image x test The anomaly score of ||·||2 represents the L2 norm. In order to highlight the difference between the test image f test Adjacent feature library features f *contribution, we scale D(f test ,f * ) to calculate the anomaly score corresponding to the test image, as shown in formula (17):

[0168]

[0169] Where s represents the abnormality score of the corresponding feature of the test image, F b The closest substitute test image feature f test The feature set consists of b patches.

[0170] Step 5: Draw an abnormal image heat map by testing the abnormal score of each feature of the image.

[0171] (4) Generate pseudo labels for various defects through a pseudo label dynamic mining strategy based on the fusion of adaptive entropy threshold and RPN proposal.

[0172] First, the regions with high scores are classified using a hierarchical clustering method to obtain pseudo labels for the preselected defect categories. The similarity or distance between all defect data is calculated. Then, the two most similar clusters are selected based on the inter-cluster distance to be merged (or the least similar cluster is split). The inter-cluster distance is recalculated based on the new clusters. The above steps are repeated until the stopping condition is reached. The average distance of all point pairs between the two clusters is used as the distance between the two clusters, as shown in Equation (18):

[0173]

[0174] Among them C i and C j Representing two distinct clusters. The three defects identified through hierarchical clustering are categorized into fracture, scratch, and spot based on their characteristics and size. The data image is then segmented using adaptive thresholding to initially identify defects (each defect corresponds to a different threshold). Here, we use entropy theory based on information theory to maximize the sum of foreground and background entropy. The threshold is selected based on the information entropy of the foreground and background in the image's grayscale histogram.

[0175] Assume the grayscale range of the image is [0, L-1], the threshold is T, P i is the probability of gray level i, then the information entropy H is defined as:

[0176] H(T)=H f (T)+H b (T) (19)

[0177] Where H f (T) is the foreground entropy, which is calculated as follows:

[0178]

[0179] H b (T) is the background entropy, which is calculated as follows:

[0180]

[0181] The threshold T is selected to maximize H(T). After calculation, the threshold for fracture defects is 225, the threshold for scratch defects is 213, and the threshold for spot defects is 156. The location of the defect is initially obtained through the adaptive entropy threshold method. Then, the location of the defect is corrected using the image edge features extracted from the pre-selected box based on the RPN proposal.

[0182] The pseudo-labels are then repositioned using the RPN proposal network. First, the RPN accepts the feature map output by the backbone network (such as ResNet or VGG) as input. This is typically a high-dimensional feature representation of the input image after passing through several convolutional layers. A 3×3 sliding window is then convolved on the input feature map, with each sliding window responsible for generating a candidate region at a specific location. A set of anchor boxes is generated at each sliding window position in the feature map. These anchor boxes are rectangular boxes with predefined sizes and aspect ratios. Different types of defects have different sizes and aspect ratios. In this invention, the sizes of the fracture defects are set to [32, 64, 128], and their aspect ratios are (1, 1), (2, 1), (1, 2). The sizes of the scratch defects are set to [16, 32, 64], and their aspect ratios are (1, 1), (2, 1), (1, 2). The sizes of the scratch defects are set to [8, 16, 32], and their aspect ratios are (1, 1), (2, 1), (1, 2).

[0183] Secondly, the defect positioning box obtained based on the adaptive entropy threshold will be combined with the candidate box of the RPN proposal and the IOU threshold will be used to filter out the candidate box with the correct position and high confidence. The calculation formula is as follows:

[0184]

[0185] Where B gt represents the real box, B pred Represents the prediction box, T iou is the intersection-over-union ratio threshold, M keep Represents a set of pre-selected boxes that meet the conditions (greater than the intersection-over-union threshold).

[0186] Then, the average coordinates of these candidate boxes are taken to obtain the candidate box based on RPN proposal fusion, as shown in formula (24):

[0187]

[0188] in, Represents the average value of the horizontal coordinate of the set of points in the upper left corner of the candidate box, Represents the average value of the vertical coordinate of the set of upper left corner points of the candidate box, N represents the number of pre-selected boxes in the set, and the lower right corner point can be obtained similarly.

[0189] The pseudo labels based on adaptive threshold segmentation are corrected by the candidate boxes based on RPN proposal fusion. The calculation formula is as follows:

[0190]

[0191] where x fianl is the horizontal coordinate of the upper left corner of the positioning frame after correction, y fianl is the vertical coordinate of the upper left corner of the corrected positioning frame, x * is the horizontal coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, y * is the vertical coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation. α and β are preset values ​​to ensure that the correction result is close to reality. The lower right corner can be obtained in the same way. Finally, the anchor box and category obtained by the defect are converted into YOLO format and pseudo-labeled. The result is as follows Figure 5 shown.

[0192] (5) The pseudo-labeled data is trained through the ultralytics defect detection network improved based on ML-Net;

[0193] The prepared pseudo-label data is input into the supervised defect detection network for training. The real-time performance of the detection network based on RCNN and transformer cannot meet the requirements, so the defect detection algorithm based on YOLOv11 is currently used. The defect detection algorithm based on YOLOv11 currently has the problems of insensitivity to multi-scale defects and weak perception of small defect features. Therefore, the present invention proposes ML-Net, which consists of two parts: multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism. The input image is decomposed into different frequency components through wavelet transform, and small-size convolution is performed on each frequency layer. Finally, the results are recombined through inverse wavelet transform, thereby realizing multi-scale analysis of the image. The MDKA attention mechanism highlights important areas through weighted fusion of feature maps. This weighted method is based on the output of the deformable kernel and can focus on key areas (such as small target features) more accurately.

[0194] b) Multi-frequency wavelet convolution

[0195] First, MFWTConv uses the two-dimensional Haar wavelet transform to perform multi-level decomposition on the input image. Let the image be a two-dimensional function f(x,y). The Haar wavelet transform uses four filters to decompose the image into four sub-bands. Its calculation formula is as follows:

[0196]

[0197] Low-frequency components (LL): capture low-frequency information of an image, such as overall shape or outline.

[0198]

[0199] Horizontal high frequency component (LH): captures horizontal edge information in the image.

[0200]

[0201] Vertical high frequency component (HL): captures vertical edge information in the image.

[0202]

[0203] Diagonal high frequency components (HH): capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled (the spatial resolution is halved), but the frequency information is decomposed more finely. Recursively performing wavelet transform (called multi-level decomposition) can obtain frequency components at different scales. The present invention uses a small convolution kernel of 3x3 or 5x5 to perform convolution operations on each decomposed sub-band. Since the wavelet transform reduces the spatial resolution of each sub-band, the small-sized convolution kernel can cover a larger area of ​​the original image, that is, the receptive field is increased. After the convolution is completed, the inverse wavelet transform is used to re-synthesize the convolution results of each sub-band into a complete output.

[0204]

[0205] Using the above operations, we can restore the original image from its decomposed low-frequency and high-frequency components. This process is similar to fusing features from different frequency levels. It's worth noting that the inverse wavelet transform operation is linear, so the convolution result can be losslessly reconstructed into the original space.

[0206] b) Multi-scale variable convolution kernel attention mechanism

[0207] Deformable convolutions enable the resampling grid to be freely deformed with integer offsets. An additional convolutional layer learns the deformation from the feature map, creating an offset field. The deformation is learned based on the features themselves, resulting in an adaptive convolution kernel. This flexible kernel shape improves the representation of lesions or organ deformations, thereby enhancing the definition of object boundaries. The convolutional layer responsible for computing the offset follows the kernel size and expansion of its corresponding convolutional layer. Bilinear interpolation is used to calculate pixel values ​​for offsets not found on the image grid. The MDKA module can be expressed as:

[0208] Attention=Conv1×1(DDW-D-Conv(DDW-Conv(F′))) (36)

[0209]

[0210] Where the input feature F∈R C×H×W ,Attention∈R C×H×W Represents the attention matrix, each value represents the relative importance of the corresponding feature. Represents a convolution operation. The multi-scale variable convolution kernel attention mechanism allows the kernel shape and size to be adaptively adjusted according to the input features during the attention calculation. This flexibility enables the model to better capture the features of objects of different shapes and scales without the need for additional normalization functions, which tend to ignore high-frequency information and thus reduce the performance of self-attention-based methods.

[0211] MFWTConv is used to replace the traditional convolution module of the C3K2 module in YOLOv11, and the MDKA module is introduced after the C2PSA module in YOLOv11. First, the initial training is performed, and the epochs=300, imgsz=640, and batch=16 are set. Then the input image is passed through the backbone network to extract features, and the decoding layer outputs the anchor box prediction. After that, the loss (localization loss, classification loss, confidence loss) is calculated using the label, and finally the weight is updated, and the model parameters are adjusted through the optimizer (SGD). The localization loss is used to measure the deviation between the predicted box and the true box. The localization loss used in this invention is as follows:

[0212]

[0213] Where IOU represents the intersection-union ratio between the predicted box and the real box. represents the Euclidean distance between the center points of the box, c represents the minimum box diagonal length of the predicted box and the real box, and v represents the aspect ratio consistency term, as shown in formula (39)

[0214]

[0215] α is the weight balance term, as shown in formula (40)

[0216]

[0217] Classification loss is used to measure the difference between the predicted category distribution and the true category. Assume that the category probability predicted by the model is The true category is p i : The classification loss formula used in the present invention is as follows:

[0218]

[0219] Confidence loss is used to measure the accuracy of whether the prediction box contains the target. Let the confidence of the prediction box be The confidence of the true box is c (the target box is 1 and the no-target box is 0), and N represents the total number of predicted boxes. The calculation formula is as follows:

[0220]

[0221] The total loss of YOLOv11 is the weighted sum of the above three losses, as shown in formula (43)

[0222]

[0223] where λ loc ,λ cls ,λ conf They are the weight hyperparameters of the positioning loss, classification loss, and confidence loss of the YOLOv11 network respectively.

[0224] After the model training is completed, the optimal weights are loaded and the image or data set is inferred. The detection results are as follows Figure 6 、 Figure 7 . Figure 6 The PR curve of the two-stage detection results of small defects in transparent components generated based on pseudo labels is shown in the figure. Figure 7 Visualization of the results of two-stage inspection of small defects on transparent components based on pseudo-label generation.

[0225] Example 1

[0226] This embodiment uses transparent lenses as a test platform, and applies the invented dual-stage detection method and system for transparent component micro-defects based on pseudo-label generation to the online detection of transparent lenses. A transparent component micro-defect data acquisition system is composed of a wide-field CCD camera, an LED light source, and a backlighting method, and then our transparent component micro-defect data acquisition system is used to preliminarily obtain our transparent component micro-defect data set. Then, a sufficient number of high-resolution transparent component defect images are generated through a multi-scale step-by-step generation framework for transparent component defects to solve the problem of insufficient samples when constructing a training data set. Finally, the constructed data set is trained using a dual-stage defect detection network framework based on pseudo-labels, and a supervised defect detection network is trained using pseudo-labels generated by an unsupervised defect detection network, and finally a transparent component micro-defect detection model is obtained. The detected results are as follows. Figure 6 、 Figure 7 As shown, Figure 6 The PR curve (precision-recall curve) of the two-stage detection results of small defects in transparent components generated based on pseudo labels is shown in Figure 1. Figure 7 Visualization of the results of two-stage inspection of small defects on transparent components based on pseudo-label generation.

[0227] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for detecting micro defects of transparent components in a dual-stage manner, wherein the method uses a dual-stage detection system for micro defects of transparent components, wherein the dual-stage detection system for micro defects of transparent components comprises an LED light source (2), a wide-field CCD camera (3), a bracket (4), an integrated cable (5), and a computer (6), wherein the LED light source (2) illuminates the transparent component (1) by backlighting, the wide-field CCD camera (3) is mounted on the bracket (4), and the wide-field CCD camera (3) is connected to the computer (6) via an integrated cable (5), and wherein: The specific process is as follows: (1) Design an image acquisition system for tiny transparent components and create a dataset of tiny transparent component defects; (2) A multi-scale step-by-step generation model for transparent component defects is proposed. The first step is to separate defects and background. The second step is to generate high-quality defect images using a generative network. The third step is to synthesize the generated high-quality defect images with the background images. (3) A spatial pyramid balanced feature fusion-adaptive dynamic modulation convolutional network (PBA-Net) is proposed. PBA-Net enhances the feature extraction capability of the PatchCore network for multi-scale defects and its adaptability to complex data through pyramid balanced feature fusion, adaptive dynamic convolution, and convolution instead of average pooling, thereby improving the accuracy and flexibility of unsupervised detection. (4) Generate pseudo labels for various defects for the unsupervised detection network results. The input image is first classified by hierarchical clustering to classify the defects detected by the unsupervised detection network. The classified image is preliminarily screened out by adaptive threshold segmentation. Then, the pseudo labels based on adaptive threshold segmentation are corrected by candidate boxes based on RPN proposal fusion. (5) Using data annotated with pseudo labels, the improved YOLOv11 defect detection network is trained and a method combining multi-frequency wavelet convolution with multi-scale variable convolution kernel attention mechanism is proposed to enhance the accuracy of feature expression.

2. The dual-stage method for detecting minute defects in transparent components according to claim 1, characterized in that: The specific process of step (1) is as follows: A transparent component micro-defect image acquisition system is designed. The system consists of three parts: a wide-field CCD camera (3), an LED light source, and a backlighting method. The wide-field CCD camera (3) converts light signals into charge signals through the photoelectric effect, and then obtains a digital image signal through charge transfer, signal amplification, and analog-to-digital conversion steps. The LED light source is used to provide lighting, and produces an observable image through the interaction between light and the object being photographed, which directly affects the quality, brightness, contrast and detail of the image. The LED light source meets the conditions of constant, adjustable light brightness, relatively uniform color, and multi-angle illumination. The LED light source is used as the light source of the system. The backlighting method illuminates the object by controlling the distribution, intensity, angle and direction of the light source. The light source is located behind the object and is used to highlight the outline of the object. Images of micro-defects of transparent components are collected by the transparent component micro-defect image acquisition system, and a transparent component micro-defect data set is established based on these images.

3. The dual-stage method for detecting minute defects in transparent components according to claim 1, characterized in that: The specific process of step (2) is as follows: The transparent component micro defect multi-scale step-by-step generation model is used to expand the transparent component micro defect dataset. In the first step, the defect foreground and platform background are separated using the background foreground separation method based on the Otsu threshold method. First, the grayscale histogram of the image is calculated, that is, the number of pixels at each grayscale level, and then the global grayscale mean is calculated, as shown in formula (1): （1）； In formula (1) The gray value is The probability of a pixel, is the total number of pixels in the image; Using the improved background average gray value calculation method, the image is divided into three parts. The first part is the background adjacent to the defect part, the second part is the background not adjacent to the defect part, and the third part is the defect part. The weight calculation formulas of the three parts are shown in formulas (2), (3), and (4): （2）； （3）； ； Where, Represents the weight of the background adjacent to the defect part, represents the weight of the background that is not adjacent to the defect part, Represents the weight of the defect part. The mean of the three parts is shown in formulas (5), (6), and (7): ； ； ； in, Represents the mean value of the background adjacent to the defect part, represents the mean of the background that is not adjacent to the defect part, Represents the mean of the defective part, combined with the weights of the three parts for each possible threshold , calculate the inter-class variance of foreground and background, as shown in formula (8): ； In the formula Represents the variance weight of the grayscale value of the background pixels adjacent to the defect part, Represents the variance weight of the grayscale value of background pixels that are not adjacent to the defect part, The value must be greater than To ensure that the contrast between the defect part and the background part is relatively large, calculate different thresholds Under the weight, mean, and intra-class variance of background and foreground, the threshold that maximizes the inter-class variance is selected. , then use the threshold to perform image binarization to extract the background foreground; The second step of the multi-scale, step-by-step generation method for transparent component defects is defect image generation based on a GAN network. The GAN network used significantly expands the network size and utilizes more complex training techniques and conditional generation methods. The second step of the multi-scale step-by-step generation model of transparent component defects is an image synthesis method based on smooth transparency transition. First, the defect image and background image and their transparency information are prepared. Then, a transparency gradient mask is created. Considering the influence of lighting and environment, the lighting model is introduced to calculate the smooth transparency transition. For each pixel, the synthesized color value The calculation formula is as follows: ； In the formula and are the color values ​​of the source image and background image respectively, is the light intensity calculated by the lighting model, is the background illumination or background light intensity, which is used to adjust the brightness of the background part in the composite image. source is the transparency value of the source image.

4. The dual-stage method for detecting minute defects in transparent components according to claim 1, characterized in that: The specific process of step (3) is as follows: Step 1: Divide the image into multiple P×P patches; The input image is divided into multiple small patches, each patch contains a part of the image area, and multiple patches are put into a list called a patch library, as shown in formula (10): ； Step 2: For each patch , use the pre-trained convolutional neural network CNN to extract its features, the convolutional neural network CNN used is ResNet, the pre-trained ResNet extracts the features of each patch, and the features extracted by ResNet are divided into three parts: large-scale features, medium-scale features, and small-scale features. After the three different scale features are further extracted by convolution operation, the three features are embedded and spliced ​​for feature fusion. First, the convolution expansion operation is used, that is, the image is divided according to the specified convolution kernel size. This operation converts the input image into multiple local small block feature representations. Assume that the sizes of the input features x and y are and , its embedding splicing formula is as follows: ； In the formula , Represents the position in the expanded feature map, , Represents rows and columns of small blocks, It is the convolution kernel. The expansion operation divides the image into small blocks according to the convolution kernel, and then the image features of the convolution expansion operation are and Perform head-to-tail splicing, as shown in formula (12): ； In the formula is the expanded image In the nth block Characteristics of the location, is an image exist Features, is the concatenated feature, with a size of ,in is the number of channels after splicing; Step 3: Use the deconvolution operation, after the splicing operation, the spliced ​​result z is folded to restore its original image size , the calculation formula is as follows: ； The image size after deconvolution becomes the size of the original image x Here, the features of the three scales are spliced, and the size of the splicing result is ,in is the number of channels of the three scale features, The height and width of the medium-scale features are mapped to a high-dimensional feature space through the feature extraction network. These patches are regarded as a feature, and the features of all normal image blocks are saved in the feature library; ； Where, It is a feature library. is from a normal image block The extracted features,the feature library is a reference library for the subsequent detection stage,,and each element in the feature library is a feature vector of a trained,normal image block; Step 4: Match the features extracted by the feature extraction module with the features in the feature library; To measure the abnormality of the test image, it is necessary to let the test image pass through the same feature extraction module to extract its patch features, and then compare the extracted patch features with the patch features in the feature library to obtain the abnormality score. , get each and the features in the feature library , as shown in formula (15): ； Next use The maximum distance to each neighbor in the feature library is used to estimate the anomaly score. The distance here uses the Euclidean distance, as shown in formula (16): ； Where, Used to estimate the features of the test image x The anomaly score, Represents the L2 norm, in order to highlight the test image Adjacent feature library features The contribution of To calculate the anomaly score corresponding to the test image, as shown in formula (17): ； in represents the abnormality score of the corresponding feature of the test image, The surrogate closest to the test image features of A feature set consisting of patches; Step 5: Draw an abnormal image heat map by testing the abnormal score of each feature of the image.

5. The dual-stage method for detecting minute defects in transparent components according to claim 1, characterized in that: The specific process of step (4) is as follows: First, the high-scoring areas are classified by hierarchical clustering to obtain pseudo-labels of pre-selected defects. The similarity or distance between all defect data is calculated. Then, the two most similar clusters are selected to merge or the least similar cluster is split according to the inter-cluster distance. The inter-cluster distance is recalculated based on the new clusters. The above steps are repeated until the stopping condition is met. The average distance of all point pairs between the two clusters is used as the distance between the two clusters, as shown in Equation (18): ； in and Representing two different clusters, the three defects obtained after hierarchical clustering are classified into cracks, scratches, and stains according to defect characteristics and size; Then, the data image is segmented using adaptive thresholding to preliminarily screen out defects. Each defect corresponds to a different threshold. The entropy theory based on information theory is used to maximize the sum of the foreground and background entropy. The threshold is selected based on the information entropy of the foreground and background in the image grayscale histogram. Assume the grayscale range of the image is , the threshold is T, is the probability of gray level i, then the information entropy H is defined as: ； Where, is the foreground entropy, and its calculation formula is: ； is the background entropy, and its calculation formula is: （21）； The threshold T is chosen so that The maximum,fracture threshold is calculated to be 225, the threshold for scratch defect is 213, and the threshold for spot defect is 156.,The adaptive entropy threshold method is used to preliminarily obtain the defect location, and then the defect location is corrected,using the image edge features extracted based on the pre-selected,box of the RPN proposal. The pseudo-labels are then corrected using the RPN proposal network. First, the RPN accepts the feature map output by the backbone network as input, which is usually a high-dimensional feature representation of the input image after several convolutional layers. A 3×3 sliding window is then applied to the input feature map for convolution. Each sliding window is responsible for generating a candidate region at a position. A set of anchor boxes is generated at each sliding window position in the feature map. These anchor boxes are a series of rectangular boxes with predefined sizes and aspect ratios. The sizes and aspect ratios are different for different types of defects. The sizes of the fracture defects are set to [32, 64, 128], and their aspect ratios are (1, 1), (2, 1), (1, 2). The sizes of the scratch defects are set to [16, 32, 64], and their aspect ratios are (1, 1), (2, 1), (1, 2). The sizes of the scratch defects are set to [8, 16, 32], and their aspect ratios are (1, 1), (2, 1), (1, 2). Secondly, the defect positioning box obtained based on the adaptive entropy threshold will be combined with the candidate box of the RPN proposal and the IOU threshold will be used to filter out the candidate box with the correct position and high confidence. The calculation formula is as follows: ； ； Where, represents the real frame, represents the prediction box, is the intersection-over-union threshold, Represents the set of pre-selected boxes that meet the conditions, that is, those with a value greater than the intersection-over-union ratio threshold; Then, the average coordinates of these candidate boxes are taken to obtain the candidate box based on RPN proposal fusion, as shown in formula (24): ； ； in, Represents the average value of the horizontal coordinate of the set of points in the upper left corner of the candidate box, Represents the average value of the vertical coordinate of the set of points in the upper left corner of the candidate box, Represents the number of pre-selected boxes in the set, and the lower right corner point can be obtained similarly; The pseudo-label based on adaptive threshold segmentation is corrected by the candidate box based on RPN proposal fusion. The calculation formula is as follows: ； ； in is the horizontal coordinate of the upper left corner of the corrected positioning frame, is the vertical coordinate of the upper left corner of the corrected positioning frame, is the horizontal coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, is the vertical coordinate of the upper left corner of the positioning box obtained by adaptive threshold segmentation, and is a preset value to ensure that the correction result is close to reality. The lower right corner point can be obtained similarly. Finally, the anchor box and category obtained from the defect are converted into YOLO format and pseudo-labeled.

6. The dual-stage method for detecting minute defects in transparent components according to claim 1, characterized in that: The specific process of step (5) is as follows: The prepared pseudo-labeled data is input into the supervised defect detection network for training. The real-time performance of the detection networks based on RCNN and transformer cannot meet the requirements. Therefore, a defect detection algorithm based on YOLOv11 is adopted and ML-Net is proposed. It consists of two parts: multi-frequency wavelet convolution and multi-scale variable convolution kernel attention mechanism. The input image is decomposed into different frequency components through wavelet transform, and small-size convolution is performed on each frequency layer. Finally, the results are recombined through inverse wavelet transform, thus realizing multi-scale analysis of the image. The MDKA attention mechanism highlights important areas through weighted fusion of feature maps. a) Multi-frequency wavelet convolution; First, MFWTConv uses two-dimensional Haar wavelet transform to perform multi-level decomposition on the input image, assuming that the image is a two-dimensional function , Haar wavelet transform uses four filters to decompose the image into four sub-bands. Its calculation formula is as follows: ； Low-frequency component LL: captures low-frequency information of the image, such as overall shape or outline; ； Horizontal high frequency component LH: captures horizontal edge information in the image; ； Vertical high-frequency component HL: captures vertical edge information in the image; ； Diagonal high-frequency components HH: capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled, that is, the spatial resolution is halved, but the frequency information is decomposed more finely. Recursively performing wavelet transform is called multi-level decomposition to obtain frequency components at different scales. A small convolution kernel of 3x3 or 5x5 is used to perform convolution operations on each decomposed subband. Since the wavelet transform reduces the spatial resolution of each subband, the small-sized convolution kernel covers a larger area of ​​the original image, that is, the receptive field is increased. After the convolution is completed, the inverse wavelet transform is used to re-synthesize the convolution results of each subband into a complete output; ； ； ； ； By using the above operations, the low-frequency and high-frequency components of the original image are restored to a complete image, and the features of different frequency levels are fused together; b) Multi-scale variable convolution kernel attention mechanism; Deformable convolution enables the adjustment of the sampling grid with integer offsets. An additional convolutional layer learns the deformation from the feature map, thereby creating an offset field. The deformation is learned based on the features themselves to obtain the adaptive convolution kernel. The convolution layer responsible for calculating the offset follows the kernel size and expansion of its corresponding convolution layer. Bilinear interpolation is used to calculate the pixel value of the offset that cannot be found on the image grid. The MDKA module can be expressed as: ； ； The input features , Represents the attention matrix, each value represents the relative importance of the corresponding feature, the operator Represents the convolution operation, and the multi-scale variable convolution kernel attention mechanism allows the shape and size of the kernel to be adaptively adjusted according to the input features in the attention calculation; After using MFWTConv to replace the traditional convolution module of the C3K2 module in YOLOv11 and introducing the MDKA module into the C2PSA module in YOLOv11, we first perform initialization training with epochs=300, imgsz=640, and batch=16. Then, the input image is passed through the backbone network to extract features, and the decoding layer outputs anchor box predictions. The labels are then used to calculate the loss, and finally the weights are updated. The model parameters are adjusted through the optimizer SGD. The localization loss is used to measure the deviation between the predicted box and the true box. The localization loss used is as follows: ； Where IOU represents the intersection-union ratio between the predicted box and the real box. represents the Euclidean distance between the center points of the boxes, Represents the minimum box diagonal length of the predicted box and the real box, represents the aspect ratio consistency term, as shown in formula (39); ； is the weight balancing term, as shown in formula (40) ； Classification loss is used to measure the difference between the predicted category distribution and the true category. Assume that the category probability predicted by the model is , the true category is : The classification loss formula used is as follows: ； Confidence loss is used to measure the accuracy of whether the prediction box contains the target. Let the confidence of the prediction box be , the true box confidence is , the target box is 1, no target is 0, N represents the total number of prediction boxes, and its calculation formula is as follows: ； The total loss of YOLO is the weighted sum of the above three losses, as shown in formula (43); ； where λ loc ,λ cls ,λ conf They are the weight hyperparameters of the positioning loss, classification loss, and confidence loss of the YOLOv11 network respectively; After the model training is completed, the optimal weights are loaded and inference is performed on the image or dataset.

Images