Quartz surface defect detection method based on visual identification

Abstract

The invention discloses a quartz surface defect detection method based on visual identification, and the method comprises the following steps: S1, carrying out the image collection of a quartz surface through a high-resolution camera, and carrying out the preprocessing of the collected image; s2, carrying out edge extraction on the image by adopting a Canny edge detection algorithm; s3, generating more defect images by using a hybrid generation model based on a variational auto-encoder and an energy guide mechanism; s4, combining the generated defect image and the real image, and performing defect classification through an image segmentation network in combination with a self-attention mechanism; s5, jointly training the improved variational auto-encoder and the image segmentation network; and S6, feeding back the category of the defect and the edge information of the defect to a production line control system in real time. According to the method, a high-resolution image acquisition technology, a variational auto-encoder and an image segmentation network are combined, and automatic detection and accurate classification of quartz surface defects are realized through fusion of deep learning and a traditional image processing method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and computer vision, and particularly relates to a quartz surface defect detection method based on visual recognition. BACKGROUND

[0002] Quartz materials are widely used in high-tech fields such as optics, electronics, and semiconductors. The surface defects of quartz directly affect the performance of the product, so the detection of quartz surface defects is crucial. Traditional quartz surface defect detection methods mainly rely on manual detection and automated detection based on image processing. Although manual detection can provide certain accuracy, it is low in efficiency, easily affected by human factors, and cannot meet the needs of large-scale production. Traditional methods based on image processing, such as edge detection and threshold segmentation, often have low precision and noise interference problems when facing complex defects and irregular surfaces.

[0003] With the development of deep learning technology, defect detection methods based on convolutional neural networks (CNN) and other deep learning models have gradually become mainstream. Deep learning methods greatly improve the efficiency and accuracy of defect detection by automatically extracting data features. However, these methods still face problems such as insufficient data, overfitting, and poor generalization ability, especially in the case of scarce defect data, the quality and quantity of the training set become the main factors restricting the performance of the model. In addition, the training process of deep learning models requires a large amount of computing resources and time, which may lead to an increase in system cost.

[0004] Generative Adversarial Networks (GAN) as a powerful generative model, performs well in image generation and data augmentation. Through the adversarial training of the generator and the discriminator, GAN can generate realistic images and effectively expand the training set in the case of data scarcity. However, the training process of GAN is complex and unstable, and the generated images often lack edge details and defect features, affecting the training effect of the subsequent model.

[0005] To solve these problems, the present application proposes a quartz surface defect detection method based on variational autoencoder (VAE) and energy guiding mechanism. Unlike GAN, the present application uses variational autoencoder to generate defect images and combines energy guiding mechanism to improve the edge details of the images, ensuring the quality of the generated images. Through this method, diverse and structurally realistic defect images can be generated in the case of insufficient data, improving the training effect and generalization ability of the model.

[0006] In addition, the present application combines an image segmentation network and a self-attention mechanism to improve the positioning and classification accuracy of defect regions. The image segmentation network combined with the self-attention mechanism can accurately identify and classify defect regions in complex backgrounds, thereby improving detection accuracy. Through this joint training of generation and classification, the present application not only overcomes the problem of overfitting and poor generalization ability caused by single model training in the prior art, but also effectively improves the efficiency and accuracy of quartz surface defect detection.

[0007] The prior art still faces problems such as insufficient training data, difficulty in ensuring accuracy, and large noise interference, especially in cases where the defect types are complex and the defect samples are scarce. Traditional methods cannot achieve efficient and accurate automatic detection. In contrast, the present application combines the advantages of generative adversarial networks and image segmentation networks, and provides a more reliable and efficient solution by combining an improved variational autoencoder with an energy guiding mechanism. This method can generate high-quality images in the case of insufficient defect samples and improve the accuracy and robustness of defect detection, and has wide application prospects. SUMMARY

[0008] The present application proposes a quartz surface defect detection method based on visual recognition, which uses a hybrid generative model combining an improved variational autoencoder (VAE) and an energy guiding mechanism, and combines an image segmentation network and a self-attention mechanism. This method generates diverse defect images through an improved variational autoencoder, solving the problem of insufficient training data, and improves the edge sharpness and accuracy of defect features of generated images through an energy guiding mechanism. By jointly training the generator and the classification network, the features of the generated images and the defect classification are more consistent, improving the overall detection performance of the system. At the same time, the image segmentation network combined with the self-attention mechanism enhances the attention to the defect region, improving the accuracy of defect positioning and classification. This method not only generates high-quality defect images, but also provides real-time feedback on defect categories and edge information, optimizing defect handling and quality control on the production line, and ultimately achieving efficient and accurate quartz surface defect detection, with strong application prospects and practical value.

[0009] According to an embodiment of the present application, a quartz surface defect detection method based on visual recognition includes the following steps:

[0010] S1, image acquisition of the quartz surface is performed by a high-resolution camera, and the acquired image is preprocessed;

[0011] S2, edge extraction is performed on the image using a Canny edge detection algorithm;

[0012] S3, a hybrid generative model based on a variational autoencoder and an energy guiding mechanism is used;

[0013] More defect images are generated;

[0014] S4, combining the generated defect image and the real image, and performing defect classification through the image segmentation network combined with the self-attention mechanism;

[0015] S5. Jointly train the improved variational autoencoder and image segmentation network;

[0016] S6. Feedback the defect category and defect edge information to the production line control system in real time.

[0017] Optionally, the image acquisition is performed on the quartz surface by using a high-resolution camera.

[0018] Optionally, the preprocessing converts the color image into a grayscale image, removes noise using Gaussian filtering, and enhances image contrast through histogram equalization to obtain the original image.

[0019] Optionally, the S2 specifically includes:

[0020] S21. Calculate the gradient magnitude and gradient direction of each pixel in the original image, and use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula of the gradient magnitude G is:

[0021]

[0022] Among them, G x and G y is the gradient value in the horizontal and vertical directions;

[0023] Next, non-maximum suppression is performed according to the gradient direction to suppress pixels that are not local maxima, thereby retaining potential edges;

[0024] S22, Canny algorithm includes edge tracking algorithm and edge tracking algorithm; use double threshold method to accurately screen the edge, the specific steps are:

[0025] Set two thresholds T1 and T2, where T1 <T2,对于原始图像中梯度幅值大于T2的像素点标记为强边缘;

[0026] Pixels with gradient magnitudes less than T1 in the original image are marked as non-edges; pixels with gradient magnitudes between T1 and T2 are marked as weak edges if they are connected to a strong edge.

[0027] S23. Use an edge tracking algorithm to refine the weak edge to ensure the continuity and integrity of the edge. The edge tracking algorithm gradually checks the connectivity between the weak edge and the strong edge. If a weak edge is found to be connected to a strong edge, it is marked as an edge; otherwise, it is removed.

[0028] S24, output the edge information of the quartz surface defect, including the edge position, shape and size of the defect.

[0029] Optionally, the S3 specifically comprises:

[0030] S31, the pre-processed quartz surface image sample is input to the encoder network of the improved variational autoencoder model, and the encoder network is based on a traditional VAE structure and is structurally improved for the sparsity, weak edges and multi-scale features of the quartz surface defect image. The structural improvement specifically comprises: using a multi-scale convolution path to extract image features to enhance the expression ability of multi-size defects such as micro-cracks, denudation and irregular spots; inputting the extracted image features into an edge guiding mechanism: generating an edge map by gradient calculation on the original image through a Sobel operator, and performing normalization and activation processing on the edge map to form an attention weight map; channel weighting fusion is performed on the attention weight map and the intermediate feature map at the corresponding position, so that the response strength of the potential defect area is prominently highlighted in the feature channel; the low-level feature map of the shallow convolution layer of the encoder network is extracted, and is fused with the medium-high layer semantic feature map through shallow residual connection;

[0031] S31, the quartz surface original image sample is input to the encoder network of the improved variational autoencoder model, and the latent space feature vector z is extracted, and the latent distribution parameters of the defect image: the mean vector and the standard deviation vector are output, and the reparameterization technique is used to generate the latent variable z:

[0032]

[0033] wherein H and W are the height and width of the image respectively, d is the dimension of the latent space, and ε is a standard Gaussian noise vector;

[0034] S32, for the characteristics that the quartz surface defect area is distributed sparsely and the morphology is complex in the image, a multi-level decoder network combining a channel attention and an edge feature guiding module is constructed to decode the latent variable z to generate a defect simulation image while maintaining the edge details of the original image, and the image reconstruction loss function L rec is used for training and optimization:

[0035]

[0036] S33, an energy guiding module of the defect image is constructed, the gradient amplitude and gradient direction of the quartz surface defect edge are taken as the guiding signal, an image energy function E(x) is defined to represent the pixel-level gradient intensity distribution in the image, and the image reconstruction loss function L The energy difference of edge energy consistency loss term L is constructed energy This loss term is used to constrain the structural distribution of the generated image in the edge area to be consistent with the real defect image, thereby improving the restoration accuracy and structural integrity of cracks, scratches, and erosion boundaries in the generated image. The energy function is defined as follows:

[0037]

[0038] S34, constructing a joint loss function L to optimize the hybrid generation model of the variational autoencoder and the energy guidance mechanism as a whole, wherein the loss function L includes the image reconstruction loss L rec , potential distribution constraint D KL and edge energy consistency loss L energy , the weights of each item are adjusted by the hyperparameters β and λ respectively to achieve balanced training between reconstruction accuracy, edge structure expression and potential space diversity. The joint loss function is defined as follows:

[0039]

[0040] Where q(z|x) represents the posterior approximate distribution, and p(z) represents the standard normal prior distribution. Guided by the loss function L, the generator can generate quartz defect images with realistic structures, clear edges, and diverse expressions in the standard latent space, providing expressive power and distribution rationality support for subsequent S35 defect simulation.

[0041] S35. After training, the hybrid generative model of the variational autoencoder and the energy guidance mechanism samples latent variables from the standard normal distribution. Input to a multi-level decoder network to generate diverse quartz defect image samples The diverse quartz defect image sample This dataset includes typical quartz surface defect types: microcracks, erosion, scratches, and irregular spots. It is used to expand the defect sample dataset and improve the subsequent image segmentation network's ability to identify rare defects.

[0042] Optionally, the S4 specifically includes:

[0043] S41, passing the generated defect image sample Merge with the real defect image x to form a training data set;

[0044] S42. Using an image segmentation network combined with a self-attention mechanism to classify and locate defects in the training dataset, wherein the image segmentation network is a network based on a U-Net architecture that can more finely capture and segment areas of quartz surface defects;

[0045] S43, enhance the attention of the network to the defect area through the self-attention mechanism, and the self-attention mechanism calculates the attention weight through the following formula:

[0046]

[0047] Wherein, q i and k j are the query vector and key vector of the input image feature respectively, and α ij represents the attention weight of the i-th pixel to the j-th pixel;

[0048] S44, the edge information of the quartz surface defect is extracted through the image segmentation network, and the segmentation image of the defect area is output, each defect area in the segmentation image is recognized, and a label is generated for each defect area, identifying the specific position and type of the defect;

[0049] S45, the features extracted by the segmentation image of the defect area are processed through the full connection layer, and the defect category of each area is obtained, including the crack, bubble, impurity and scratch on the quartz surface;

[0050] S46, the prediction probability of the defect category is output through the Softmax activation function, and the classification result is calculated through the following formula:

[0051]

[0052] Wherein, z i is the output of the full connection layer, and y i is the prediction probability of the corresponding category;

[0053] S47, the classification error is calculated through the cross entropy loss function L C , and the loss function L C is calculated through the following formula:

[0054] L C =-∑ i t i log(y i );

[0055] Wherein, t i is the true label, and y i is the prediction probability output by the image segmentation network;

[0056] S48, the parameters in the image segmentation network are optimized through the back propagation algorithm, and the classification and segmentation accuracy is gradually improved until the classification error of the network reaches the predetermined threshold, and the defect classification task is completed.

[0057] Optionally, the joint training specifically includes: alternating training of the improved variational autoencoder and the image segmentation network; the improved variational autoencoder minimizes the difference between the generated defect image and the real defect image so that the generated image conforms to the characteristics of the real defect image; the image segmentation network optimizes the accuracy of defect classification by minimizing the classification error;

[0058] Optionally, the S6 specifically includes:

[0059] S61, feed back the extracted defect category and defect edge information to the production line control system in real time,

[0060] S62. Based on the feedback defect information, the production line control system can automatically adjust the production process, handle defects, and optimize production quality;

[0061] S63. During the real-time feedback process, bounding boxes are used to identify defective areas to ensure the accuracy of the detection results and to be used for subsequent quality control and automated repair processes.

[0062] The beneficial effects of the present invention are:

[0063] First, by introducing an improved edge tracking algorithm, this invention effectively improves the accuracy and robustness of image edge detection. Traditional edge detection methods typically only extract edges based on strong edges, ignoring the potential information of weaker edges. Our technology deeply analyzes the connectivity between weak and strong edges, preserving the original image features while extracting more details, significantly improving the comprehensiveness and accuracy of edge detection.

[0064] Secondly, the present invention employs a more intelligent judgment mechanism when processing weak edges. When a weak edge connects to a strong edge, the system automatically marks the weak edge as a valid edge. This innovative step provides more complete edge information in the image, avoiding the image information loss caused by the loss of weak edges. This mechanism significantly improves the quality of image segmentation and recognition tasks, and is particularly suitable for scenes with complex backgrounds or rich details.

[0065] Furthermore, the present invention optimizes the algorithm's computational efficiency. By fine-tuning the processing of weak and strong edges, unnecessary computation is reduced, resulting in a higher processing speed and lower computational resource consumption in practical applications, making the algorithm more adaptable to real-time or large-scale data processing needs.

[0066] Through these technological innovations, the present invention not only solves the shortcomings of traditional edge detection methods, but also greatly improves the effect and efficiency of image processing. It has important application value in image recognition, image segmentation, autonomous driving, medical image analysis and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are meant to explain the present application and are not intended to limit the application. In the drawings:

[0068] Figure 1 A flow chart of a quartz surface defect detection method based on visual recognition is proposed for the present application.

[0069] Figure 2 An encoder network structure diagram of an improved variational autoencoder model is proposed for the present application. DETAILED DESCRIPTION

[0070] The present application will now be further described in greater detail in conjunction with the accompanying drawings. These drawings are simplified schematic diagrams which only show the basic structure of the present application in a schematic manner, and thus only show the components related to the present application.

[0071] Reference Figure 1-2 A quartz surface defect detection method based on visual recognition includes the following steps:

[0072] S1, image acquisition of the quartz surface is performed by a high-resolution camera, and the acquired image is preprocessed;

[0073] S2, edge extraction is performed on the image using a Canny edge detection algorithm;

[0074] S3, a hybrid generative model based on variational autoencoder and energy guiding mechanism is used;

[0075] More defect images are generated;

[0076] S4, the generated defect images and real images are combined, and defect classification is performed through an image segmentation network combined with a self-attention mechanism;

[0077] S5, the improved variational autoencoder and the image segmentation network are jointly trained;

[0078] S6, the category of the defect and the edge information of the defect are fed back to the production line control system in real time.

[0079] The quartz surface image is collected by a high-resolution camera and preprocessed, and then the edge information of the image is extracted by using a Canny edge detection algorithm. On this basis, a hybrid generative model of an improved variational autoencoder and an energy guiding mechanism is used to generate more defect images to enhance the training data set. Then, combined with the generated defect images and real images, the defects are classified and located by an image segmentation network combined with a self-attention mechanism. This method optimizes the synergistic effect of image generation and defect recognition by jointly training the improved variational autoencoder and the image segmentation network, and feeds back the defect categories and edge information to the production line control system in real time to realize automatic defect detection and processing.

[0080] In the embodiment, the image acquisition is performed on the quartz surface by a high-resolution camera.

[0081] In the embodiment, the preprocessing converts the color image into a grayscale image, removes noise using a Gaussian filter, and enhances the image contrast through histogram equalization to obtain the original image.

[0082] In the embodiment, S2 specifically includes:

[0083] S21, the gradient amplitude and gradient direction of each pixel point in the original image are calculated, the horizontal and vertical gradients of the image are calculated using a Sobel operator, and the calculation formula of the gradient amplitude G is:

[0084]

[0085] wherein G x and G y are the gradient values in the horizontal and vertical directions;

[0086] Then, non-maximum suppression is performed according to the gradient direction to suppress the pixel points that are not local maximum values, thereby retaining potential edges;

[0087] S22, the Canny algorithm includes an edge tracking algorithm and an edge tracking algorithm; a double-threshold method is used to accurately screen the edges, and the specific steps are:

[0088] Two thresholds T1 and T2 are set, where T1 < T2, and the pixel points in the original image with a gradient amplitude greater than T2 are marked as strong edges;

[0089] The pixel points in the original image with a gradient amplitude less than T1 are marked as non-edges; for the pixel points with a gradient amplitude between T1 and T2, if they are connected to strong edges, they are marked as weak edges;

[0090] S23, using edge tracking algorithm to refine the weak edge, ensuring the continuity and integrity of the edge, the edge tracking algorithm checks the connectivity of the weak edge and the strong edge step by step, if it is found that the weak edge is connected with the strong edge, it is marked as edge; otherwise, it is removed;

[0091] S24, output the edge information of the quartz surface defect, including the edge position, shape and size of the defect.

[0092] By calculating the gradient amplitude and direction of each pixel point in the original image, and using Sobel operator to calculate the horizontal and vertical gradient of the image, then through non-maximum suppression to retain the potential edge of local maximum. In the Canny edge detection algorithm, a double threshold method is used to accurately screen the edge, two thresholds are set to classify the gradient amplitude into strong edge, weak edge and non-edge, and the edge tracking algorithm is used to refine the weak edge, ensuring the continuity and integrity of the edge. Finally, the edge information of the quartz surface defect is output, including the edge position, shape and size of the defect.

[0093] In this embodiment, S3 specifically comprises:

[0094] S31, the preprocessed quartz surface image sample is input into the encoder network of the improved variational autoencoder model, and the encoder network is based on the traditional VAE structure and is improved in structure for the sparsity, weak edge and multi-scale features of the quartz surface defect image. The structural improvement specifically includes: using a multi-scale convolution path to extract image features to enhance the expression ability of multi-size defects such as micro-cracks, denudation and irregular spots; inputting the extracted image features into an edge guiding mechanism: generating an edge map by gradient calculation on the original image using a Sobel operator, and performing normalization and activation processing on the edge map to form an attention weight map; channel weighting fusion is performed on the attention weight map and the intermediate feature map at the corresponding position, so that the response strength of the potential defect area is prominently highlighted in the feature channel; the low-level feature map of the shallow convolution layer of the encoder network is extracted, and is fused with the medium-high layer semantic feature map through shallow residual connection;

[0095] S31, the original image sample of the quartz surface is input into the encoder network of the improved variational autoencoder model, and the latent space feature vector z is extracted, and the latent distribution parameters of the defect image: the mean vector and the standard deviation vector are output, and the reparameterization trick is used to generate the latent variable z:

[0096]

[0097] where H and W are the height and width of the image respectively, d is the dimension of the latent space, and ε is a standard Gaussian noise vector.

[0098] S32, in view of the characteristics that the quartz surface defect region is sparse in distribution and complex in shape in the image, a multi-level decoder network combining channel attention and edge feature guiding module is constructed to decode the latent variable z to generate a defect simulation image while maintaining the edge details of the original image, an image reconstruction loss function L rec Training and optimization:

[0099]

[0100] S33, an energy guiding module of the defect image is constructed, the gradient amplitude and gradient direction of the quartz surface defect edge are taken as the guiding signal, an image energy function E(x) is defined to represent the pixel-level gradient intensity distribution in the image, and an edge energy consistency loss term L is constructed by calculating the energy difference between the real image x and the generated image energy , which is used to constrain the structure distribution of the generated image in the edge region to be consistent with the real defect image, improve the restoration accuracy and structural integrity of the crack, scratch and erosion boundary in the generated image, and the energy function is defined as follows:

[0101]

[0102] S34, a joint loss function L is constructed to optimize the hybrid generative model of the variational autoencoder and the energy guiding mechanism, the loss function L includes an image reconstruction loss L rec , a latent distribution constraint term D KL and an edge energy consistency loss L energy , the weights of each term are adjusted by hyperparameters β and λ respectively to realize balanced training between reconstruction accuracy, edge structure expression and latent space diversity, and the joint loss function is defined as follows:

[0103]

[0104] where q(z|x) represents the posterior approximation distribution, and p(z) represents the standard normal prior distribution. Under the guidance of the loss function L, the generator can generate quartz defect images with true structure, clear edges and diverse expressions in the standard latent space, providing expression ability and distribution rationality support for subsequent S35 defect simulation.

[0105] S35, after the training of the hybrid generative model of the variational autoencoder and the energy guiding mechanism, the latent variable is sampled from the standard normal distribution and input into the multi-level decoder network to generate diversified quartz defect image samples The diversified quartz defect image samples The quartz surface contains typical defect types: microcracks, denudation, scratches, irregular spots; for expanding the defect sample data set, improving the subsequent image segmentation network's recognition ability of rare defects.

[0106] A defect image generation and training method combining an improved variational autoencoder (VAE) with an energy guiding mechanism. First, the preprocessed quartz surface image is processed by an encoder network, which extracts features of different scales using a multi-scale convolution path, and combines edge guiding mechanism and shallow residual connection to highlight the potential defect area, generating the latent distribution parameters (mean and standard deviation) of the image. Then, a multi-level decoder network is constructed, which decodes the latent variables to generate simulated defect images through channel attention and edge feature guiding module, and maintains edge details. Next, define the energy guiding module, constrain the edge of the generated image to be consistent with the real image through edge energy consistency loss, and improve the boundary accuracy of the generated image. The joint loss function optimizes the image reconstruction loss, latent distribution constraint and edge energy consistency loss at the same time, achieving a balance between the accuracy of the generated image, the expression of edge structure and the diversity of latent space. Finally, after training, a variety of quartz defect image samples are generated, providing data enhancement for the subsequent image segmentation network and improving the recognition ability of rare defects.

[0107] In this embodiment, the S4 specifically includes:

[0108] S41, merge the generated defect image samples with the real defect image x to form a training data set;

[0109] S42, use the image segmentation network combined with the self-attention mechanism to classify and locate the defects in the training data set, the image segmentation network is a network based on U-Net architecture, which can capture and segment the area of quartz surface defects more finely;

[0110] S43, enhance the network's attention to the defect area through the self-attention mechanism, which calculates the attention weight through the following formula:

[0111]

[0112] Where q i and k j are the query vector and key vector of the input image feature, respectively, and a ij represents the attention weight of the i-th pixel to the j-th pixel;

[0113] S44, extract the edge information of the quartz surface defects through the image segmentation network, output the segmentation image of the defect area, identify each defect area in the segmentation image, and generate a label for each defect area to identify the specific location and type of the defect;

[0114] S45, processing the features extracted from the segmented image of the defect area through the full connection layer to obtain the defect category of each area, including the crack, bubble, impurity and scratch on the quartz surface;

[0115] S46, outputting the prediction probability of the defect category through the Softmax activation function, and the classification result is calculated through the following formula:

[0116]

[0117] wherein, z i is the output of the full connection layer, y i is the prediction probability of the corresponding category;

[0118] S47, calculating the classification error through the cross entropy loss function L C , and the loss function L C is calculated through the following formula:

[0119] L C = -∑ i t i log(y i );

[0120] wherein, t i is the true label, and y i is the prediction probability output by the image segmentation network;

[0121] S48, optimizing the parameters in the image segmentation network through the back propagation algorithm, gradually improving the classification and segmentation accuracy, until the classification error of the network reaches the predetermined threshold, and the defect classification task is completed.

[0122] In the embodiment, the joint training specifically includes: the improved variational autoencoder and the image segmentation network are alternately trained; the improved variational autoencoder minimizes the difference between the generated defect image and the real defect image, so that the generated image conforms to the characteristics of the real defect image; the image segmentation network optimizes the accuracy of defect classification by minimizing the classification error;

[0123] In the embodiment, the S6 specifically includes:

[0124] S61, feeding back the extracted defect category and defect edge information to the production line control system in real time,

[0125] S62, according to the feedback defect information, the production line control system can automatically adjust the production process, carry out defect processing, and optimize the production quality;

[0126] S63、In the real-time feedback process, the bounding box is used to identify the defect area, ensuring the accuracy of the detection results and being used for subsequent quality control and automatic repair processes.

[0127] Embodiment 1:

[0128] To verify the feasibility of the application in practical application, we apply the application to the production line of a semiconductor factory for detecting defects on the surface of quartz. The quartz material produced by the factory is widely used in the manufacturing process of high-end semiconductor equipment, so the defects on the surface of quartz have a significant impact on product quality. Therefore, there is an urgent need for an efficient and accurate detection method to ensure product quality.

[0129] In the quartz processing production line of the factory, after the quartz surface has undergone multiple processes such as cutting and polishing, it will finally enter the detection link through an automated production line. In this link, the factory hopes to automatically identify various defects on the surface of quartz, such as cracks, bubbles, impurities, and scratches, through a visual detection system, to ensure that each quartz plate meets high-quality standards. To achieve this goal, the application installs a high-resolution camera on the production line to continuously capture high-definition images of the quartz surface. The captured images are converted into grayscale images through a preprocessing process, removing noise and enhancing the contrast of the images. Subsequently, the Canny edge detection algorithm is used to extract edge information from the images, identifying potential defect areas.

[0130] To improve the accuracy and generalization ability of the model, the application combines a generative variational autoencoder to enhance the training data set. By generating synthetic images similar to real defect images, the model can still have efficient learning ability even in the case of limited defect type data. Using image segmentation networks and self-attention mechanisms, the application can accurately classify and locate different types of defect areas (such as cracks, bubbles, etc.) in the images. These detection results are fed back to the production line control system in real time, automatically repairing defects or further manual inspection, thereby greatly improving the automation level and production efficiency of the production line.

[0131] In the training process of the model, we use an image data set containing 2000 quartz plates. The image of each quartz plate is captured by a high-resolution camera, with an image size of 1024x1024 pixels. To ensure the diversity of the data set, we perform data augmentation, generating additional training samples through rotation, flipping, scaling, and brightness changes, resulting in a data set size of over 8000 labeled images. The data set contains different types of defects, including cracks, bubbles, impurities, scratches, etc. Through these diverse image samples, the model can better identify and classify different defects.

[0132] During the training process, we adopted a combination method of generative variational autoencoder and convolutional neural network (CNN). We used an image segmentation network based on U-Net architecture to locate the defect area, and combined with the self-attention mechanism to improve the attention of the defect area. Variational autoencoder is used to generate synthetic images similar to the actual defect images, which together with the real images form the training set, enhancing the generalization ability of the model. During the training process, we used the Adam optimizer, set the learning rate to 0.0002, the batch size to 16, and the training rounds to 50 epochs. We chose the cross-entropy loss function to optimize the accuracy of defect classification, and used the IoU (Intersection over Union) loss function to optimize the precision of image segmentation. Through the backpropagation algorithm, the parameters of the generator and discriminator are gradually optimized, so that the difference between the generated defect images and the real images is continuously reduced, thereby improving the detection performance of the model.

[0133] After a period of training, we divided the dataset into training and test sets, with 70% of the data used for training and 30% for testing. On the test set, we evaluated the performance of the model and obtained the following evaluation results: accuracy of 98.5%, recall of 97.2%, F1 score of 97.8%, and IoU value of 0.92. These indicators show that our model has shown extremely high accuracy and robustness in the defect detection task.

[0134] To further demonstrate the effectiveness of the invention, we conducted defect detection on 2000 quartz plates on the production line from January 2024 to January 2025. The average defect detection time for each quartz plate was 15 minutes when detected by humans, and there was about an 8% misjudgment rate (i.e., incorrectly labeling defects or missing defects). After using the detection method of the invention, the detection time for each quartz plate was significantly reduced to within 2 minutes, and the misjudgment rate was reduced to less than 1.5%, and the recognition accuracy of various defects (cracks, bubbles, impurities, scratches, etc.) was improved. In particular, in the detection of cracks and bubbles, the accuracy reached 98.5% and 97.2%, respectively.

[0135] To further demonstrate the effectiveness of the invention, the following is a statistical table of the detection data for 2000 quartz plates:

[0136] Table 1 Performance comparison of quartz surface defect detection methods

[0137]

[0138] As can be seen from the table, the detection method of the present application has significant advantages in detection time and misjudgment rate compared with traditional manual detection, and performs excellently in defect type recognition accuracy, especially in the detection of cracks and bubbles, with accuracy reaching 98.5% and 97.2% respectively. Through these data, we can fully prove the superiority of the present application in improving the efficiency and accuracy of quartz surface defect detection, significantly improving the automation degree of the production line, and effectively reducing the errors caused by manual detection.

[0139] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes within the technical range disclosed by the present application according to the technical solutions and inventive concepts of the present application, which should be covered within the protection scope of the present application.

Claims

1. A quartz surface defect detection method based on visual recognition, characterized in that: It includes the following steps: S1. Use a high-resolution camera to collect images of the quartz surface and preprocess the collected images; S2. Adopt the Canny edge detection algorithm to extract edges from the images; S3. Use a hybrid generation model based on an improved variational autoencoder and an energy guidance mechanism to generate more defect images; S4. Combine the generated defect images and real images, and perform defect classification through an image segmentation network combined with a self-attention mechanism; S5. Jointly train the improved variational autoencoder and the image segmentation network; S6. Real-time feedback the category of the defect and the edge information of the defect to the production line control system.

2. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The image collection uses a high-resolution camera to collect images of the quartz surface.

3. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The preprocessing converts the color image into a grayscale image, uses Gaussian filtering to remove noise, and enhances the image contrast through histogram equalization to obtain the original image.

4. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The specific content of S2 includes: S21. Calculate the gradient magnitude and gradient direction of each pixel point in the original image, use the Sobel operator to calculate the horizontal and vertical gradients of the image, and perform non-maximum suppression according to the gradient direction to suppress pixel points that are not local maxima, thereby retaining potential edges; S22. The Canny algorithm includes an edge tracking algorithm and an edge tracking algorithm; use a double-threshold method to accurately screen edges. The specific steps are as follows: Set two thresholds T1 and T2, where T1 < T2. Pixel points in the original image with a gradient magnitude greater than T2 are marked as strong edges; Pixel points in the original image with a gradient magnitude less than T1 are marked as non-edges; for pixel points with a gradient magnitude between T1 and T2, if they are connected to strong edges, they are marked as weak edges; S23. Use the edge tracking algorithm to refine the weak edges. The edge tracking algorithm gradually checks the connectivity between the weak edges and the strong edges. If it is found that the weak edges are connected to the strong edges, the weak edges are marked as edges; otherwise, the weak edges are removed; S24. Output the edge information of the quartz surface defects, including the edge position, shape and size of the defects.

5. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The specific content of S3 includes: S31, the pre-processed quartz surface image sample Input to the encoder network of the improved variational autoencoder model: use multi-scale convolution paths to extract image features; input the extracted image features into the edge guidance mechanism: use the Sobel operator to calculate the gradient of the original image to generate an edge map, and then normalize and activate the edge map to form an attention weight map; perform channel-weighted fusion of the attention weight map with the intermediate feature map at the corresponding position; extract the low-level feature map of the shallow convolution layer of the encoder network and fuse it with the mid- and high-level semantic feature maps through shallow residual connections; integrate the multi-path outputs through the feature fusion module and generate the latent distribution parameters: mean vector μ(x) and standard deviation vector σ(x); finally, construct the latent variable z through the reparameterization technique; S32. Construct a multi-level decoder network combining channel attention and edge feature guidance modules to decode the latent variable z to generate defect simulation images. At the same time, the edge details of the original image are maintained, and the image reconstruction loss function L rec Perform training optimization; S33, construct an energy guidance module for defect images, use the gradient amplitude and gradient direction of the quartz surface defect edge as the guidance signal, define the image energy function E(x) to represent the pixel-level gradient intensity distribution in the image, and generate the defect simulation image by calculating the real defect image x The energy difference of edge energy consistency loss L is constructed energy ; S34, constructing a joint loss function L to optimize the hybrid generation model of the variational autoencoder and the energy guidance mechanism as a whole, wherein the loss function L includes the image reconstruction loss L rec , potential distribution constraint D KL and edge energy consistency loss L energy , the weights of each item are adjusted by the hyperparameters β and λ respectively to achieve balanced training between reconstruction accuracy, edge structure expression and potential space diversity. The joint loss function is defined as follows: Among them, q(z|x) represents the posterior approximate distribution, and p(z) represents the standard normal prior distribution. Guided by the loss function L, the generator can generate quartz defect images with real structures, clear edges and diverse expressions in the standard latent space; S35. After training, the hybrid generative model of the variational autoencoder and the energy guidance mechanism samples latent variables from the standard normal distribution. Input to a multi-level decoder network to generate diverse quartz defect image samples 6. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The specific content of S4 includes: S41, passing the generated defect image sample Merge with the real defect image x to form a training data set; S42. Use an image segmentation network combined with a self-attention mechanism to perform defect classification and localization on the training data set. The image segmentation network is a network based on the U-Net architecture; S43. Enhance the network's attention to the defect area through the self-attention mechanism. The self-attention mechanism calculates the attention weights through the following formula: Among them, q i and k j are the query vector and key vector of the input image features, α ij represents the attention weight of the i-th pixel to the j-th pixel; S44. Extract the edge information of the quartz surface defects through the image segmentation network, output the segmentation image of the defect area, identify each defect area in the segmentation image, and generate a label for each defect area to identify the specific position and type of the defect; S45. Process the features extracted from the segmentation image of the defect area through the fully connected layer to obtain the defect category of each area, including cracks, bubbles, impurities, scratches on the quartz surface; S46, output the predicted probability y of the defect category through the Softmax activation function i ; S47, through the cross entropy loss function L C Calculate classification error; S48. Optimize the parameters in the image segmentation network through the back propagation algorithm.

7. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The joint training specifically includes: alternating training of the improved variational autoencoder and the image segmentation network; the improved variational autoencoder minimizes the difference between the generated defect image and the real defect image so that the generated image conforms to the characteristics of the real defect image; the image segmentation network optimizes the accuracy of defect classification by minimizing the classification error.

8. The method for detecting quartz surface defects based on visual recognition according to claim 1, characterized in that: The S6 specifically includes: S61, feed back the extracted defect category and defect edge information to the production line control system in real time, S62. Based on the feedback defect information, the production line control system can automatically adjust the production process and handle the defects; S63. During the real-time feedback process, a bounding box is used to identify the defect area.

Images