SMT surface defect detection method

Abstract

The invention discloses an SMT (Surface Mount Technology) surface defect detection method, which specifically comprises the following steps: S1, sample preparation: randomly selecting a template image from a template image set of a patch element, randomly selecting a to-be-detected image from a to-be-detected image set of the patch element, and randomly selecting a to-be-detected image from a to-be-detected image set of the patch element; the invention relates to the technical field of defect detection. According to the SMT surface defect detection method, the depth feature and the semantic difference mapping feature are fused, so that the defect of the SMT element can be more accurately identified, the false detection rate and the omission ratio are reduced, automatic detection can be realized, manual re-judgment is not needed, the detection efficiency is greatly improved, the requirements of a modern production line for high-speed production are met, and the production cost is reduced. By reducing the requirement for manual re-judgment, the labor cost can be remarkably reduced, the economic benefits of enterprises can be improved, and the detection method is suitable for patch elements of different sizes, types and shapes and has high adaptability and flexibility.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, in particular to a method for detecting surface defects of SMT. Background Art

[0002] In the context of intelligent manufacturing, SMT technology has been widely used. However, in the component mounting process, after the electronic patch components are soldered on the PCB board through solder paste, the quality of the component patches needs to be tested, such as checking whether there are defects such as tombstones, displacement, and empty solder joints. These defects have a crucial impact on the function of the motherboard. Even the missing of a part will affect the use of the entire motherboard.

[0003] Existing industrial defect detection is mainly divided into manual visual inspection and automated optical inspection (AOI). The emergence of AOI has reduced the cost of manual visual inspection in many scenarios, but the false detection rate of AOI is very high. For example, on the actual production line of a well-known consumer electronics manufacturer, up to 60% of the patch components need to be manually re-inspected, while the number of patch components that are actually defective after manual inspection only accounts for 2% of the number of defective patch components detected by AOI. Therefore, in SMT production lines with high detection accuracy requirements, the industry generally adopts an inspection method that combines AOI with manual re-judgment. However, since the patch defects in SMT production lines are usually relatively small, manual re-judgment often leads to missed detection due to factors such as fatigue, and the average processing time of manual re-judgment is 15 seconds / item / person, which cannot meet the requirements of modern production lines with high-speed production.

[0004] In recent years, artificial intelligence technology and advanced manufacturing technology have been deeply integrated, becoming the core driving force of a new round of industrial revolution. With the rapid development of big data and high-performance computing intelligence technology, deep learning has made significant progress in computer vision, natural language processing, and pattern recognition. However, artificial intelligence methods represented by deep learning still have obvious deficiencies in continuous generalization, transparent credibility, and stable reliability, and cannot fully cope with the challenges in the process of intelligent upgrading of the manufacturing industry. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a SMT surface defect detection method, which solves the problems mentioned in the above background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a SMT surface defect detection method, specifically comprising the following steps:

[0007] S1. Sample preparation: randomly select a template image from the template image set of the patch component, which should be a standard image without defects, and randomly select an image to be tested from the image set of the patch component to be tested, which is an image of an actual patch component on the production line. According to the model, category and other information of the patch component, match the template image and the image to be tested to form a patch component sample pair;

[0008] S2. Feature extraction: Use a deep convolutional neural network as the encoder network to extract features from the template image and the image to be tested. In the output layer of the encoder network, use a feature fusion module to fuse the deep features of the template image and the image to be tested to obtain fused features. At the same time, introduce a semantic difference mapping module to calculate the difference between the template image and the image to be tested at the semantic level and generate semantic difference mapping features.

[0009] S3, feature fusion and defect segmentation: The fused features and semantic difference mapping features are fused again to obtain the final fused features, and the fused features are sent to the decoder network for defect segmentation. On the defect segmentation map, a threshold is set to determine whether there is a defect. If the pixel value of a certain area exceeds the threshold, it is considered that the area has a defect;

[0010] S4, model training and optimization: Use a large number of labeled patch component samples to train the defect detection model, introduce a loss function to evaluate the performance of the model, and optimize the model parameters by minimizing the loss function;

[0011] S5. Detection application: Apply the trained defect detection model to the SMT production line in actual production. On the production line, obtain the test image of the patch component through a camera or image acquisition device, input the test image into the defect detection model, and obtain the defect detection result. According to the detection result, mark or remove the defective patch components to ensure product quality and production efficiency.

[0012] Preferably, in S1, if the patch components corresponding to the two images belong to the same category and have the same position and direction, the label is set to 0, indicating no defect; if the above conditions are not met, the label is set to 1, indicating a defect.

[0013] Preferably, in S2, the encoder network includes multiple convolutional layers, pooling layers and activation functions, which are used to extract deep features of the image layer by layer.

[0014] Preferably, in S3, the decoder network includes multiple upsampling layers, convolutional layers and activation functions, which are used to restore the fused features to a defect segmentation map with the same size as the original image.

[0015] Preferably, in S4, during the training process, strategies such as cross-validation, learning rate adjustment, and regularization are used to prevent overfitting and improve the generalization ability of the model. The loss function includes two parts: classification loss and segmentation loss, which are used to evaluate the performance of the model on classification tasks and segmentation tasks, respectively.

[0016] Preferably, in S5, the detection result includes information such as the location, size and type of the defect.

[0017] Preferably, before feature extraction, the image is preprocessed, such as denoising, contrast enhancement, grayscale, etc. These operations help to improve the clarity and recognizability of the image, thereby improving the detection accuracy. In order to deal with patch components and defects of different sizes, a multi-scale detection strategy is adopted. During the detection process, the size and resolution of the input image are adjusted to achieve the detection of patch components and defects of different sizes.

[0018] Beneficial Effects

[0019] The present invention provides a method for detecting SMT surface defects. Compared with the prior art, it has the following beneficial effects:

[0020] The SMT surface defect detection method specifically comprises the following steps: S1, sample preparation; S2, feature extraction; S3, feature fusion and defect segmentation; S4, model training and optimization; S5, detection application; by constructing a fusion data-driven defect detection model, combining deep learning and semantic difference mapping features, accurate defect detection of the patch component to be tested image is realized, and at the same time, details such as illumination conditions, image preprocessing, multi-scale detection and real-time optimization are considered in the detection process, so as to further improve the detection accuracy and efficiency, and the method has the advantages of high detection accuracy, high detection efficiency, low cost and strong adaptability, and is of great significance to promoting the intelligent development of SMT production lines. By integrating deep features and semantic difference mapping features, the defects of the patch components can be more accurately identified, the false detection rate and the missed detection rate can be reduced, and automatic detection can be realized without manual re-judgment, which greatly improves the detection efficiency and meets the requirements of modern production lines with high-speed production. By reducing the need for manual re-judgment, the invention can significantly reduce labor costs and improve the economic benefits of enterprises. The detection method is suitable for patch components of different sizes, types and shapes, and has strong adaptability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] See also Figure 1 The present invention provides two embodiments: a SMT surface defect detection method, specifically including the following embodiments:

[0024] Example 1

[0025] S1. Sample preparation: randomly select a template image from the template image set of the patch component, which should be a standard image without defects, and randomly select an image to be tested from the image set of the patch component to be tested, which is an image of an actual patch component on the production line. According to the model, category and other information of the patch component, match the template image and the image to be tested to form a patch component sample pair;

[0026] S2. Feature extraction: Use a deep convolutional neural network as the encoder network to extract features from the template image and the image to be tested. In the output layer of the encoder network, use a feature fusion module to fuse the deep features of the template image and the image to be tested to obtain fused features. At the same time, introduce a semantic difference mapping module to calculate the difference between the template image and the image to be tested at the semantic level and generate semantic difference mapping features.

[0027] S3, feature fusion and defect segmentation: The fused features and semantic difference mapping features are fused again to obtain the final fused features, and the fused features are sent to the decoder network for defect segmentation. On the defect segmentation map, a threshold is set to determine whether there is a defect. If the pixel value of a certain area exceeds the threshold, it is considered that the area has a defect;

[0028] S4, model training and optimization: Use a large number of labeled patch component samples to train the defect detection model, introduce a loss function to evaluate the performance of the model, and optimize the model parameters by minimizing the loss function;

[0029] S5. Detection application: Apply the trained defect detection model to the SMT production line in actual production. On the production line, obtain the test image of the patch component through a camera or image acquisition device, input the test image into the defect detection model, and obtain the defect detection result. According to the detection result, mark or remove the defective patch components to ensure product quality and production efficiency.

[0030] Example 2

[0031] S1. Sample preparation: randomly select a template image from the template image set of the patch component, which should be a standard image without defects, and randomly select an image to be tested from the image set of the patch component to be tested, which is an image of an actual patch component on the production line. According to the model, category and other information of the patch component, match the template image and the image to be tested to form a patch component sample pair;

[0032] S2. Feature extraction: Use a deep convolutional neural network as the encoder network to extract features from the template image and the image to be tested. In the output layer of the encoder network, use a feature fusion module to fuse the deep features of the template image and the image to be tested to obtain fused features. At the same time, introduce a semantic difference mapping module to calculate the difference between the template image and the image to be tested at the semantic level and generate semantic difference mapping features.

[0033] S3, feature fusion and defect segmentation: The fused features and semantic difference mapping features are fused again to obtain the final fused features, and the fused features are sent to the decoder network for defect segmentation. On the defect segmentation map, a threshold is set to determine whether there is a defect. If the pixel value of a certain area exceeds the threshold, it is considered that the area has a defect;

[0034] S4, model training and optimization: Use a large number of labeled patch component samples to train the defect detection model, introduce a loss function to evaluate the performance of the model, and optimize the model parameters by minimizing the loss function;

[0035] S5. Detection application: Apply the trained defect detection model to the SMT production line in actual production. On the production line, obtain the test image of the patch component through a camera or image acquisition device, input the test image into the defect detection model, and obtain the defect detection result. According to the detection result, mark or remove the defective patch components to ensure product quality and production efficiency.

[0036] In the present invention, in S1, if the patch components corresponding to the two images belong to the same category and have the same position and direction, the label is set to 0, indicating no defect; if the above conditions are not met, the label is set to 1, indicating a defect.

[0037] In the present invention, in S2, the encoder network includes multiple convolutional layers, pooling layers and activation functions, which are used to extract deep features of the image layer by layer.

[0038] In the present invention, in S3, the decoder network includes multiple upsampling layers, convolutional layers and activation functions, which are used to restore the fused features to a defect segmentation map with the same size as the original image.

[0039] In the present invention, in S4, during the training process, strategies such as cross-validation, learning rate adjustment, and regularization are used to prevent overfitting and improve the generalization ability of the model. The loss function includes two parts: classification loss and segmentation loss, which are used to evaluate the performance of the model on classification tasks and segmentation tasks, respectively.

[0040] In the present invention, in S5, the detection result includes information such as the location, size and type of the defect.

[0041] In the present invention, before feature extraction, the image is preprocessed, such as denoising, contrast enhancement, grayscale, etc. These operations help to improve the clarity and recognizability of the image, thereby improving the detection accuracy. In order to deal with patch components and defects of different sizes, a multi-scale detection strategy is adopted. During the detection process, the size and resolution of the input image are adjusted to achieve the detection of patch components and defects of different sizes.

[0042] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0043] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0044] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A SMT surface defect detection method, characterized in that: The specific steps include: S1. Sample preparation: randomly select a template image from the template image set of the patch component, which should be a standard image without defects, and randomly select an image to be tested from the image set of the patch component to be tested, which is an image of an actual patch component on the production line. According to the model, category and other information of the patch component, match the template image and the image to be tested to form a patch component sample pair; S2. Feature extraction: Use a deep convolutional neural network as the encoder network to extract features from the template image and the image to be tested. In the output layer of the encoder network, use a feature fusion module to fuse the deep features of the template image and the image to be tested to obtain fused features. At the same time, introduce a semantic difference mapping module to calculate the difference between the template image and the image to be tested at the semantic level and generate semantic difference mapping features. S3, feature fusion and defect segmentation: The fused features and semantic difference mapping features are fused again to obtain the final fused features, and the fused features are sent to the decoder network for defect segmentation. On the defect segmentation map, a threshold is set to determine whether there is a defect. If the pixel value of a certain area exceeds the threshold, it is considered that the area has a defect; S4, model training and optimization: Use a large number of labeled patch component samples to train the defect detection model, introduce a loss function to evaluate the performance of the model, and optimize the model parameters by minimizing the loss function; S5. Detection application: Apply the trained defect detection model to the SMT production line in actual production. On the production line, obtain the test image of the patch component through a camera or image acquisition device, input the test image into the defect detection model, and obtain the defect detection result. According to the detection result, mark or remove the defective patch components to ensure product quality and production efficiency.

2. A SMT surface defect detection method according to claim 1, characterized in that: In S1, if the patch components corresponding to the two images belong to the same category and have the same position and direction, the label is set to 0, indicating no defect; if the above conditions are not met, the label is set to 1, indicating a defect.

3. A SMT surface defect detection method according to claim 1, characterized in that: In S2, the encoder network includes multiple convolutional layers, pooling layers and activation functions, which are used to extract deep features of the image layer by layer.

4. A SMT surface defect detection method according to claim 1, characterized in that: In S3, the decoder network includes multiple upsampling layers, convolutional layers and activation functions, which are used to restore the fused features to a defect segmentation map with the same size as the original image.

5. The SMT surface defect detection method according to claim 1, characterized in that: In S4, during the training process, strategies such as cross-validation, learning rate adjustment, and regularization are used to prevent overfitting and improve the generalization ability of the model. The loss function includes two parts: classification loss and segmentation loss, which are used to evaluate the performance of the model on classification tasks and segmentation tasks respectively.

6. A SMT surface defect detection method according to claim 1, characterized in that: In S5, the detection result includes information such as the location, size and type of the defect.

7. A SMT surface defect detection method according to claim 1, characterized in that: Before feature extraction, the image is preprocessed, such as denoising, contrast enhancement, grayscale, etc. These operations help improve the clarity and recognizability of the image, thereby improving the detection accuracy. In order to deal with patch components and defects of different sizes, a multi-scale detection strategy is adopted. During the detection process, the size and resolution of the input image are adjusted to achieve the detection of patch components and defects of different sizes.

Images