Apparent defect detection system and false point filtering method and system for PCB (Printed Circuit Board)

Abstract

The invention discloses an apparent defect detection system and a false point filtering method and system for a PCB (Printed Circuit Board). Comprising the following steps: defect image collection: carrying out original image collection on a PCB on a production line through a high-definition camera; preprocessing the data, and labeling and expanding the original defect image; model training and deployment, which is responsible for model training and deployment; and a false point filtering step. According to the method, a user does not need to specify a defect type, the system can fully automatically complete image defect detection only by giving the input image, tedious operation is avoided, and the use difficulty of the user is reduced. Defect early warning can be realized, intelligent defect classification and graded management and control requirements of customers are met, the reinspection efficiency and accuracy of reinspection personnel are improved, and the labor cost and the error rate are reduced. Therefore, manufacturers can improve the production efficiency and the product quality, and the market competitiveness is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a PCB appearance defect detection system and a false point filtering method and system. Background Art

[0002] Traditional PCB inspection methods mainly rely on manual visual inspection and test equipment, which is inefficient, error-prone, and highly subjective. With the continuous improvement and complexity of PCB manufacturing processes, traditional inspection methods have been unable to meet the needs of high-quality and high-efficiency inspections.

[0003] With the continuous development of computer vision technology, research on machine vision detection methods has made significant progress. In particular, the application of convolutional neural networks (CNNs) can automatically and effectively learn image features for learning tasks without the need for prior knowledge and experience by using multi-layer CNNs to directly supervise large amounts of data. This method not only eliminates the tedious process of feature design and extraction, but also learns more abstract and higher-level image features in samples, improving the robustness of features.

[0004] However, in the actual production process, due to the influence of different environmental conditions, equipment settings, shooting angles and other factors, many good boards will be mistakenly detected as defects by the machine due to imaging reasons, making the final inspection effect unsatisfactory, and the inspection efficiency needs to be further improved. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings and deficiencies of the prior art and provide a PCB appearance defect detection system and a false point filtering method and system. The present invention can intelligently filter out false points after identifying defects.

[0006] The present invention is achieved through the following technical solutions: A PCB appearance defect detection system and false point filtering system, comprising: Defect image collection module, used to collect original images of PCBs on the production line through high-definition cameras; Data preprocessing module, used to annotate and expand the original defect image; Model training and deployment module, used for model training and deployment; The false point filtering module is used to filter false points.

[0007] A PCB appearance defect detection system and false point filtering method, comprising the following steps: S1, defect image collection, collects original images of PCBs on the production line through high-definition cameras; S2, data preprocessing, marking and expanding the original defect image; S3, model training and deployment, responsible for model training and deployment; S4, false point filtering.

[0008] Step S1 collects original images of the PCB on the production line through a high-definition camera, which is specifically divided into the following sub-steps: S1-1, continuously inspect multiple circuit boards. When a circuit board reaches the area to be inspected, an ultra-high-definition camera is used to take a picture of the area to be inspected on the production line to obtain an original image; the original image is used to match and align with the template.

[0009] S1-2, according to the set matching template, locate the preliminary position of the entire PCB to be inspected in the area to be inspected, and further subdivide the obtained PCB image into different layers such as PAD layer, hole ring layer, circuit layer, green oil layer, etc. according to the PCB structure information; S1.3, after the collected original image is subdivided into different layers such as PAD layer, hole ring layer, line layer, green oil layer, etc., the corresponding level parameters are set, and defect detection is performed through dynamic threshold segmentation, uniformity algorithm, line gap detection algorithm, green oil bridge detection algorithm, concave-convex detection algorithm, etc., and the defect position is intercepted in the detection image to obtain the target defect image; S1-4, the target defect image obtained after the detection by the detection end algorithm, including the defect type name in the target defect image, the coordinate information of the defect in the image, the position information of the target defect image in the entire PCB original image, and the position information of the image in GERBER, is recorded and saved to a designated storage medium; S1-5, after each PCB is inspected and the image information is saved, the work service station sends instruction information. The false point filtering module goes to the designated path to read the target defect image, defect image information, and target defect image GERBER information in real time for reasoning according to the instruction information sent by the service station, and completes the sorting.

[0010] The target defect image refers to an image captured from the original PCB image according to a specific algorithm rule that has been set. The target defect image contains suspected defect features in the original image to be detected; The target defect image is obtained through preliminary screening by a specific algorithm rule that has been set. The target defect image obtained under this rule does not output the final result of defect recognition to the terminal, but outputs the position of the defect in the original image to be detected and intercepts it at the position according to the given length and width. Under this rule, only the suspected defects in the image to be identified are output.

[0011] Step S2, in which the original defect image is annotated and expanded, specifically includes the following sub-steps: S2-1, create a new project, input the newly obtained dataset image that needs to be annotated into a trained multimodal combination defect detection model; the multimodal combination defect detection model outputs the defect category pseudo label corresponding to the image, and the classification label contains the classification name and classification level predicted by the model; S2-2, create a new labeling task, form the final defect recognition table based on the self-developed model recognition table and the terminal concern defect table, use the semi-automatic labeling software imagelabeling to manually label the pre-labeled defect images according to the coco dataset standard to correct the labeling errors caused by the defect category pseudo-labels; S2-3, after the defect images are manually annotated, a series of data expansion samples and defect feature enhancement samples are obtained by using geometric transformations such as translation, transposition, mirroring, rotation, scaling, and cropping, as well as image processing methods such as noise, blurring, color transformation, erasing, and filling for large model training; S2-4, test the trained large model with new samples, check the training results, adjust the model parameters, and reclassify the labels and levels for the data with data pull; The classification labels need to clarify the classification targets and categories, collect and preprocess data, use labels for training, and verify and optimize labels, and adjust them according to the actual situation of the terminal.

[0012] Step S3 is responsible for model training and deployment, which specifically includes the following sub-steps: S3-1, large model incremental learning: The processed image data, including defect images, scanned images, and Gerber images, are input into the designed pre-trained neural network model (large model) for weight fine-tuning training; after the multi-head self-attention mechanism module and convolution module extract features from the image data, the network outputs the distribution of defect categories and establishes a supervision relationship with the label information to achieve incremental learning of defect feature information by the large model; S3-2, large model knowledge distillation: Based on the model distillation technology of deep learning, the trained large model is used as the teacher model, and another model (small model) based on a neural network and with a small parameter scale is used as the student model; the labeled defect data of a certain factory is used to input into the teacher and student models respectively, and the corresponding defect category distribution is output respectively; the defect distribution of the teacher model is used as a benchmark, and the loss of the output distribution of the two models is calculated and the gradient back propagation of the small model is performed to achieve the supervision and guidance of the large model on the small model and the defect knowledge transfer effect, thereby achieving the goal of using the large model to generate a customized small model adapted to the defect detection of the factory; S3-3: After the distillation training of the large model is completed and the test results are compared to make the small model meet the generalization requirements of defect detection, the small model is exported, a deployable file is generated, and deployed to the client. The OTA (on the air) method is subsequently used to update the client model deployed remotely.

[0013] The false point filtering in step S4 includes the following sub-steps: The inference model deployed on the client side will obtain the target defect image information captured by the PCB appearance inspection machine in real time through the service station instructions, and read the target image information from the computer hard disk to the inference processor; during the inference process, according to the parameter size of the training model, the defect ROI is intercepted again to reduce the amount of inference data processing. After the false point inference system completes the inference, the inference-generated detection result information package is written into the target image data information package; after the inference information is written, the service station is notified that the inference is completed, and the next batch of target image inference is entered; According to the customer-defined defect classification, the false point filtering module can intelligently classify the detection information, provide defect warnings and filter false points; finally, the detection results output by the false point filtering module will enter the sorting system of the appearance machine to determine whether the current PCB is a good board or a defective board, and based on the classification information output by the false point filtering module, the client's customized defect category and defect level management and control requirements are realized.

[0014] Compared with the prior art, the present invention has the following advantages and effects: The present invention proposes for the first time to use deep learning (large model) technology to achieve matching between PCB defect images and template images to perform defect detection on images.

[0015] The present invention is different from other image detection methods in the defect detection link. The method proposed by the present invention uses the self-attention mechanism to calculate the similarity of the image block features of the defect image, the scan image, and the Gerber image through incremental learning of the large model, so as to perceive the abnormal area of ​​the defect image and enable it to have the ability to detect common PCB defects. In addition, the knowledge distillation technology is used to transfer the defect feature knowledge learned by the large model to the small model, so that the small model has a more customized defect detection capability to adapt to each manufacturer.

[0016] The present invention does not require the user to specify the defect type. Only an input image is required, and the system can automatically complete the detection of image defects, thus avoiding cumbersome operations and reducing the difficulty of use for users.

[0017] The present invention can realize defect early warning, meet the needs of customers for intelligent defect classification and graded control, improve the efficiency and accuracy of re-inspection personnel, and reduce labor costs and error rates. This helps manufacturers improve production efficiency and product quality and enhance market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a system flow chart of the present invention.

[0019] Figure 2 It is a schematic diagram of large model incremental learning of the present invention.

[0020] Figure 3 It is a schematic diagram of the knowledge distillation of the large model of the present invention. DETAILED DESCRIPTION

[0021] The present invention is further described in detail below in conjunction with specific embodiments.

[0022] Figure 1 As shown, the present invention discloses a PCB appearance defect detection system and a false point filtering method, which is suitable for a device with PCB image processing capability.

[0023] The high-definition camera obtains the image to be inspected. The image is first preliminarily aligned with the template at the detection end. After the alignment is completed, the difference points of each feature layer and the image are matched. The matched part with the difference is the target defect image, and the image is intercepted.

[0024] After the defect area is captured, the image and information are input into the defect image annotation software, and the image is enhanced and amplified.

[0025] The annotated images are input into the training model for feature extraction and model training. The results of the model training are manually compared to test the model training effect. If the effect does not meet the usage conditions, the image is returned for image annotation and enhancement. If the model effect meets the usage conditions, the model is exported and deployed to the client.

[0026] At the customer terminal, batch PCBs are inspected in real time. The false point filtering system infers and outputs the results based on the real-time image information. According to the custom defect threshold set by the customer, intelligent classification is performed and alarm information is generated. The appearance machine completes the sorting process of good and bad boards based on the inference output results and alarm information.

[0027] Figure 2 This is a schematic diagram of incremental learning of a large model. After image enhancement, the selected defect images, scanned images, and Gerber images are input into the designed large model for weight fine-tuning training.

[0028] After the multi-head self-attention mechanism module and convolution module extract features from the image data, non-maximum suppression removes redundancy, and adjusts the model structure, hyperparameters or data enhancement strategy based on the results, a supervision relationship is established with the label information, and the curve distribution is output to form a defect label to achieve incremental learning of defect feature information by the large model. Figure 3 It is a schematic diagram of knowledge distillation of a large model. The teacher is often a large pre-trained neural network, trained on a comprehensive defect dataset, with a high ability to learn and generalize from data. The student model is a smaller and simpler neural network, which improves its own performance and generalization ability by specifying a customer terminal defect map dataset and learning from the output of the teacher model.

[0029] As described above, the present invention can be better implemented.

[0030] The implementation methods of the present invention are not limited to the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement methods and shall be included in the protection scope of the present invention.

Claims

1. A PCB appearance defect detection system and false point filtering system, characterized in that include: Defect image collection module, used to collect original images of PCBs on the production line through high-definition cameras; Data preprocessing module, used to annotate and expand the original defect image; Model training and deployment module, used for model training and deployment; The false point filtering module is used to filter false points.

2. A PCB appearance defect detection system and false point filtering method, characterized in that The steps include: S1, defect image collection step, collecting original images of PCBs on the production line through a high-definition camera; S2, data preprocessing step, annotating and expanding the original defect image; S3, model training and deployment step, responsible for model training and deployment; S4, false point filtering step.

3. The PCB appearance defect detection system and false point filtering method according to claim 2, characterized in that: Step S1 collects original images of the PCB on the production line through a high-definition camera, which is specifically divided into the following sub-steps: S1-1, continuously inspect multiple circuit boards. When a circuit board reaches the area to be inspected, an ultra-high-definition camera is used to take a picture of the area to be inspected on the production line to obtain an original image; the original image is used for matching and alignment with the template; S1-2, according to the set matching template, locate the preliminary position of the entire PCB to be inspected in the area to be inspected, and further subdivide the obtained PCB image into different layers according to the PCB structure information; S1-3, after the collected original image is subdivided into different layers, the corresponding level parameters are set, and the defect detection is performed through the detection algorithm, and the defect position is intercepted in the detection image to obtain the target defect image; S1-4, the target defect image obtained after the detection by the detection end algorithm, including the defect type name in the target defect image, the coordinate information of the defect in the image, the position information of the target defect image in the entire PCB original image, and the position information of the image in GERBER, is recorded and saved to a designated storage medium; S1-5, after each PCB is inspected and the image information is saved, the work service station sends instruction information. The false point filtering module goes to the designated path to read the target defect image, defect image information, and target defect image GERBER information in real time for reasoning according to the instruction information sent by the service station, and completes the sorting.

4. The PCB appearance defect detection system and false point filtering method according to claim 3, characterized in that: The target defect image refers to an image captured from the original PCB image according to a specific algorithm rule that has been set. The target defect image contains suspected defect features in the original image to be detected; The target defect image is obtained through preliminary screening through a specific algorithm rule that has been set. The target defect image obtained under this rule does not output the final result of defect recognition to the terminal, but outputs the position of the defect in the original image to be detected and cuts it at the position according to the given length and width; under this rule, only the suspected defects in the image to be identified are output.

5. The PCB appearance defect detection system and false point filtering method according to claim 2, characterized in that: Step S2, in which the original defect image is annotated and expanded, specifically includes the following sub-steps: S2-1, create a new project, input the newly obtained dataset image that needs to be annotated into a trained multimodal combination defect detection model; the multimodal combination defect detection model outputs the defect category pseudo label corresponding to the image, and the classification label contains the classification name and classification level predicted by the model; S2-2, create a new labeling task, form the final defect recognition table based on the self-developed model recognition table and the terminal concern defect table, use the semi-automatic labeling software imagelabeling to manually label the pre-labeled defect images according to the coco dataset standard to correct the labeling errors caused by the defect category pseudo-labels; S2-3, after manually annotated defect images, a series of data expansion samples and defect feature enhancement samples are obtained using geometric transformation and image processing methods for large model training; S2-4, test the trained large model with new samples, check the training results, adjust the model parameters for data with data pull, and re-classify the labels and levels.

6. The PCB appearance defect detection system and false point filtering method according to claim 5, characterized in that: The classification labels need to clarify the classification targets and categories, collect and preprocess data, use labels for training, and verify and optimize labels, and adjust them according to the actual situation of the terminal.

7. The PCB appearance defect detection system and false point filtering method according to claim 2, characterized in that: Step S3. The model training and deployment is specifically responsible for the following sub-steps: S3-1, large model incremental learning: The processed image data, including defect images, scanned images, and Gerber images, are input into the designed pre-trained neural network model, i.e., the large model, for weight fine-tuning training; after the multi-head self-attention mechanism module and convolution module extract features from the image data, the network outputs the distribution of defect categories and establishes a supervision relationship with the label information to achieve incremental learning of defect feature information by the large model; S3-2, large model knowledge distillation: Based on the model distillation technology of deep learning, the trained large model is used as the teacher model, and another model based on a neural network with a small parameter scale is used as the student model; the labeled defect data of a certain factory is used to input into the teacher and student models respectively, and the corresponding defect category distributions are output respectively; the defect distribution of the teacher model is used as a benchmark, and the loss of the output distribution of the two models is calculated and the gradient back propagation is performed on the small model to achieve the supervision and guidance of the large model on the small model and the defect knowledge transfer effect, thereby achieving the goal of using the large model to generate a customized small model adapted to the defect detection of the factory; S3-3: After the distillation training of the large model is completed and the test and comparison results are made so that the small model meets the generalization requirements of defect detection, the small model is exported, a deployable file is generated, and deployed to the client; subsequently, the OTA method is used to update the remote client model.

8. The PCB appearance defect detection system and false point filtering method according to claim 2, characterized in that: The false point filtering in step S4 includes the following sub-steps: The inference model deployed on the client side will obtain the target defect image information captured by the PCB appearance inspection machine in real time through the service station instructions, and read the target image information from the computer hard disk to the inference processor; during the inference process, according to the parameter size of the training model, the defect ROI is intercepted again to reduce the amount of inference data processing. After the false point inference system completes the inference, the inference-generated detection result information package is written into the target image data information package; after the inference information is written, the service station is notified that the inference is completed, and the next batch of target image inference is entered; According to the customer-defined defect classification, the false point filtering module can intelligently classify the detection information, provide defect warnings and filter false points; finally, the detection results output by the false point filtering module will enter the sorting system of the appearance machine to determine whether the current PCB is a good board or a defective board, and based on the classification information output by the false point filtering module, the client's customized defect category and defect level management and control requirements are realized.

9. The PCB appearance defect detection system and false point filtering method according to claim 3, characterized in that: The different layers described in step S1-2 include a PAD layer, a hole ring layer, a circuit layer, and a green oil layer; The detection algorithm in step S1-3 includes dynamic threshold segmentation, uniformity algorithm, line gap detection algorithm, green oil bridge detection algorithm, and concave-convex detection algorithm.

10. The PCB appearance defect detection system and false point filtering method according to claim 5, characterized in that: In step S2-3, the geometric transformation method includes translation, transposition, mirroring, rotation, scaling, and cropping; the image processing method includes noise, blurring, color transformation, erasing, and filling.

Images