A PCB appearance defect detection system and false point filtering method and system

Abstract

The application discloses a PCB appearance defect detection system and false point filtering method and system; including: defect image collection, collecting original images of PCBs on the production line through a high-definition camera; data preprocessing, labeling and expanding the original defect images; model training and deployment, responsible for model training and deployment; false point filtering step. The application does not require the user to specify the defect type, only needs to give the input image, and the system can automatically complete the image defect detection, avoids complicated operation, and reduces the use difficulty of the user. The application can realize defect early warning, meets the customer defect intelligent classification and grading control demand, improves the re-inspection efficiency and accuracy of the re-inspection personnel, and reduces the labor cost and error rate. This helps manufacturers to improve production efficiency and product quality, and enhances market competitiveness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a PCB appearance defect detection system and false point filtering method and system. BACKGROUND

[0002] Traditional PCB detection methods mainly rely on manual visual inspection and test equipment, which has problems such as low efficiency, easy to make mistakes, and strong subjectivity. With the continuous improvement and complication of PCB manufacturing process, traditional detection methods have been difficult to meet the detection needs of high quality and high efficiency.

[0003] With the continuous development of computer vision technology, the research based on machine vision detection method has made significant progress. Especially the application of convolutional neural network (CNN), through the use of multi-layer CNN to directly supervise the learning of a large amount of data, the network can automatically and effectively learn the image features for the learning task without prior knowledge and experience. This method not only saves the tedious feature design and extraction process, but also can learn more abstract and high-level image features in the sample, improving the robustness of the features.

[0004] However, in the actual production link, due to the influence of different environmental conditions, equipment settings and shooting angles and other factors, many good boards will be misdetected as defects by the machine due to imaging reasons, so that the final detection effect is not satisfactory, and the detection efficiency needs to be further improved. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a PCB appearance defect detection system and false point filtering method and system. The present application can intelligently filter out false points after identifying defects.

[0006] The present application is realized by the following technical solutions:

[0007] A PCB appearance defect detection system and false point filtering system, comprising:

[0008] A defect image collection module for collecting original images of PCBs on the production line through a high-definition camera;

[0009] A data preprocessing module for labeling and expanding the original defect images;

[0010] A model training and deployment module for model training and deployment;

[0011] A false point filtering module for filtering false points.

[0012] A PCB appearance defect detection system and false point filtering method, comprising the following steps:

[0013] S1, defect image collection, collecting original images of PCBs on the production line through a high-definition camera;

[0014] S2, data preprocessing, labeling and expanding the original defect images;

[0015] S3, model training and deployment, responsible for model training and deployment;

[0016] S4, false point filtering.

[0017] Step S1 collects original images of PCBs on the production line through a high-definition camera, and the specific sub-steps are as follows:

[0018] S1-1, continuously detecting multiple circuit boards, when a circuit board reaches the detection area, an ultra-high-definition camera is used to take a picture of the detection area on the production line to obtain an original image; the original image is used for matching and positioning with a template.

[0019] S1-2, according to the set matching template, positioning the preliminary position of the whole PCB that needs to be detected in the detection area, and according to the PCB structure information, further subdividing the obtained PCB picture into different layers such as PAD layer, hole ring layer, circuit layer, and green oil layer.

[0020] S1-3, after the collected original image is subdivided into different layers such as PAD layer, hole ring layer, circuit layer, and green oil layer, corresponding level parameters are set, and through dynamic threshold segmentation, uniformity algorithm, circuit gap detection algorithm, green oil bridge detection algorithm, concave-convex detection algorithm, etc., defect detection is performed, and the target defect picture is obtained by intercepting in the detection image according to the defect position;

[0021] S1-4, the target defect picture obtained after detection by the detection end algorithm includes the defect type name in the target defect picture, the coordinate information of the defect in the picture, the position information of the target defect picture in the whole PCB original picture, and the position information of the picture in GERBER, which are all recorded and saved to a specified storage medium;

[0022] S1-5, after each PCB completes detection and saves the completed image information, the working service station sends instruction information, and the false point filtering module reads the target defect image, defect image information, and target defect image GERBER information in real time according to the instruction information sent by the service station to a specified path for reasoning; and completes sorting.

[0023] The target defect image refers to an image obtained by intercepting according to a specific algorithm rule set in advance, and the target defect image contains suspected defect features in the original image to be detected.

[0024] The target defect image is obtained by preliminary screening according to a specific algorithm rule, and the target defect image obtained under the rule does not output the final result of defect recognition to the terminal, but outputs the position of the defect in the original image to be detected and intercepts the position according to the given length and width. Only the suspected defect in the image to be recognized is output under the rule.

[0025] The step S2 of labeling and expanding the original defect image specifically includes the following sub-steps:

[0026] S2-1, new project, newly obtained data set pictures needing labeling are input into a trained multi-modal combined defect detection model; the multi-modal combined defect detection model outputs the defect category pseudo-label corresponding to the image, and the classification label contains the classification name and classification level predicted by the model;

[0027] S2-2, create a new labeling task, form a final defect cognition table according to the self-developed model cognition table and the terminal attention defect table, and use the semi-automatic labeling software imagelabeling to manually label the pre-labeled defect image according to the coco data set standard to correct the labeling error caused by the defect category pseudo-label;

[0028] S2-3, after the defect image is manually labeled, a series of data expansion samples and defect feature enhancement samples are obtained by using geometric transformation such as translation, transposition, mirror image, rotation, scaling and cropping, and image processing methods such as noise, blur, color transformation, erasing and filling, which are used for large model training;

[0029] S2-4, test the trained large model with new samples, view the training results, and adjust the model parameters, re-classify the labels and classify the levels for the data with data pulling;

[0030] The classification label needs to clearly classify the target and category, collect and preprocess the data, use the label for training, and verify and optimize the label according to the actual situation of the terminal.

[0031] The step S3 of labeling and expanding the original defect image specifically includes the following sub-steps:

[0032] S3-1, large model incremental learning: input the processed image data including defect map, scan map and Gerber map into the designed pre-trained neural network model (large model) for weight fine-tuning training; after feature extraction of the image data by the multi-head self-attention mechanism module and the convolution module, the network outputs the distribution of defect categories, and establishes a supervision relationship with the label information, so as to realize the incremental learning of the large model on the defect feature information;

[0033] S3-2, large model knowledge distillation: according to the model distillation technology of deep learning, taking the trained large model as the teacher model, and taking another neural network-based and small parameter size model (small model) as the student model; using the defect data labeled by a factory, inputting into the teacher and student models respectively, and outputting the corresponding defect category distribution; taking the defect distribution of the teacher model as the benchmark, calculating the loss of the output distribution of the two models and performing gradient back propagation on the small model, so as to achieve the supervision and guidance of the large model to the small model and the defect knowledge transfer effect, thereby realizing the goal of generating a customized small model suitable for the factory defect detection by using the large model.

[0034] S3-3, after the distillation training of the large model is completed and the test comparison result is obtained, the small model meets the generalization requirement of defect detection, the small model is exported to generate a deployable file, and is deployed to the client. Subsequently, the model deployed in the remote client is updated in an OTA (on the air) manner.

[0035] The false point filtering in step S4 includes the following sub-steps:

[0036] The inference model deployed on the client side will acquire the target defect image information photographed by the PCB appearance detection machine in real time through the service station instruction, read the target image information from the computer hard disk to the inference processor; in the inference process, according to the parameter size of the training model, the defect ROI is intercepted again to reduce the inference data processing amount, and the false point inference system completes the inference, and then writes the inference generated detection result information package into the target picture data information package; after the inference information is written, the service station is informed that the inference is completed, and the next batch of target pictures is inferred;

[0037] According to the defect classification defined by the customer, the false point filtering module can intelligently classify, defect early warning and false point filtering; finally, the detection result output by the false point filtering module enters the sorting system of the appearance machine, judges whether the current PCB is a good board or a defective board, and realizes the customer's customized defect category and defect level control requirement according to the classification information output by the false point filtering module.

[0038] Compared with the prior art, the present application has the following advantages and effects:

[0039] The present application first proposes to realize the matching of PCB defect images and template images by using deep learning (large model) technology, and to detect defects in images.

[0040] The application detects defects, which is different from other image detection methods. The method proposed by the application performs incremental learning on a large model, calculates the similarity of image block features of a defect image and a scan image and a Gerber image by using a self-attention mechanism, perceives abnormal areas of the defect image, and has the ability to detect PCB general defects. In addition, the knowledge distillation technology is used to transfer the defect feature knowledge learned by the large model to the small model, so that the small model has more customized adaptation to the defect detection ability of each manufacturer.

[0041] The application does not need the user to specify the defect type, and only needs to give the input image, so that the system can automatically complete the image defect detection, avoid complicated operation, and reduce the use difficulty of the user.

[0042] The application can realize defect early warning, meet the customer's intelligent classification and grading control requirements, improve the re-inspection efficiency and accuracy of the re-inspection personnel, and reduce the labor cost and error rate. This helps manufacturers to improve production efficiency and product quality, and enhances market competitiveness. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a system flowchart of the application.

[0044] Figure 2 is a large model incremental learning schematic diagram of the application.

[0045] Figure 3 is a large model knowledge distillation schematic diagram of the application. DETAILED DESCRIPTION

[0046] The application will be further specifically and in detail described below in combination with specific embodiments.

[0047] Figure 1 As shown, the application discloses a PCB appearance defect detection system and a false point filtering method, which is suitable for a device with PCB image processing capability.

[0048] A high-definition camera acquires a to-be-detected image. The image is first preliminarily aligned with a template at a detection end, after the alignment, each feature layer is matched with the image for difference points, the part with difference after matching is a target defect image, and the image is intercepted.

[0049] After the defect part is intercepted, the image and information are input to a defect image labeling software, and image enhancement and expansion are performed.

[0050] The labeled image is input to a training model for feature extraction and model training, the model training result is manually compared, the model training effect is tested, if the effect does not meet the use condition, the image labeling and enhancement are returned, if the model effect meets the use condition, the model is exported and deployed to a client.

[0051] In the customer terminal, the batch PCB is detected in real time, the false point filtering system carries out reasoning and outputs results according to the picture information transmitted back in real time, according to the custom defect threshold set by the customer, intelligently attributes and classifies and forms alarm information, and the appearance machine completes the sorting process of good boards and bad boards according to the reasoning output results and the alarm information.

[0052] Figure 2 It is a large model incremental learning schematic diagram. The selected defect graph, scan graph and Gerber graph are input to the designed large model for weight fine-tuning training after picture enhancement.

[0053] After the image data is extracted by the multi-head self-attention mechanism module and the convolution module, the non-maximum suppression removes redundancy, and according to the results, the model structure, hyperparameters or data enhancement strategy are adjusted, and then a supervised relationship is established with the label information, the output curve distribution is formed, and the defect label is formed, so as to realize the incremental learning of the large model on the defect feature information.

[0054] Figure 3 It is a large model knowledge distillation schematic diagram. The teacher is a large pre-training neural network, which is trained on a comprehensive defect data set and has high ability to learn and generalize from data. The student model is a smaller and simpler neural network, which improves its performance and generalization ability by learning from the output of the teacher model and the specified customer terminal defect graph data set.

[0055] As described above, the application can be better implemented.

[0056] The embodiments of the application are not limited by the above examples, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the application shall be equivalent replacement modes and shall be included in the protection scope of the application.

Claims

1. A PCB appearance defect detection system and false point filtering method, characterized by Comprising the following steps: S1, a defect image collection step, collecting original images of PCBs on the production line through a high-definition camera; S2, a data preprocessing step, labeling and expanding the original defect images; S3, a model training and deployment step, responsible for model training and deployment; S4, a false point filtering step; The step S1 collects original images of PCBs on the production line through a high-definition camera, and the specific sub-steps are as follows: S1-1, continuously detecting multiple circuit boards, when a circuit board reaches the detection area, an ultra-high-definition camera is used to take pictures of the detection area on the production line to obtain original images; the original images are used for matching and positioning with the template; S1-2, according to the set matching template, the initial position of the whole PCB to be detected is located in the detection area, and according to the PCB structure information, the obtained PCB picture is further subdivided into different layers; S1-3, after the collected original images are subdivided into different layers, the corresponding level parameters are set, the defect detection is performed through the detection algorithm, and the target defect picture is obtained by intercepting in the detection picture according to the defect position; S1-4, the target defect picture obtained after the detection of the detection end algorithm includes the defect type name in the target defect picture, the coordinate information of the defect in the picture, the position information of the target defect picture in the whole PCB original picture, and the position information of the picture in the GERBER, which are all recorded and saved to the specified storage medium; S1-5, after each PCB completes detection and saves the completed image information, the work service station sends instruction information, and the false point filtering module reads the target defect image, defect image information and target defect image GERBER information in real time according to the instruction information sent by the service station to complete reasoning and sorting; Step S3. The model training and deployment step specifically includes the following sub-steps: S3-1, large model incremental learning: input the processed image data including defect map, scan map and Gerber map into the designed pre-trained neural network model, i.e. large model, for weight fine-tuning training; after feature extraction of image data by multi-head self-attention mechanism module and convolution module, the network outputs the distribution of defect categories, and establishes a supervision relationship with the label information, so as to realize the incremental learning of large model on defect feature information; S3-2, large model knowledge distillation: according to the model distillation technology of deep learning, the trained large model is used as the teacher model, and another neural network-based model with small parameters is used as the student model; input the labeled defect data of a factory into the teacher and student models respectively, and output the corresponding defect category distribution; taking the defect distribution of the teacher model as the benchmark, the loss of the output distribution of the two models is calculated and the gradient is back propagated to the small model, so as to achieve the supervision guidance of the large model to the small model and the defect knowledge transfer effect, thereby realizing the goal of generating a small model suitable for the factory defect detection by using the large model; S3-3, after the distillation training of the large model is completed, and the test comparison result is tested, the small model meets the generalization requirement of defect detection, the small model is exported, a deployable file is generated, and is deployed to the client; subsequently, the model deployed in the remote client is updated in an OTA manner; The false point filtering of step S4 includes the following sub-steps: The inference model deployed on the client side can obtain the target defect image information captured by the PCB appearance detection machine in real time through the service station instruction, read the target defect image information from the computer hard disk to the inference processor; in the inference process, according to the parameter size of the training model, the defect ROI is intercepted again to reduce the inference data processing amount, and the false point inference system completes the inference, and writes the inference generated detection result information package into the target picture data information package; after the inference information is written, the service station is informed that the inference is completed, and the next batch of target picture inference is entered; According to the defect classification defined by the customer, the false point filtering can intelligently classify, defect early warning and false point filtering; finally, the detection result output after the false point filtering enters the sorting system of the appearance machine, judges whether the current PCB is a good board or a defective board, and realizes the customer's self-defined defect category and defect level control requirement according to the classification information output by the false point filtering.

2. The appearance defect inspection system of a PCB and false point filtering method according to claim 1, wherein The target defect image refers to the image intercepted according to the set specific algorithm rule in the original PCB image, and the target defect image contains the suspected defect features in the original image to be detected; The target defect image is obtained by preliminary screening according to the set specific algorithm rule, and the target defect image obtained under the rule does not output the final result of defect recognition to the terminal, but outputs the position of the defect in the original image to be detected and intercepts in the position according to the given length and width; only the suspected defect in the image to be recognized is output under the rule.

3. The appearance defect inspection system of a PCB and false point filtering method according to claim 1, wherein Step S2 includes the following sub-steps: S2-1, a new project is created, the newly obtained data set picture needing to be labeled is input into a trained multi-modal combined defect detection model; the multi-modal combined defect detection model outputs the defect category pseudo label corresponding to the image, and the pseudo label contains the classification name and classification level predicted by the model; S2-2, a new labeling task is created, a final defect cognition table is formed according to the self-research model cognition table and the terminal attention defect table, a semi-automatic labeling software imagelabeling is used, and the pre-labeled defect image is manually labeled according to the coco data set standard to correct the labeling error caused by the defect category pseudo label; S2-3, after the manually labeled defect image is obtained, a series of data expansion samples and defect feature enhancement samples are obtained by using geometric transformation and image processing method, which are used for large model training; S2-4, test the trained large model with new samples, view the training result, and adjust the model parameters, re-classify the labels and levels for the data with data pulling.

4. The appearance defect inspection system of a PCB and false point filtering method according to claim 1, wherein The classification label needs to clearly classify the target and category, collect data, preprocess the data, use the label for training, and verify and optimize the label, and adjust according to the actual situation of the terminal.

5. The appearance defect inspection system of a PCB and false point filtering method according to claim 1, wherein The different layers in the step S1-2 include a PAD layer, a hole ring layer, a circuit layer, and a green oil layer. The detection algorithm in the step S1-3 includes dynamic threshold segmentation, uniformity algorithm, circuit gap detection algorithm, green oil bridge detection algorithm, and concave-convex detection algorithm.

6. The appearance defect inspection system of a PCB and false point filtering method according to claim 3, wherein In the step S2-3, the geometric transformation includes translation, transposition, mirroring, rotation, scaling, and clipping; and the image processing method includes noise, blurring, color transformation, erasing, and filling.

7. A system for use in the appearance defect detection system and false point filtering method of any one of claims 1-6, characterized by The method comprises the steps of: a defect image collection module for collecting original images of PCBs on a production line through a high-definition camera; a data preprocessing module for labeling and expanding the original defect images; a model training and deployment module for model training and deployment; a false point filtering module for filtering false points.

Images