An SMT defect generation and image quality simulation method for AI image recognition
By developing a method for generating SMT defects and simulating image quality for AI image recognition, we have solved the problem of insufficient training data for artificial intelligence models in the PCBA industry, enabling rapid expansion of the training set and improving the efficiency and accuracy of defect recognition in the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI GLORYSOFT CO LTD
- Filing Date
- 2024-11-29
- Publication Date
- 2026-05-29
AI Technical Summary
In the PCBA industry, the efficiency of AI models in defect identification is low. Existing methods are unable to quickly acquire a large number of defect images for training, and artificially created defects cannot fully simulate the actual defect morphology, resulting in slow improvement in model accuracy and low efficiency.
We employ an AI-based image recognition-oriented method for SMT defect generation and image quality simulation. This method simulates a large amount of defect data using a small dataset, including steps such as background generation, pad generation, component generation, and defect generation. We use style transfer and generative adversarial networks for image simulation to generate a dataset that matches actual defects.
This approach enables rapid expansion of the training set while ensuring image authenticity, thereby improving the efficiency and accuracy of defect identification in deep learning models, simplifying the dataset expansion process, and enhancing the model's recognition capabilities.
Smart Images
Figure CN122115301A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and more specifically to a method for generating SMT defects and simulating image quality for AI image recognition. Background Technology
[0002] In the field of industrial automation quality inspection based on image detection algorithms, especially in the PCBA field, defects are typically detected using AOI (Automated Optical Inspection) machines. On the detected images, defects are then manually assessed to determine whether a product is acceptable. Defect assessment usually involves comparing the captured image with a standard image. Due to the characteristics of AOI machines, this method results in an excessive number of manually reviewed images, leading to low efficiency and accuracy. Whether using manual inspection or traditional image processing methods, defect detection in these images often results in difficulties in defect identification, inaccurate assessments, and excessive false alarms.
[0003] Using AI-based image recognition technology to train a model on a large dataset to identify defects has been widely applied in various visual inspection industries in recent years, including the PCBA industry. However, during the deployment of AI models, the limited number of actual defects on the production line makes it difficult to quickly obtain a large number of defect images for training, resulting in slow improvement in model accuracy, difficulty in completely replacing manual labor, and low execution efficiency.
[0004] In the deployment of AI algorithms, artificial defects are often used to expand the training set. This involves manually placing man-made defects such as tombstoning, misalignment, and insufficient solder on the circuit board surface. The sample circuit board is then imaged using an AOI system, and the images are collected for training. The drawbacks of this method are that artificially created defects cannot fully simulate actual defect morphologies, are time-consuming and labor-intensive, and cannot cover all component types. Furthermore, for defects that cannot be artificially created, such as cold solder joints, missing solder, breakage, and improper solder temperature, this method cannot expand the training set, and its efficiency remains slow. Summary of the Invention
[0005] To address the technical problems existing in the prior art, this invention provides a method for generating SMT defects and simulating image quality for AI image recognition, which uses a small dataset to simulate a large amount of defect data to expand the training set and improve algorithm efficiency.
[0006] To achieve the above objectives, the technical solution of the present invention is as follows:
[0007] A method for generating SMT defects and simulating image quality for AI image recognition includes the following steps:
[0008] Step 1: Label small samples and set parameters;
[0009] Step 2: Background generation;
[0010] Step 3: Pad generation;
[0011] Step four: Component generation and defect generation;
[0012] Step 5, image quality simulation;
[0013] Step 6: Output the annotation file.
[0014] As a preferred technical solution, in step one, the content of the small sample is a small number of defect images. When annotating, polygon annotation is used to distinguish the categories. The categories include foreground, component type, component distortion degree, and whether the component contains a defective background. At the same time, rectangles are used to mark the location of foreign object defects.
[0015] As a preferred technical solution, in step one, the parameters configured in the parameter setting stage include background color, type of immersion gold, lighting mode, contrast mode, noise mode, and information on the proportion of the component to be generated, the proportion to grade N product, and the number of defect images to be generated for the component.
[0016] As a preferred technical solution, in step two, in the background generation stage, a style transfer-based background generation model is used. The model is pre-trained with several feature images of ink on the PCB surface and ink on the circuit. The model uses a generative adversarial network model to simulate the images.
[0017] As a preferred technical solution, the model consists of a pre-trained classifier and a reverse inference module. The pre-trained classifier effectively classifies images using multiple labels, including the color of the ink used for grinding ink and the color of the ink on the line. The reverse inference module expands the image using a random seed and then performs classifier discrimination on the image. When the discrimination result of the classifier is different from the preset result, the reverse inference module is backtracked and adjusted until the preset image can be generated.
[0018] As a preferred technical solution, in step three, the artificial intelligence model generates pads based on the set component positions and component types. The generated pads have corresponding pad characteristics. Different pads are generated multiple times from the same background image for data expansion in the next step.
[0019] As a preferred technical solution, in step four, two parallel generation modules are entered, namely the component generation module and the defect generation module. In the component generation module, the components in the labeled small sample or the corresponding component images are extracted from the local database in advance. The images are fitted and compared with the pad positions in the current image. After scaling and rotation, the images are fitted. According to the preset defect type and defect distribution, the foreground components are simulated in different forms, including mis-keying, inversion, tombstoning, and offset.
[0020] As a preferred technical solution, in the defect generation module, the defect location is first cropped out based on the defect location in the small sample annotation, and the defect is simulated using an adversarial generative network. Then, the simulated defect is randomly stitched together in the image. The foreground defects that can be fitted include foreign objects, beads, peeling, cold sweat, and hair fibers. The image features in the small sample annotation or the relevant defects in the pre-stored industry dataset are used.
[0021] As a preferred technical solution, in step five, the image with the data multiplied is entered into a unified style transfer model for image quality simulation, including color difference, lighting, contrast, and noise.
[0022] As a preferred technical solution, in step six, the images that have undergone data multiplication are uniformly entered into the annotation file output stage. The annotations are output according to the preset annotation format, including the position and background color information of the generated bonding element; the bonding position, type and defect status of the element itself; and the position, status and type of the bonding defects.
[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0024] (1) This invention starts with real defect images from production lines, classifies defects into different simulation methods for different types of defects, and specifically invents different defect multiplication algorithms. While ensuring the authenticity of the images, the defect images can be effectively expanded, allowing deep learning models to identify defect features faster and more accurately.
[0025] (2) In the background simulation stage, the invention style transfer method simulates PCB board color, lighting, contrast, immersion gold, noise, etc., and presets pad positions. In the foreground component simulation, defects such as misaligned keys, inverted components, tombstoning, and offset are preset, and their positions and angles can be simulated. In the foreground defect simulation stage, key image features such as foreign objects, adsorption beads, peeling, and cold solder joints are multiplied by small sample data.
[0026] (3) The overall data multiplication process is deployed in a fully automatic manner. It presets information such as the number of labeled small samples, target quantity, and generation parameters for each type of defect, and can automatically generate datasets and label files. Attached Figure Description
[0027] Figure 1 This is a flowchart of the SMT defect generation and image quality simulation method for AI image recognition according to the present invention;
[0028] Figure 2 These are a small number of defect images in the SMT defect generation and image quality simulation method for AI image recognition of the present invention.
[0029] Figure 3 This is a small sample annotation example image from the SMT defect generation and image quality simulation method for AI image recognition in this invention;
[0030] Figure 4 This is a flowchart of the reverse reasoning module in the SMT defect generation and image quality simulation method for AI image recognition of the present invention.
[0031] Figure 5 This is a component generation diagram in the SMT defect generation and image quality simulation method for AI image recognition of the present invention;
[0032] Figure 6 This is a defect generation diagram in the SMT defect generation and image quality simulation method for AI image recognition of the present invention. Detailed Implementation
[0033] The technical solution of the present invention will be further described below with reference to specific embodiments:
[0034] like Figure 1 As shown, a method for generating SMT defects and simulating image quality for AI image recognition includes the following steps:
[0035] Step 1: Before training and generating the subject, two types of prior information are required: labeled small samples and parameter settings.
[0036] Specifically, such as Figure 2 As shown, the annotation of the small sample consists of a small number of defect images. Polygonal annotation is required to distinguish categories, strictly separating the foreground and background. Category differentiation includes foreground, component type, component distortion degree, and whether the component contains a defective background. Specific information such as background color and gold plating color must also be distinguished. Additionally, rectangular boxes should be used to mark the locations of foreign object defects. Since this invention uses a small sample for amplified annotation with limited data, it can contain only a small number of defects. In the parameter setting stage, background color, gold plating type, lighting method, contrast method, and noise method can be manually configured. Furthermore, the proportion of the components to be generated, the proportion to N-grade products, and the number of defect images to be generated for each component can be configured. Small sample annotation example: Figure 3 As shown.
[0037] Step 2: Background generation;
[0038] Specifically, in the background generation stage, a style transfer-based background generation model is used. This model is pre-trained with several feature images of ink on the PCB surface and ink on circuit lines, and can perform reasonable color replication based on the ink colors specified in the parameter settings. The model uses a generative adversarial network (GAN) model for image simulation. The model consists of two parts. The first part is a pre-trained classifier that can effectively classify images using multiple labels, including information such as the ink color of the PCB surface and the ink color on the circuit lines. The second part is a reverse reasoning module, such as... Figure 4 As shown, this module can randomly expand an image using random seeds. The background generative adversarial network (GAN) first expands the image using random seeds, then performs a classifier to classify the image. When the classifier's result differs from the preset result, the reverse inference module is adjusted backtrackingly until the preset image can be generated.
[0039] Step 3: Pad generation;
[0040] Specifically, in the pad generation module, the artificial intelligence model generates pads based on the set component locations and types, and the generated pads have corresponding pad characteristics. The same background image can generate different pads multiple times for subsequent data expansion.
[0041] Step four: Component generation and defect generation;
[0042] After filling the pad image, the process proceeds to two parallel generation modules: the component generation module and the defect generation module. These two modules are relatively independent; they can generate only components, only defects, or both simultaneously. For example... Figure 5 As shown, in the component generation module, the artificial intelligence algorithm pre-uses components from the labeled small sample or extracts corresponding component images from the local database, fits and compares the images with the pad positions in the current image, and then fits the images after scaling and rotation. Based on the preset defect types and defect distribution, the foreground components can be simulated in different forms such as mis-keying, inverted mounting, tombstoning, and offset.
[0043] like Figure 6 As shown, in the defect generation module, the defect location is first cropped out based on the defect location in the small sample annotation. The defect is then simulated using an adversarial generative network. The simulated defect is then randomly stitched together in the image. Foreground defects that can be fitted include foreign objects, beads, peeling, cold sweat, and hair fibers. The image features in the small sample annotation can also be used, or relevant defects in the pre-stored industry dataset can be used.
[0044] Step 5, image quality simulation;
[0045] Specifically, all images that have undergone data multiplication are ultimately fed into a unified style transfer model for image quality simulation, including color difference, lighting, contrast, and noise. The main function of this module is to simulate the image differences produced by different AOI machines and under different lighting conditions on the same component, and it can also compensate for step noise and random noise generated during the signal transmission through the video stream.
[0046] Step 6: Output the annotation file.
[0047] All images that have undergone data multiplication are uniformly entered into the annotation file output stage. In this output stage, the annotations can be output according to the preset annotation format, including the position and background color information of the generated bonding components; the bonding position, type and defect status of the component itself; and the position, status and type of bonding defects.
[0048] After completing the above steps, a new dataset is generated, which can be used to train an AI model.
[0049] This embodiment is merely a further explanation of the present invention and is not intended to limit the present invention. Those skilled in the art can make non-creative modifications to this embodiment as needed after reading this specification, but as long as they are within the scope of the claims of the present invention, they are protected by patent law.
Claims
1. A method for generating SMT defects and simulating image quality for AI image recognition, characterized in that, Includes the following steps: Step 1: Label small samples and set parameters; Step 2: Background generation; Step 3: Pad generation; Step four: Component generation and defect generation; Step 5, image quality simulation; Step 6: Output the annotation file.
2. The SMT defect generation and image quality simulation method for AI image recognition according to claim 1, characterized in that, In step one, the content of the small sample is a small number of defect images. When annotating, polygon annotation is used to distinguish the categories. The categories include foreground, component type, component distortion degree, and whether the component contains a defective background. At the same time, rectangles are used to mark the location of foreign object defects.
3. The SMT defect generation and image quality simulation method for AI image recognition according to claim 1, characterized in that, In step one, the parameters configured in the parameter setting stage include background color, type of immersion gold, lighting mode, contrast mode, noise mode, and information on the scale of the components to be generated, the scale with N-grade products, and the number of defect images to be generated for the components.
4. The SMT defect generation and image quality simulation method for AI image recognition according to claim 1, characterized in that, In step two, during the background generation stage, a style transfer-based background generation model is used. This model is pre-trained with several feature images of ink on the PCB surface and ink on the circuit lines. The model uses a generative adversarial network model to simulate the images.
5. The SMT defect generation and image quality simulation method for AI image recognition according to claim 4, characterized in that, The model consists of a pre-trained classifier and a reverse inference module. The pre-trained classifier performs multi-label classification of images, with labels including ink color and ink color on lines. The reverse inference module expands the image using a random seed and then performs classifier discrimination on the image. When the classifier's discrimination result is different from the preset result, the reverse inference module is backtracked and adjusted until the preset image can be generated.
6. The SMT defect generation and image quality simulation method for AI image recognition according to claim 1, characterized in that, In step three, the artificial intelligence model generates pads based on the set component locations and component types. The generated pads have corresponding pad characteristics. Different pads are generated multiple times from the same background image to prepare for data expansion in the next step.
7. The SMT defect generation and image quality simulation method for AI image recognition according to claim 1, characterized in that, In step four, two parallel generation modules are entered: the component generation module and the defect generation module. In the component generation module, components from the labeled small sample or corresponding component images extracted from the local database are used in advance. The images are fitted and compared with the pad positions in the current image. After scaling and rotation, the images are fitted together. Based on the preset defect types and defect distribution, the foreground components are simulated in different forms, including mis-keying, inversion, tombstoning, and offset.
8. The SMT defect generation and image quality simulation method for AI image recognition according to claim 7, characterized in that, In the defect generation module, the defect location is first cropped out based on the defect location in the small sample annotation, and the defect is simulated using an adversarial generative network. Then, the simulated defect is randomly stitched together in the image. The foreground defects that can be fitted include foreign objects, beads, peeling, cold sweat, and hair fibers. The image features in the small sample annotation or the relevant defects in the pre-stored industry dataset are used.
9. The SMT defect generation and image quality simulation method for AI image recognition according to claim 1, characterized in that, In step five, the image with the data multiplied is then entered into a unified style transfer model for image quality simulation, including color difference, lighting, contrast, and noise.
10. The SMT defect generation and image quality simulation method for AI image recognition according to claim 6, characterized in that, In step six, the images that have undergone data multiplication are uniformly entered into the annotation file output stage. The annotations are output according to the preset annotation format, including the position and background color information of the generated bonding element; the bonding position, type and defect status of the element itself; and the position, status and type of the bonding defects.