Machine learning image segmentation system and system for generating flawless circuit board image

Abstract

A machine learning image segmentation system includes a machine learning image segmentation model, an image scanning device configured to scan a circuit board to generate a circuit board image; the machine learning image segmentation model outputs the two-dimensional image of the board to be detected as a mass production detection standard two-dimensional image according to the circuit board image; the image segmentation model training method comprises the following steps: using a circuit board image and an artificial mark detection standard two-dimensional image as a training sample; inputting the training sample into an image segmentation model to generate a two-dimensional image, and calculating an error between the two-dimensional image output by the model and an artificial mark detection standard two-dimensional image; according to the network parameters of the error optimization model, repeatedly executing the previous steps to minimize the error so as to complete the establishment of an image segmentation model; and then, inputting a circuit board image to the model, and outputting the two-dimensional image of the board to be detected as a mass production detection standard two-dimensional image.

Description

Technical Field

[0001] The present invention relates to an image segmentation system and a system for generating a circuit board image, and more particularly to a machine learning image segmentation system and a system for generating a flawless circuit board image. Background Art

[0002] Automated Optical Inspection (AOI) is now widely used in various fields, especially in the circuit board industry, using optical lenses to replace human eyes to detect the position of components and check whether there are missing, reversed or incorrect components. Generally speaking, before performing automatic optical inspection on a circuit board, it is necessary to provide a detection standard circuit board image for the circuit board to be inspected, so that the automatic optical inspection equipment can inspect the circuit board to be inspected according to the detection standard circuit board image.

[0003] There are two existing methods for providing the inspection standard circuit board image. One is to use the negative image (i.e., Gerber file format) of the circuit board to be inspected when it was designed as the inspection standard circuit board image. However, the negative image (i.e., Gerber file format) of the circuit board when it was designed is an ideal circuit board (i.e., zero defect). Since the negative image and the inspection motherboard image have appearance differences, it is easy to judge the differences that do not constitute product problems as product defects, resulting in over-killing, which affects the product yield figures.

[0004] Another method of providing the inspection standard circuit board image is to use a defect-free circuit board that has been inspected as an inspection mother board, and to scan the negative image (i.e., Gerber file format) generated by the inspection mother board as the inspection standard circuit board image. However, each circuit board has individual differences during actual manufacturing. The method of using the negative image generated by the inspection mother board as the inspection standard circuit board image is prone to misjudgment due to the individual differences of each circuit board, affecting the product yield figures. Summary of the invention

[0005] Therefore, the present invention provides an image segmentation system and a system for generating a flawless circuit board image to solve the above-mentioned problems.

[0006] In order to achieve the above-mentioned purpose, the present invention discloses an image segmentation system, which inputs a circuit board image generated by scanning a circuit board through an image scanning device into the image segmentation system. The image segmentation system includes a machine learning image segmentation model, and outputs N two-dimensional images of the board to be inspected according to the circuit board image as a mass production inspection standard two-dimensional image group, where N is a positive integer.

[0007] According to one embodiment of the present invention, the circuit board image is further used to generate N standard two-dimensional images for artificial mark detection, and the N standard two-dimensional images for artificial mark detection are input into the image segmentation system. The machine learning image segmentation model generates N two-dimensional images based on the circuit board image and the N standard two-dimensional images for artificial mark detection; calculates an error function between the N standard two-dimensional images for artificial mark detection and the N two-dimensional images; adjusts the weight parameters of the machine learning image segmentation model based on the error function; and sets the weight parameter that makes the error function a relative minimum value as a training completion weight parameter.

[0008] In order to achieve the above-mentioned purpose, the present invention further discloses an image segmentation system, which inputs a circuit board image generated by scanning a circuit board through an image scanning device into the image segmentation system, and inputs N film images of the circuit board when it is designed into the image segmentation system. The image segmentation system includes a machine learning image segmentation model, which outputs N two-dimensional images of the board to be inspected as a standard two-dimensional image group for mass production inspection based on the circuit board image and the N film images, where N is a positive integer.

[0009] According to one embodiment of the present invention, the circuit board image is further used to generate N standard two-dimensional images for artificial mark detection, and the N standard two-dimensional images for artificial mark detection are input into the image segmentation system. The machine learning image segmentation model generates N two-dimensional images based on the circuit board image, the N film images and the N standard two-dimensional images for artificial mark detection; calculates an error function between the N standard two-dimensional images for artificial mark detection and the N two-dimensional images; adjusts the weight parameters of the machine learning image segmentation model based on the error function; and sets the weight parameter that makes the error function a relative minimum value as a training completion weight parameter.

[0010] In order to achieve the above-mentioned purpose, the present invention further discloses a system for generating a defect-free circuit board image. A plurality of circuit board images generated by scanning a plurality of circuit boards through an image scanning device and a standard circuit board image generated by scanning a golden sample circuit board are input into the system. The system includes a screening system for calculating the difference between each of the circuit board images and the standard circuit board image to generate a plurality of difference values. When one of the plurality of difference values ​​does not exceed a threshold, one of the plurality of difference values ​​is retained; or when another one of the plurality of difference values ​​exceeds the threshold, another one of the plurality of difference values ​​is deleted.

[0011] According to one embodiment of the present invention, the screening system calculates an average or a median of the pixel color value of one of the plurality of difference values ​​to output the defect-free circuit board image.

[0012] In summary, the present invention uses the scanned circuit board image and / or the N negative film images (i.e., Gerber file format) of the circuit board during design as the input of the machine learning image segmentation model, and allows the machine learning image segmentation model to output the N two-dimensional images based on the circuit board image and / or the N negative film images (i.e., Gerber file format). Since the input includes at least the circuit board image generated by scanning the physical circuit board, rather than just the ideal negative film image during design, it is possible to avoid over-killing by judging differences that do not constitute product problems as defective products. In addition, by training the machine learning image segmentation model, the mass production inspection standard two-dimensional image generated by the trained machine learning image segmentation model can eliminate the differences between each physical circuit board to eliminate misjudgment caused by individual differences of each circuit board. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Schematic diagram of an image segmentation system according to an embodiment of the present invention.

[0014] Figure 2 The figure is a flow chart of a method for training an image segmentation system according to an embodiment of the present invention.

[0015] Figure 3 FIG. 4 is a schematic diagram of an image segmentation system according to another embodiment of the present invention.

[0016] Figure 4 The figure is a flow chart of a method for training an image segmentation system according to an embodiment of the present invention.

[0017] Figure 5 FIG. 2 is a schematic diagram of a system for generating a defect-free circuit board image according to another embodiment of the present invention. DETAILED DESCRIPTION

[0018] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an image segmentation system 1000 according to an embodiment of the present invention. Figure 1As shown, the image scanning device 1 is used to scan the circuit board 3' to generate the circuit board image 3, and the circuit board image 3 generated by scanning the circuit board 3' by the image scanning device 1 is input into the image segmentation system 1000. Further, the image segmentation system 1000 includes a machine learning image segmentation model 2, and the machine learning image segmentation model 2 outputs N two-dimensional images of the board to be detected according to the circuit board image 3 as a mass production inspection standard two-dimensional image group 4, wherein N is a positive integer, and in the present embodiment, N is 5, that is, the machine learning image segmentation model 2 outputs 5 two-dimensional images of the board to be detected according to the circuit board image 3, that is, the mass production inspection standard two-dimensional image group 4 includes 5 two-dimensional images of the board to be detected, and the 5 two-dimensional images of the board to be detected are respectively a first two-dimensional image 41 of the board to be detected, a second two-dimensional image 42 of the board to be detected, a third two-dimensional image 43 of the board to be detected, a fourth two-dimensional image 44 of the board to be detected, and a fifth two-dimensional image 45 of the board to be detected.

[0019] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a method for training an image segmentation system 1000 according to an embodiment of the present invention. Figure 2 As shown, the method for training the image segmentation system 1000 comprises the following steps:

[0020] Step S100: Generate N standard two-dimensional images for manual marking detection from the circuit board image, and input the circuit board image and the N standard two-dimensional images for manual marking detection into the image segmentation system.

[0021] Step S101: Generate N two-dimensional images according to the circuit board image.

[0022] Step S102: Calculate an error function between the N manual marking detection standard two-dimensional images and the N two-dimensional images.

[0023] Step S103: adjusting the weight parameters of the machine learning image segmentation model according to the error function.

[0024] Step S104: The weight parameter that makes the error function a relative minimum value is determined as a training completion weight parameter.

[0025] The above steps are described in detail below. First, the image scanning device 1 scans the circuit board 3' to generate the circuit board image 3. In step S100, the circuit board image 3 can generate N artificial mark detection standard two-dimensional images (i.e., the first artificial mark detection standard two-dimensional image 30, the second artificial mark detection standard two-dimensional image 31, the third artificial mark detection standard two-dimensional image 32, the fourth artificial mark detection standard two-dimensional image 33 and the fifth artificial mark detection standard two-dimensional image 34), and the circuit board image 3 and the artificial mark detection standard two-dimensional image are input into the machine learning image segmentation model 2. In step S101, the machine learning image segmentation model 2 can generate N two-dimensional images (not shown) based on the circuit board image 3. In practice, each artificial mark detection standard two-dimensional image can be generated manually in the form of computer drawing, so the N artificial mark detection standard two-dimensional images are an electronic drawing file, which serves as a reference for the machine learning image segmentation model 2 to learn to segment the circuit board image 3 into the N two-dimensional images. In other words, the N manually labeled detected standard two-dimensional images provide a learning basis for the machine learning image segmentation model 2 to generate the N two-dimensional images, that is, the N manually labeled detected standard two-dimensional images can be the labels (Label) of the machine learning image segmentation model 2 in the learning process, as the basis for defining the error function (LossFunction).

[0026] Therefore, in step S102, the machine learning image segmentation model 2 can calculate an error function between the N manual marking detection standard two-dimensional images and the N two-dimensional images. In this embodiment, the error function satisfies the following equation:

[0027]

[0028] Where X i is the pixel value of one of the N two-dimensional images output, Y i The pixel value of one of the N manually marked detection standard two-dimensional images, where N is 5.

[0029] In step S103, the weight parameters of the machine learning image segmentation model 2 are adjusted according to the error function. In this embodiment, a set of weight parameters is randomly selected and substituted into the machine learning image segmentation model 2, so that the corresponding N two-dimensional images can be output. Then, the N two-dimensional images and the N manually marked detection standard two-dimensional images are compared according to equation (1), and the error between the output of the machine learning image segmentation model 2 and the manual marking is calculated under the set of weight parameters.

[0030] In step S104, the above random selection process is repeated (for example, 10,000 times) and a minimum error value is found among the errors generated by all the selected weight parameters, which is defined as a relative minimum value, and then the weight parameter that makes the error function the relative minimum value is defined as a trained weight parameter to complete the training of the machine learning image segmentation model 2. It is worth mentioning that in the training method of the machine learning image segmentation model 2, the method of randomly selecting weight parameters is only an example of an embodiment of the present invention, and the present invention is not limited thereto. For example, the gradient descent method can also be used to select weight parameters, and then the weight parameters selected by the gradient descent method are substituted into the machine learning image segmentation model 2 to find the weight parameter that can make the error function the relative minimum value, and the weight parameter is defined as a trained weight parameter to complete the training of the machine learning image segmentation model 2.

[0031] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an image segmentation system 1000′ of another embodiment of the present invention. The main difference between the image segmentation system 1000′ and the above-mentioned image segmentation system 1000 is that, in addition to inputting the circuit board image 3 and the standard two-dimensional image for manual mark detection into the image segmentation system 1000′, the circuit board 3′ (such as Figure 1 As shown in FIG. 1 , N negative film images are input into the image segmentation system 1000 ′ during design. In this embodiment, the circuit board 3 ′ (eg Figure 1 ) is designed with a total of 5 negative film images, namely a first negative film image 51, a second negative film image 52, a third negative film image 53, a fourth negative film image 54 and a fifth negative film image 55, but the present invention is not limited thereto.

[0032] In other words, the standard two-dimensional image detected by the manual marking is used as a reference for the machine learning image segmentation model 2 to learn to segment the circuit board image 3 and the first film image 51, the second film image 52, the third film image 53, the fourth film image 54 and the fifth film image 55 into the N two-dimensional images, so as to serve as the basis for defining the error function (Loss Function). The components in this embodiment with the same number as those in the above embodiment have the same structure and function, and are not repeated here for the sake of brevity.

[0033] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of a method for training an image segmentation system 1000′ according to an embodiment of the present invention. Figure 4 As shown, the method for training the image segmentation system 1000' comprises the following steps:

[0034] Step S200: Generate N standard two-dimensional images for manual marking detection from the circuit board image, and input the circuit board image and the N standard two-dimensional images for manual marking detection into the image segmentation system.

[0035] Step S201: Generate N two-dimensional images according to the circuit board image and the N film images.

[0036] Step S202: Calculate an error function between the N manual marking detection standard two-dimensional images and the N two-dimensional images.

[0037] Step S203: adjusting the weight parameters of the machine learning image segmentation model according to the error function.

[0038] Step S204: The weight parameter that makes the error function a relative minimum value is determined as a training completion weight parameter.

[0039] Depend on Figure 2 Method for training image segmentation system 1000 and Figure 4 It can be seen from the method of training image segmentation system 1000' that the difference between the two is that the method of training image segmentation system 1000' adds the first film image 51, the second film image 52, the third film image 53, the fourth film image 54 and the fifth film image 55 as the input of the machine learning image segmentation model 2 to generate N two-dimensional images, and the rest are the same. For the sake of brevity, they are not repeated here.

[0040] It is worth mentioning that the present invention may further include a pre-processing operation, which is used to generate a defect-free circuit board image. Figure 5 , Figure 5 FIG. 2 is a schematic diagram of a system 2000 for generating a defect-free circuit board image according to another embodiment of the present invention.

[0041] The system 2000 includes a screening system. In practice, the screening system can be a functional module of the machine learning image segmentation model 2 (i.e., a neural network). In addition, a golden sample circuit board can be manually selected, i.e., a circuit board that is manually identified as relatively free of major defects, and then the golden sample circuit board is scanned to generate a standard circuit board image 6, and then the standard circuit board image 6 is input into the system 2000. In addition, a plurality of circuit board images (four in this embodiment, i.e., a first circuit board image 70, a second circuit board image 71, a third circuit board image 72, and a fourth circuit board image 73) generated by scanning a plurality of circuit boards are input into the system 2000.

[0042] Thus, the screening system is used to calculate the difference between each of the PCB images (i.e., the first PCB image 70, the second PCB image 71, the third PCB image 72, and the fourth PCB image 73) and the standard PCB image 6 to generate a plurality of difference values. When one of the plurality of difference values ​​does not exceed a threshold, one of the plurality of difference values ​​is retained; when one of the plurality of difference values ​​exceeds a threshold, one of the plurality of difference values ​​is deleted. In other words, all the PCB images of the plurality of difference values ​​that do not exceed the threshold are retained, and all the PCB images of the plurality of difference values ​​that exceed the threshold are deleted. In this embodiment, the PCB image 70 and the fourth PCB image 73 have a defect 91 and a defect 90, respectively, so the PCB image 70 and the fourth PCB image 73 exceed the threshold and are deleted; the second PCB image 71 and the third PCB image 72 do not have obvious defects, so the second PCB image 71 and the third PCB image 72 do not exceed the threshold and are retained.

[0043] In this way, the screening system can further select the circuit board image (with Figure 5 For example, the pixel color values ​​of the second PCB image 71 and the third PCB image 72 are averaged or median calculated to output the flawless PCB image.

[0044] Compared to the prior art, the present invention uses the scanned circuit board image and / or the N negative film images (i.e., Gerber file format) of the circuit board during design as the input of the machine learning image segmentation model, and allows the machine learning image segmentation model to output the N two-dimensional images based on the circuit board image and / or the N negative film images (i.e., Gerber file format). Since the input includes at least the circuit board image generated by scanning the physical circuit board, rather than just the ideal negative film image during design, it can avoid over-killing by judging differences that do not constitute product problems as defective products. In addition, by training the machine learning image segmentation model, the mass production inspection standard two-dimensional image generated by the trained machine learning image segmentation model can eliminate the differences between each physical circuit board to eliminate misjudgment caused by individual differences of each circuit board.

Claims

1. An image segmentation system, wherein a circuit board image generated by scanning a circuit board through an image scanning device is input into the image segmentation system, wherein the image segmentation system is characterized by: The machine learning image segmentation model outputs N two-dimensional images of the board to be inspected according to the circuit board image as a standard two-dimensional image group for mass production inspection, where N is a positive integer.

2. The image segmentation system according to claim 1, characterized in that: The circuit board image is further used to generate N standard two-dimensional images for artificial marking detection, and the N standard two-dimensional images for artificial marking detection are input into the image segmentation system. The machine learning image segmentation model: Generate N two-dimensional images according to the circuit board image; Calculating an error function between the N standard two-dimensional images for manual marking detection and the N two-dimensional images; According to the error function, adjusting the weight parameters of the machine learning image segmentation model; and The weight parameter that makes the error function a relative minimum value is defined as a training completed weight parameter.

3. The image generation system according to claim 2, characterized in that: The error function satisfies the following equation: Among them C i is the pixel value of one of the N two-dimensional images output, Y i A pixel value of one of the N manually marked detection standard two-dimensional images.

4. An image segmentation system, wherein a circuit board image generated by scanning a circuit board by an image scanning device is input into the image segmentation system, and N negative images of the circuit board when it is designed are input into the image segmentation system, wherein: The machine learning image segmentation model outputs N two-dimensional images of the board to be inspected as a standard two-dimensional image group for mass production inspection based on the circuit board image and the N negative film images, where N is a positive integer.

5. The image segmentation system according to claim 4, characterized in that: The circuit board image is further used to generate N standard two-dimensional images for artificial marking detection, and the N standard two-dimensional images for artificial marking detection are input into the image segmentation system. The machine learning image segmentation model: Generate N two-dimensional images according to the circuit board image and the N negative film images; Calculating an error function between the N standard two-dimensional images for manual marking detection and the N two-dimensional images; According to the error function, adjusting the weight parameters of the machine learning image segmentation model; and The weight parameter that makes the error function a relative minimum value is defined as a training completed weight parameter.

6. The image generation system according to claim 5, characterized in that: The error function satisfies the following equation: Where X i is the pixel value of one of the N two-dimensional images output, Y i A pixel value of one of the N manually marked detection standard two-dimensional images.

7. A system for generating a flawless circuit board image, wherein a plurality of circuit board images generated by scanning a plurality of circuit boards by an image scanning device and a standard circuit board image generated by scanning a golden sample circuit board are input into the system, wherein: The screening system calculates the difference between each of the circuit board images and the standard circuit board image to generate a plurality of difference values, wherein: When one of the plurality of difference values ​​does not exceed a threshold, retaining one of the plurality of difference values; or When another one of the plurality of difference values ​​exceeds the threshold, another one of the plurality of difference values ​​is deleted.

8. The system according to claim 7, characterized in that The screening system calculates an average or a median of the pixel color values ​​of the circuit board image of one of all the plurality of difference values ​​to output the defect-free circuit board image.

Images