Defect sample generation method and device, automatic optical detector and storage medium

Abstract

The application discloses a kind of defect sample generation method, device, automatic optical detector and storage medium.The defect sample generation method includes: the normal sample output after being detected with AOI detection equipment as research object, and using component positioning method to extract solder joint image, by analyzing the characteristics of defect sample solder joint area, the qualitative characteristics of defect are converted into the quantitative characteristics convenient for modeling analysis, and the complete scheme of defect sample generation is provided using graphic modeling and image processing technology.The generation of a variety of defect samples is realized, and it is convenient to batch generate defect samples, and has strong innovation and application value.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology for automatic optical inspection instruments, and in particular to a defect sample generation method, apparatus, automatic optical inspection instrument, and storage medium. Background Technology

[0002] AOI (Auto Optical Inspection) is an important piece of equipment in SMT (Surface Mount Technology) processes. It is based on optical principles to detect common defects encountered in the soldering production process. Using optical principles, the camera on the AOI inspection equipment scans the PCB to acquire PCB images, and compares the solder joint data on the acquired PCB images with qualified data in the machine's database. After image processing, the PCB soldering quality is marked.

[0003] With the rapid development of deep learning image detection technology and its application in multiple fields, it has demonstrated excellent performance in automatic feature extraction and end-to-end detection. Applying deep learning technology to AOI defect detection has significant research value and practical significance. Currently, researchers mainly use three methods to generate defect samples: First, synthesizing defect sample images using 2D image processing software. However, defect samples synthesized by image processing software have a certain degree of subjectivity and cannot synthesize texture characteristics with randomness. This requires manual interaction and cannot be applied in industrial automated production. Second, generating defect sample images through 3D modeling. However, 3D modeling cannot simulate the tower light source of real AOI inspection equipment. Third, generating defect sample images based on GAN (Greek Adversarial Network). However, the effect of the generated defect samples differs significantly from real defects. Summary of the Invention

[0004] This invention provides a method, apparatus, automatic optical inspection instrument, and storage medium for generating defect samples, in order to solve the problem that current defect sample generation cannot be applied in industrial automation.

[0005] According to one aspect of the present invention, a defect sample generation method is provided, the defect sample generation method comprising:

[0006] An AOI inspection device is used to inspect the printed circuit board under test to obtain an initial printed circuit board sample image, and the component solder joint area is obtained by locating the components based on the initial printed circuit board sample image.

[0007] The component solder joint area is divided into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, and a corresponding color library image is established based on the color features of the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area.

[0008] An elliptical model is created in the solder joint area of ​​the component, and a defect sample image of the printed circuit board is generated based on the color library image and the elliptical model.

[0009] Optionally, the step of obtaining an initial printed circuit board sample image by inspecting the printed circuit board to be tested using an AOI inspection device includes:

[0010] An AOI (Automated Optical Inspection) device captures images of the printed circuit board under test from top to bottom through the light source in the AOI device, obtaining an initial image of the printed circuit board sample. The light source consists of red, green, and blue light sources distributed from top to bottom.

[0011] Optionally, after obtaining an initial printed circuit board sample image by inspecting the printed circuit board under test using an AOI inspection device, the method further includes:

[0012] The initial printed circuit board sample image is filtered.

[0013] The initial printed circuit board sample image after filtering is separated into RGB three channels, and the red image of the printed circuit board sample corresponding to the R channel is extracted and binarized.

[0014] Optionally, the step of locating components and obtaining component solder joint areas based on the initial printed circuit board sample image includes:

[0015] The first and second coordinates of the component are determined based on the red image of the printed circuit board sample after binarization, and the component is located based on the first and second coordinates to obtain the component area on the initial printed circuit board sample image.

[0016] The component solder joint area is determined based on the component area.

[0017] Optionally, the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area respectively include a first solder joint red area, a second solder joint red area, a first solder joint green area, a second solder joint green area, a first solder joint blue area, and a second solder joint blue area;

[0018] The step of establishing a corresponding color library image based on the color features corresponding to the initial red area, the initial green area, and the initial blue area of ​​the solder joint includes:

[0019] The color features of the red area of ​​the first solder joint, the red area of ​​the second solder joint, the green area of ​​the first solder joint, the green area of ​​the second solder joint, the blue area of ​​the first solder joint, and the blue area of ​​the second solder joint are extracted respectively, and a corresponding color library image is established based on the color features.

[0020] Optionally, the step of establishing an elliptical model in the solder joint area of ​​the component and generating a printed circuit board defect sample image based on the color library image and the elliptical model includes:

[0021] In the component solder joint area, a first solder joint area segmentation point and a second solder joint area segmentation point are determined, and a corresponding elliptical model is established based on the first solder joint area segmentation point and the second solder joint area segmentation point.

[0022] Based on the elliptical model, the corresponding color is selected from the corresponding color library to generate a printed circuit board defect sample image.

[0023] Optionally, the defect sample generation method further includes:

[0024] When the abscissa of the first solder joint area segmentation point is within the first threshold range and the abscissa of the second solder joint area segmentation point is within the second threshold range, the generated printed circuit board defect sample image is a printed circuit board insufficient solder defect sample image.

[0025] When the abscissa of the first solder joint area segmentation point is within the third threshold range and the abscissa of the second solder joint area segmentation point is within the fourth threshold range, the generated printed circuit board defect sample image is a printed circuit board cold solder joint defect sample image.

[0026] According to another aspect of the present invention, a defect sample generation apparatus is provided, the defect sample generation apparatus comprising:

[0027] The component solder joint area determination module is used to perform the following: the AOI inspection equipment is used to inspect the printed circuit board under test to obtain an initial printed circuit board sample image, and the components are located and the component solder joint area is obtained based on the initial printed circuit board sample image.

[0028] The color library creation module is used to divide the component solder joint area into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, and to create a corresponding color library image based on the color features of the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area.

[0029] The defect sample image generation module is used to create an elliptical model in the solder joint area of ​​the component and generate a printed circuit board defect sample image based on the color library image and the elliptical model.

[0030] According to another aspect of the present invention, an automatic optical inspection instrument is provided, the automatic optical inspection instrument comprising:

[0031] At least one processor; and

[0032] A memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the defect sample generation method according to any embodiment of the present invention.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the defect sample generation method according to any embodiment of the present invention.

[0035] The technical solution of this invention involves using an AOI (Automated Optical Inspection) device to inspect the printed circuit board (PCB) under test and obtain an initial PCB sample image. Based on this initial PCB sample image, components are located to obtain their solder joint areas. These solder joint areas are then divided into initial red, green, and blue regions. A corresponding color library is established based on the color features of these three regions. An elliptical model is created within the solder joint areas, and a PCB defect sample image is generated based on the color library and the elliptical model. This solution solves the problem that current defect sample generation methods cannot be applied in industrial automation, enabling the generation of multiple defect samples and facilitating batch generation. It demonstrates strong innovation and application value.

[0036] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of a defect sample generation method provided in Embodiment 1 of the present invention;

[0039] Figure 2 This is a schematic diagram of the imaging principle of obtaining an initial printed circuit board sample image by using an AOI inspection device to inspect a printed circuit board under test, according to an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram illustrating the principle of component solder joint area modeling according to an embodiment of the present invention;

[0041] Figure 4 This is a flowchart of a defect sample generation method provided in Embodiment 2 of the present invention;

[0042] Figure 5 This is a schematic diagram of a red image of a printed circuit board sample after binarization processing, provided by an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram of the positioning of component areas on an initial printed circuit board sample image according to an embodiment of the present invention;

[0044] Figure 7 This is a schematic diagram of the positioning of component solder joint images on an initial printed circuit board sample image according to an embodiment of the present invention;

[0045] Figure 8 This is a schematic diagram of component solder joint image segmentation on an initial printed circuit board sample image according to an embodiment of the present invention;

[0046] Figure 9 This is a schematic diagram of a defect sample generation device according to Embodiment 3 of the present invention;

[0047] Figure 10 This is a schematic diagram of the structure of an automatic optical inspection instrument that implements the defect sample generation method of this invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] Example 1

[0051] Figure 1 This is a flowchart of a defect sample generation method provided in Embodiment 1 of the present invention. This embodiment is applicable to the generation of AOI defect samples in an automatic optical inspection instrument based on image processing and solder joint modeling. This defect sample generation method can be executed by a defect sample generation device, which can be implemented in hardware and / or software and can be configured within the automatic optical inspection instrument. Figure 1 As shown, the defect sample generation method includes:

[0052] S110. The AOI inspection equipment is used to inspect the printed circuit board to be tested to obtain an initial printed circuit board sample image, and the components are located and the component solder joint area is obtained based on the initial printed circuit board sample image.

[0053] Printed circuit boards (PCBs) are providers of electrical connections for electronic components. In this embodiment, the PCB under test is a conventional PCB with components arranged on it, such as resistors, capacitors, or other devices.

[0054] AOI inspection equipment, also known as AOI optical automatic inspection equipment, has become an important inspection tool and process quality control tool for ensuring product quality in the electronics manufacturing industry. In this embodiment, the AOI inspection equipment is used to photograph the components on the printed circuit board under test. Optionally, an initial printed circuit board sample image can be obtained by taking pictures with the digital camera in the AOI inspection equipment.

[0055] It is understood that the initial printed circuit board sample image is a normal sample output by the AOI inspection equipment. The AOI inspection equipment captures the image of the printed circuit board under test from top to bottom through the light source in the AOI inspection equipment, and the light source consists of red, green and blue light sources distributed from top to bottom.

[0056] The light source in the AOI inspection equipment can be an LED light source or other light sources; this embodiment does not impose any restrictions on this. Optionally, the light source in the AOI inspection equipment can be a red LED light source, a green LED light source, and a blue LED light source distributed from top to bottom.

[0057] For details, see Figure 2 Taking digital camera 21 as an example, the digital camera 21 in the AOI inspection device takes pictures of the printed circuit board to be tested from top to bottom through the light source in the AOI inspection device, and obtains the initial printed circuit board sample image of the printed circuit board 25 to be tested. The light source is a red light source 22, a green light source 23 and a blue light source 24 distributed from top to bottom.

[0058] For a qualified solder joint pattern of the printed circuit board 25, if the red light source 22 illuminates the solder pad area of ​​the printed circuit board 25 perpendicularly (i.e., the solder pad area of ​​the printed circuit board 25 is sufficiently flat), then the solder pad area of ​​the printed circuit board 25 will appear red. If the solder joint area of ​​the printed circuit board 25 has a slope or other structure, then the light from the green light source 23 and the blue light source 24 will be reflected back to the digital camera 21, and at this time, the solder joint area of ​​the printed circuit board 25 will appear blue-green.

[0059] Furthermore, after acquiring the initial printed circuit board sample image, due to factors such as deformation of the carrier or board material, micro-vibration of the platform, instability of the light source, and dirt on the pads, the acquired initial printed circuit board sample image often contains various kinds of noise. In order to facilitate subsequent image analysis and understanding, the initial printed circuit board sample image is filtered. Optionally, the filtering process can be implemented by a mean filter.

[0060] Based on the above, the initial printed circuit board sample image after filtering is separated into RGB three channels, and the red image of the printed circuit board sample corresponding to the R channel is extracted and binarized. The component area on the initial printed circuit board sample image is obtained based on the binarized red image of the printed circuit board sample, and the component solder joint area is determined based on the component area.

[0061] S120. Divide the component solder joint area into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, and establish a corresponding color library map based on the color features of the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area.

[0062] After determining the component solder joint area, the component solder joint area can be divided into the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area by the edge segmentation algorithm. Other existing segmentation algorithms can also be used to achieve region segmentation, and this embodiment does not impose any restrictions on this.

[0063] The initial red area of ​​the solder joint includes the first red area of ​​the solder joint and the second red area of ​​the solder joint; the initial green area of ​​the solder joint includes the first green area of ​​the solder joint and the second green area of ​​the solder joint; and the initial blue area of ​​the solder joint includes the first blue area of ​​the solder joint and the second blue area of ​​the solder joint.

[0064] Furthermore, the color features corresponding to the red area of ​​the first solder joint, the red area of ​​the second solder joint, the green area of ​​the first solder joint, the green area of ​​the second solder joint, the blue area of ​​the first solder joint, and the blue area of ​​the second solder joint are extracted respectively, that is, the colors of the six areas are extracted, and a corresponding color library map is established based on the color features.

[0065] S130. An elliptical model is established in the solder joint area of ​​the component, and a printed circuit board defect sample image is generated based on the color library image and the elliptical model.

[0066] Based on the above, a grayscale transformation is performed on the model according to the position and color characteristics of the component solder joint area. Specifically, a Cartesian coordinate system is established with the printed circuit board sample particle as the origin, as shown below. Figure 3 As shown, since the ellipse has symmetry, that is, F and H are symmetrical with G and E, we take points F and H as an example to illustrate. By adjusting the horizontal coordinates of points F and H, we can reflect different types of defect samples and thus obtain images of different types of defect samples.

[0067] It is understandable that, for different types of defects, this embodiment can obtain corresponding printed circuit board defect sample images, i.e., multiple printed circuit board defect sample images, which facilitates the batch generation of defect samples. This is of great help to the testing of AOI defect detection algorithms, and at the same time, it is more conducive to the research and development and testing of deep learning image detection algorithms.

[0068] The sizes of the three-color regions in the solder joint area of ​​a printed circuit board defect sample image are related. An elliptical model is created in the solder joint area, and the vertices of the ellipse are adjusted to change the size of each color region, thereby generating the defect sample. See also... Figure 3The first ellipse model is obtained by fitting A, B, C, D, E, and F, and the second ellipse model is obtained by fitting A, B, C, D, G, and H. Different color regions can be determined based on the boundaries between the ellipse and the rectangle. Finally, the corresponding color is selected from the color library to generate a printed circuit board defect sample image.

[0069] Based on the above embodiments, there are certain texture features in the red, green and blue transition area of ​​the component solder joint area. The process of obtaining the elliptical model may also have obvious gaps at the edges. These gaps can be eliminated by feathering algorithm or by other existing conventional algorithms. This embodiment does not impose any restrictions.

[0070] The technical solution of this invention involves using an AOI (Automated Optical Inspection) device to inspect the printed circuit board (PCB) under test and obtain an initial PCB sample image. Based on this initial PCB sample image, components are located to obtain their solder joint areas. These solder joint areas are then divided into initial red, green, and blue regions. A corresponding color library is established based on the color features of these three regions. An elliptical model is created within the solder joint areas, and a PCB defect sample image is generated based on the color library and the elliptical model. This solution solves the problem that current defect sample generation methods cannot be applied in industrial automation, enabling the generation of multiple defect samples and facilitating batch generation. It demonstrates strong innovation and application value.

[0071] Example 2

[0072] Figure 4 This is a flowchart of a defect sample generation method provided in Embodiment 2 of the present invention. This embodiment provides an optional implementation method based on the above embodiments. Figure 4 As shown, the defect sample generation method includes:

[0073] S410. The AOI inspection equipment takes pictures of the printed circuit board to be tested from top to bottom through the light source in the AOI inspection equipment to obtain an initial printed circuit board sample image. The light source is a red light source, a green light source and a blue light source distributed from top to bottom.

[0074] Specifically, the printed circuit board to be inspected is placed in an AOI inspection device, and the AOI inspection device takes pictures of the printed circuit board from top to bottom through the light source in the AOI inspection device to obtain an initial sample image of the printed circuit board.

[0075] S420. Filter the initial printed circuit board sample image.

[0076] S430. The initial printed circuit board sample image after filtering is separated into RGB three channels, and the red image of the printed circuit board sample corresponding to the R channel is extracted and binarized.

[0077] S440. Determine the first coordinate and the second coordinate of the component based on the red image of the printed circuit board sample after binarization, and locate the component based on the first coordinate and the second coordinate to obtain the component area on the initial printed circuit board sample image.

[0078] See Figure 5 As shown, the red image of the printed circuit board sample after image binarization is a black and white image with pixel grayscale values ​​of only 0 and 255. Further, the first coordinate 51 and the second coordinate 52 of the components are determined using a rectangle fitting algorithm, i.e., as shown... Figure 5 The top-left corner coordinates (x0, y0) are the first coordinate 51, and the bottom-right corner coordinates (x1, y1) are the second coordinate 52. Then, a rectangle is drawn on the initial printed circuit board sample image with the first and second coordinates as diagonal vertices. See [link to relevant documentation]. Figure 6 As shown.

[0079] S450. Determine the component solder joint area based on the component area.

[0080] Specifically, by setting the pixel grayscale value of the component area to match the background color, an image of the component solder joint with a clear contrast to the background grayscale can be obtained, such as... Figure 7 As shown, the component solder joint area on the left is: Ω1=(0,x0)*(0,h), and the component solder joint area on the right is: Ω2=(x1,w)*(0,h), where w is the width of the initial printed circuit board sample image and h is the height of the initial printed circuit board sample image.

[0081] S460. Divide the component solder joint area into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, wherein the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area respectively include a first solder joint red area, a second solder joint red area, a first solder joint green area, a second solder joint green area, a first solder joint blue area, and a second solder joint blue area.

[0082] S470. Extract the color features of the red area of ​​the first solder joint, the red area of ​​the second solder joint, the green area of ​​the first solder joint, the green area of ​​the second solder joint, the blue area of ​​the first solder joint, and the blue area of ​​the second solder joint, and establish a corresponding color library image based on the color features.

[0083] For details, please refer to [link / reference]. Figure 8, after the component solder joint area is segmented in step S460, the component solder joint area can be divided into 6 corresponding areas, the colors of the 6 areas are extracted and a corresponding color library diagram is created, as shown in Table 1 below.

[0084] Table 1 Color Library Table

[0085]

[0086] S480. Determine the first solder joint area segmentation point and the second solder joint area segmentation point in the component solder joint area, and establish a corresponding ellipse model according to the first solder joint area segmentation point and the second solder joint area segmentation point.

[0087] Exemplarily, continue to refer to Figure 3 , establish a rectangular coordinate system with the mass point of the initial printed circuit board sample image as the origin, and obtain the coordinates of points A, B, C, D, E, F, G, and H on the initial printed circuit board sample image. Among them, the first solder joint area segmentation point is F(a, b), and the second solder joint area segmentation point is H(c, d). Due to the symmetry of the ellipse, different types of defect samples can be generated according to the change of the abscissas of the two points, the first solder joint area segmentation point F and the second solder joint area segmentation point H.

[0088] It can be known that the first threshold range is [T1, T2], and the second threshold range is [T3, T4]. When the abscissa of the first solder joint area segmentation point is within the first threshold range and the abscissa of the second solder joint area segmentation point is within the second threshold range, the generated printed circuit board defect sample image is a printed circuit board solder shortage defect sample image, that is, when T1 < a < T2 and T3 < b < T4, the generated printed circuit board defect sample image is a printed circuit board solder shortage defect sample image.

[0089] The third threshold range is [T5, T6], and the fourth threshold range is [T7, T8]. When the abscissa of the first solder joint area segmentation point is within the third threshold range and the abscissa of the second solder joint area segmentation point is within the fourth threshold range, the generated printed circuit board defect sample image is a printed circuit board dry joint defect sample image, that is, when T5 < a < T6 and T7 < b < T8, the generated printed circuit board defect sample image is a printed circuit board dry joint defect sample image.

[0090] Among them, T1, T2, T3, T4, T5, T6, T7, and T8 are thresholds.

[0091] Furthermore, when a and b are randomly taken within the thresholds, the coordinates of the two points, the first solder joint area segmentation point F and the second solder joint area segmentation point H, will also change. Then, the first ellipse model is obtained by fitting A, B, C, D, E, and F, and the second ellipse model is obtained by fitting A, B, C, D, G, and H.

[0092] S490. Select the corresponding color from the corresponding color library according to the ellipse model to generate a printed circuit board defect sample image.

[0093] It should be noted that the above description of printed circuit board defects is only an example. By establishing the elliptical model, this embodiment can also generate various defect samples such as tombstone, offset, bridging, crack, porosity, missing parts, and wrong parts, which will not be elaborated here.

[0094] The technical solution of this invention utilizes a component localization method to extract component solder joint images, demonstrating strong innovation and application value. Furthermore, existing defect generation methods often rely on grayscale transformations of component solder joint areas based on empirical thresholds, limiting their application value. This invention, however, designs a mapping relationship between pixel values ​​and positions based on the positional relationship between the component solder joint area and the component itself. This transforms the qualitative features of defects into quantitative features that facilitate modeling and analysis, enabling the batch generation of defect samples. Simultaneously, it greatly assists in the testing of AOI defect detection algorithms and further leverages the research and testing of deep learning algorithms.

[0095] Example 3

[0096] Figure 9 This is a schematic diagram of a defect sample generation device provided in Embodiment 3 of the present invention.

[0097] like Figure 9 As shown, the defect sample generation device includes:

[0098] The component solder joint area determination module 910 is used to perform the following: the AOI inspection equipment is used to inspect the printed circuit board to be tested to obtain an initial printed circuit board sample image, and the components are located and the component solder joint area is obtained based on the initial printed circuit board sample image.

[0099] The color library creation module 920 is used to divide the component solder joint area into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, and to create a corresponding color library image based on the color features of the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area.

[0100] The defect sample image generation module 930 is used to perform the following: establish an elliptical model in the solder joint area of ​​the component, and generate a printed circuit board defect sample image based on the color library image and the elliptical model.

[0101] Optionally, the step of obtaining an initial printed circuit board sample image by inspecting the printed circuit board to be tested using an AOI inspection device includes:

[0102] An AOI (Automated Optical Inspection) device captures images of the printed circuit board under test from top to bottom through the light source in the AOI device, obtaining an initial image of the printed circuit board sample. The light source consists of red, green, and blue light sources distributed from top to bottom.

[0103] Optionally, after obtaining an initial printed circuit board sample image by inspecting the printed circuit board under test using an AOI inspection device, the method further includes:

[0104] The initial printed circuit board sample image is filtered.

[0105] The initial printed circuit board sample image after filtering is separated into RGB three channels, and the red image of the printed circuit board sample corresponding to the R channel is extracted and binarized.

[0106] Optionally, the step of locating components and obtaining component solder joint areas based on the initial printed circuit board sample image includes:

[0107] The first and second coordinates of the component are determined based on the red image of the printed circuit board sample after binarization, and the component is located based on the first and second coordinates to obtain the component area on the initial printed circuit board sample image.

[0108] The component solder joint area is determined based on the component area.

[0109] Optionally, the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area respectively include a first solder joint red area, a second solder joint red area, a first solder joint green area, a second solder joint green area, a first solder joint blue area, and a second solder joint blue area;

[0110] The step of establishing a corresponding color library image based on the color features corresponding to the initial red area, the initial green area, and the initial blue area of ​​the solder joint includes:

[0111] The color features of the red area of ​​the first solder joint, the red area of ​​the second solder joint, the green area of ​​the first solder joint, the green area of ​​the second solder joint, the blue area of ​​the first solder joint, and the blue area of ​​the second solder joint are extracted respectively, and a corresponding color library image is established based on the color features.

[0112] Optionally, the defect sample image generation module includes:

[0113] In the component solder joint area, a first solder joint area segmentation point and a second solder joint area segmentation point are determined, and a corresponding elliptical model is established based on the first solder joint area segmentation point and the second solder joint area segmentation point.

[0114] Based on the elliptical model, the corresponding color is selected from the corresponding color library to generate a printed circuit board defect sample image.

[0115] Optionally, the defect sample generation device 930 further includes:

[0116] When the abscissa of the first solder joint area segmentation point is within the first threshold range and the abscissa of the second solder joint area segmentation point is within the second threshold range, the generated printed circuit board defect sample image is a printed circuit board insufficient solder defect sample image.

[0117] When the abscissa of the first solder joint area segmentation point is within the third threshold range and the abscissa of the second solder joint area segmentation point is within the fourth threshold range, the generated printed circuit board defect sample image is a printed circuit board cold solder joint defect sample image.

[0118] The defect sample generation device provided in the embodiments of the present invention can execute the defect sample generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the defect sample generation method.

[0119] Example 4

[0120] Figure 10 A schematic diagram of an automated optical inspection device, 1010, is shown that can be used to implement embodiments of the present invention. The automated optical inspection device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The automated optical inspection device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0121] like Figure 10As shown, the automatic optical inspection instrument 1010 includes at least one processor 1011 and a memory, such as a read-only memory (ROM) 1012 or a random access memory (RAM) 1013, communicatively connected to the at least one processor 1011. The memory stores computer programs executable by the at least one processor. The processor 1011 can perform various appropriate actions and processes based on the computer program stored in the ROM 1012 or loaded from storage unit 1018 into the RAM 1013. The RAM 1013 can also store various programs and data required for the operation of the automatic optical inspection instrument 1010. The processor 1011, ROM 1012, and RAM 1013 are interconnected via a bus 1014. An input / output (I / O) interface 1015 is also connected to the bus 1014.

[0122] Multiple components in the automatic optical inspection instrument 1010 are connected to the I / O interface 1015, including: an input unit 1016, such as a keyboard, mouse, etc.; an output unit 1017, such as various types of displays, speakers, etc.; a storage unit 1018, such as a disk, optical disk, etc.; and a communication unit 1019, such as a network card, modem, wireless transceiver, etc. The communication unit 1019 allows the automatic optical inspection instrument 1010 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0123] Processor 1011 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 1011 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 1011 performs the various methods and processes described above, such as defect sample generation methods.

[0124] In some embodiments, the defect sample generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 1018. In some embodiments, part or all of the computer program may be loaded and / or mounted on the automated optical inspection instrument 1010 via ROM 1012 and / or communication unit 1019. When the computer program is loaded into RAM 1013 and executed by processor 1011, one or more steps of the defect sample generation method described above may be performed. Alternatively, in other embodiments, processor 1011 may be configured to perform the defect sample generation method by any other suitable means (e.g., by means of firmware).

[0125] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0128] To provide user interaction, the systems and techniques described herein can be implemented on an automated optical inspection instrument having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the automated optical inspection instrument. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0129] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0130] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0131] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating defect samples, characterized in that, include: An AOI inspection device is used to inspect the printed circuit board under test to obtain an initial printed circuit board sample image, and the component solder joint area is obtained by locating the components based on the initial printed circuit board sample image. The component solder joint area is divided into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, and a corresponding color library image is established based on the color features of the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area. An elliptical model is established in the solder joint area of ​​the component, and a defect sample image of the printed circuit board is generated based on the color library image and the elliptical model. The step of establishing an elliptical model in the solder joint area of ​​the component and generating a printed circuit board defect sample image based on the color library image and the elliptical model includes: In the component solder joint area, a first solder joint area segmentation point and a second solder joint area segmentation point are determined, and a corresponding elliptical model is established based on the first solder joint area segmentation point and the second solder joint area segmentation point. Based on the elliptical model, the corresponding color is selected from the corresponding color library to generate a printed circuit board defect sample image; The defect sample generation method further includes: When the abscissa of the first solder joint area segmentation point is within the first threshold range and the abscissa of the second solder joint area segmentation point is within the second threshold range, the generated printed circuit board defect sample image is a printed circuit board insufficient solder defect sample image. When the abscissa of the first solder joint area segmentation point is within the third threshold range and the abscissa of the second solder joint area segmentation point is within the fourth threshold range, the generated printed circuit board defect sample image is a printed circuit board cold solder joint defect sample image.

2. The defect sample generation method according to claim 1, characterized in that, The process of obtaining an initial printed circuit board sample image by inspecting the printed circuit board under test using AOI inspection equipment includes: An AOI (Automated Optical Inspection) device captures images of the printed circuit board under test from top to bottom through the light source in the AOI device, obtaining an initial image of the printed circuit board sample. The light source consists of red, green, and blue light sources distributed from top to bottom.

3. The defect sample generation method according to claim 2, characterized in that, After obtaining an initial sample image of the printed circuit board (PCB) through AOI (Automated Optical Inspection) equipment, the process also includes: The initial printed circuit board sample image is filtered. The initial printed circuit board sample image after filtering is separated into RGB three channels, and the red image of the printed circuit board sample corresponding to the R channel is extracted and binarized.

4. The defect sample generation method according to claim 3, characterized in that, The step of locating components and obtaining component solder joint areas based on the initial printed circuit board sample image includes: The first and second coordinates of the component are determined based on the red image of the printed circuit board sample after binarization, and the component is located based on the first and second coordinates to obtain the component area on the initial printed circuit board sample image. The component solder joint area is determined based on the component area.

5. The defect sample generation method according to claim 1, characterized in that, The initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area respectively include a first solder joint red area, a second solder joint red area, a first solder joint green area, a second solder joint green area, a first solder joint blue area, and a second solder joint blue area; The step of establishing a corresponding color library image based on the color features corresponding to the initial red area, the initial green area, and the initial blue area of ​​the solder joint includes: The color features of the red area of ​​the first solder joint, the red area of ​​the second solder joint, the green area of ​​the first solder joint, the green area of ​​the second solder joint, the blue area of ​​the first solder joint, and the blue area of ​​the second solder joint are extracted respectively, and a corresponding color library image is established based on the color features.

6. A defect sample generation apparatus, controlled by the defect sample generation method as described in any one of claims 1-5, characterized in that, include: The component solder joint area determination module is used to perform the following: the AOI inspection equipment is used to inspect the printed circuit board under test to obtain an initial printed circuit board sample image, and the components are located and the component solder joint area is obtained based on the initial printed circuit board sample image. The color library creation module is used to divide the component solder joint area into an initial solder joint red area, an initial solder joint green area, and an initial solder joint blue area, and to create a corresponding color library image based on the color features of the initial solder joint red area, the initial solder joint green area, and the initial solder joint blue area. The defect sample image generation module is used to create an elliptical model in the solder joint area of ​​the component and generate a printed circuit board defect sample image based on the color library image and the elliptical model.

7. An automatic optical inspection instrument, characterized in that, The automatic optical inspection instrument includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the defect sample generation method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the defect sample generation method according to any one of claims 1-5.

Images