An automatic optical inspection method and apparatus for PCB panelization, and a storage medium.

Abstract

This invention relates to the field of automated optical inspection technology, and discloses an automated optical inspection method, apparatus, and storage medium for PCB panelization. The invention obtains a complete image of the PCB panelization by stitching together partial images, which saves costs. Secondly, the invention identifies components in the image to be tested. For each component: based on the component, its soldering information and detection rules are read from a preset relational database; the image of the component in the image to be tested is matched with its soldering information to obtain the detection area of ​​the component in the image to be tested; the detection area is then inspected based on the preset detection rules to obtain the detection result. Compared to inspecting the entire PCB panelization, the method of reading each component and performing targeted inspection in this invention is more efficient and takes less time.

Description

Technical Field

[0001] This invention relates to the field of automatic optical inspection technology, and in particular to an automatic optical inspection method and apparatus for PCB panelization, as well as a storage medium. Background Technology

[0002] For printed circuit board (PCB) panel defect inspection, the missed detection rate of manual inspection is as high as 20% (see Journal of Electronic Packaging, 2003, Vol. 125), and it is difficult to detect tiny defects (size <0.11mm, see Machine Vision and Applications, 2011, Vol. 22). Automated Optical Inspection (AOI) has emerged to address this issue. AOI systems aim to replace traditional manual inspection, solving its problems of low efficiency, high subjectivity, and susceptibility to fatigue, and are widely used in PCB panel defect inspection.

[0003] Existing automatic optical inspection methods for PCB panels involve capturing optical images of the entire PCB panel and then using pre-defined algorithms for inspection. However, this approach has several drawbacks: First, when dealing with large PCB panels, obtaining more accurate images requires higher-performance cameras, increasing costs. Second, the detection algorithms for the entire PCB panel are inefficient, resulting in longer inspection times. Summary of the Invention

[0004] This invention provides an automatic optical inspection method and apparatus for PCB panels, as well as a storage medium, which can obtain more reliable PCB panel image data without affecting costs by image stitching; and then achieve more efficient automatic optical inspection of PCB panels.

[0005] The first aspect of this invention discloses an automatic optical inspection method for PCB panels, the method comprising:

[0006] The camera is controlled to capture multiple local positions corresponding to the PCB panel under test, resulting in multiple local images; the images under test are then stitched together based on these multiple local images.

[0007] Identify the components on the image under test, for each component:

[0008] Based on the component, the welding information and detection rules of the component are read from a preset relational database; the image of the component on the image to be tested is matched with the welding information of the component to obtain the detection area of ​​the component on the image to be tested; the detection area is detected based on the preset detection rules to obtain the detection result;

[0009] The relational database is obtained by analyzing the design information of the PCB panel under test, and the relational database stores at least component soldering information and testing rules.

[0010] As an optional implementation, in the first aspect of the present invention, the relational database further includes a detection area mapping table, which is obtained by analyzing the PCB panel under test and is used to represent the correspondence between pads and pins;

[0011] And, the step of detecting the detection area based on preset detection rules to obtain detection results includes:

[0012] The pads in the detection area are detected based on preset detection rules to obtain the detection results of the pins corresponding to each pad.

[0013] As an optional implementation, in a first aspect of the present invention, the controlled camera captures multiple local positions corresponding to the PCB panel under test to obtain multiple local images, including:

[0014] A pure black background image is generated based on the camera resolution and the size of the PCB panel to be tested.

[0015] The camera is controlled to move relative to the PCB panel under test along a preset path and captures multiple local images during the movement. Any two adjacent local images have at least a partial overlap.

[0016] And, the step of stitching together multiple local images to obtain the image to be tested includes:

[0017] Obtain the camera coordinates at each shooting position, obtain the camera calibration parameters, and map each local image onto the pure black background image based on the camera coordinates and the calibration parameters;

[0018] Based on a preset image stitching strategy, all local images in the pure black background image are stitched together to obtain the image to be tested.

[0019] As an optional implementation, in a first aspect of the present invention, the control camera moves relative to the PCB panel under test along a preset path and captures multiple partial images during the movement, including:

[0020] The camera is controlled to move relative to the PCB panel under test along a preset path so that the camera passes directly above multiple components. When the camera moves directly above the preset multiple components, it takes multiple partial images. Any two adjacent partial images have at least a partial overlap area.

[0021] And, the method of stitching together all local images in the pure black background image based on a preset image stitching strategy to obtain the image to be tested includes:

[0022] Identify the overlapping region between any two adjacent local images. For each overlapping region:

[0023] An edge detection algorithm is used to detect all edge lines in the overlapping area. Several first splicing lines with the fewest intersections with all the edge lines are obtained from the overlapping area. From all the first splicing lines, the target splicing line with the shortest overlap distance with all the edge lines is selected.

[0024] By stitching together all the corresponding overlapping areas according to all the target stitching lines, the image to be tested is obtained.

[0025] As an optional implementation, in a first aspect of the present invention, the image to be tested comprises N stitched images; and the step of stitching together the multiple local images to obtain the image to be tested includes:

[0026] Obtain the camera coordinates at each shooting position and obtain the calibration parameters of the camera;

[0027] For each local image, based on the corresponding camera coordinates and the camera calibration parameters, N local image regions in the local image that are located in different spatial ranges relative to the corresponding camera coordinates are determined, wherein the positions of all pixels in each local image region are within a preset spatial range relative to the corresponding camera coordinates.

[0028] Based on N local image regions located in different spatial ranges on all local images, N stitched images are obtained according to a preset image stitching strategy.

[0029] As an optional implementation, in a first aspect of the present invention, the controlled camera captures multiple local positions corresponding to the PCB panel under test to obtain multiple local images, including:

[0030] Control the camera to capture M local positions corresponding to the PCB panel under test, and obtain M local images of the same size, where M is a positive integer greater than or equal to N;

[0031] Furthermore, for each local image, based on the corresponding camera coordinates and the camera's calibration parameters, determining N local image regions located in different spatial ranges relative to the corresponding camera coordinates includes:

[0032] For each local image, based on the corresponding camera coordinates and the camera calibration parameters, the local image regions located at the first angle, the second angle, and the third angle relative to the corresponding camera coordinates are determined, where the first angle, the second angle, and the third angle are preset spatial angles.

[0033] And, the step of obtaining N stitched images based on N local image regions located in different spatial ranges on all local images, according to a preset image stitching strategy, includes:

[0034] Based on the local image regions located at the first angle in all local images, a first stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the second angle in all local images, a second stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the third angle in all local images, a third stitched image is obtained according to a preset image stitching strategy.

[0035] As an optional implementation, in a first aspect of the present invention, the step of retrieving the component's soldering information and inspection rules from a preset relational database based on the component includes:

[0036] For each component in each stitched image, the welding information and initial detection rules of the component are read from a preset relational database based on the component. Based on the spatial range relationship between the component, the stitched image and the corresponding camera coordinates, the detection parameters corresponding to the initial detection rules are set to obtain the detection rules. The detection parameters include at least a threshold parameter for measuring whether a defect exists.

[0037] And, the step of detecting the detection area based on preset detection rules to obtain detection results includes:

[0038] For each component, the component image on each stitched image is matched with the component welding information to obtain N detection areas of the component on N stitched images; for each detection area, the corresponding target detection rule is determined according to the spatial range relationship between the detection area and the corresponding camera coordinates, and the detection area is detected based on the target detection rule to obtain the area detection result;

[0039] For each of the components, the detection result corresponding to that component is obtained based on the detection results of all the regions corresponding to that component.

[0040] A second aspect of this invention discloses an automatic optical inspection device for PCB panels, the device comprising:

[0041] The control module is used to control the camera to capture multiple local positions corresponding to the PCB panel under test, thereby obtaining multiple local images; and to stitch together the multiple local images to obtain the image under test.

[0042] The detection module is used to identify components on the image to be tested, and for each component, performs the following operations:

[0043] Based on the component, the welding information and detection rules of the component are read from a preset relational database; the image of the component on the image to be tested is matched with the welding information of the component to obtain the detection area of ​​the component on the image to be tested; the detection area is detected based on the preset detection rules to obtain the detection result;

[0044] The relational database is obtained by analyzing the design information of the PCB panel under test, and the relational database stores at least component soldering information and testing rules.

[0045] As an optional implementation, in a second aspect of the present invention, the relational database further includes a detection area mapping table, which is obtained by analyzing the PCB panel under test and is used to represent the correspondence between pads and pins.

[0046] Furthermore, the specific operation method by which the detection module detects the detection area based on preset detection rules and obtains the detection result includes:

[0047] The pads in the detection area are detected based on preset detection rules to obtain the detection results of the pins corresponding to each pad.

[0048] As an optional implementation, in a second aspect of the present invention, the specific operation method of the control module controlling the camera to capture multiple local positions corresponding to the PCB panel under test to obtain multiple local images includes:

[0049] A pure black background image is generated based on the camera resolution and the size of the PCB panel to be tested.

[0050] The camera is controlled to move relative to the PCB panel under test along a preset path and captures multiple local images during the movement. Any two adjacent local images have at least a partial overlap.

[0051] Furthermore, the specific operation method of the detection module to stitch together multiple local images to obtain the image to be tested includes:

[0052] Obtain the camera coordinates at each shooting position, obtain the camera calibration parameters, and map each local image onto the pure black background image based on the camera coordinates and the calibration parameters;

[0053] Based on a preset image stitching strategy, all local images in the pure black background image are stitched together to obtain the image to be tested.

[0054] As an optional implementation, in a second aspect of the present invention, the specific operation method of the control module controlling the camera to move relative to the PCB panel under test along a preset path and capturing multiple local images during the movement includes:

[0055] The camera is controlled to move relative to the PCB panel under test along a preset path so that the camera passes directly above multiple components. When the camera moves directly above the preset multiple components, it takes multiple partial images. Any two adjacent partial images have at least a partial overlap area.

[0056] Furthermore, the specific operation method by which the detection module stitches together all local images in the pure black background image based on a preset image stitching strategy to obtain the image to be tested includes:

[0057] Identify the overlapping region between any two adjacent local images. For each overlapping region:

[0058] An edge detection algorithm is used to detect all edge lines in the overlapping area. Several first splicing lines with the fewest intersections with all the edge lines are obtained from the overlapping area. From all the first splicing lines, the target splicing line with the shortest overlap distance with all the edge lines is selected.

[0059] By stitching together all the corresponding overlapping areas according to all the target stitching lines, the image to be tested is obtained.

[0060] As an optional implementation, in a second aspect of the present invention, the image to be tested includes N stitched images; and the specific operation method by which the detection module stitches together the multiple local images to obtain the image to be tested includes:

[0061] Obtain the camera coordinates at each shooting position and obtain the calibration parameters of the camera;

[0062] For each local image, based on the corresponding camera coordinates and the camera calibration parameters, N local image regions in the local image that are located in different spatial ranges relative to the corresponding camera coordinates are determined, wherein the positions of all pixels in each local image region are within a preset spatial range relative to the corresponding camera coordinates.

[0063] Based on N local image regions located in different spatial ranges on all local images, N stitched images are obtained according to a preset image stitching strategy.

[0064] As an optional implementation, in a second aspect of the present invention, the specific operation method of the control module controlling the camera to capture multiple local positions corresponding to the PCB panel under test to obtain multiple local images includes:

[0065] Control the camera to capture M local positions corresponding to the PCB panel under test, and obtain M local images of the same size, where M is a positive integer greater than or equal to N;

[0066] Furthermore, for each local image, the detection module determines the specific operation method of N local image regions located in different spatial ranges relative to the corresponding camera coordinates in that local image, based on the corresponding camera coordinates and the camera's calibration parameters, including:

[0067] For each local image, based on the corresponding camera coordinates and the camera calibration parameters, the local image regions located at the first angle, the second angle, and the third angle relative to the corresponding camera coordinates are determined, where the first angle, the second angle, and the third angle are preset spatial angles.

[0068] Furthermore, the detection module, based on N local image regions located in different spatial ranges across all local images, and according to a preset image stitching strategy, obtains N stitched images through specific operational methods, including:

[0069] Based on the local image regions located at the first angle in all local images, a first stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the second angle in all local images, a second stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the third angle in all local images, a third stitched image is obtained according to a preset image stitching strategy.

[0070] As an optional implementation, in a second aspect of the invention, the specific operation method by which the detection module reads the component's welding information and detection rules from a preset relational database based on the component includes:

[0071] For each component in each stitched image, the welding information and initial detection rules of the component are read from a preset relational database based on the component. Based on the spatial range relationship between the component, the stitched image and the corresponding camera coordinates, the detection parameters corresponding to the initial detection rules are set to obtain the detection rules. The detection parameters include at least a threshold parameter for measuring whether a defect exists.

[0072] Furthermore, the specific operation method by which the detection module detects the detection area based on preset detection rules and obtains the detection result includes:

[0073] For each component, the component image on each stitched image is matched with the component welding information to obtain N detection areas of the component on N stitched images; for each detection area, the corresponding target detection rule is determined according to the spatial range relationship between the detection area and the corresponding camera coordinates, and the detection area is detected based on the target detection rule to obtain the area detection result;

[0074] For each of the components, the detection result corresponding to that component is obtained based on the detection results of all the regions corresponding to that component.

[0075] A third aspect of the present invention discloses an automatic optical inspection system for PCB panels, the system comprising: a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute some or all of the steps in any of the automatic optical inspection methods for PCB panels according to the first aspect of the present invention.

[0076] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the automatic optical inspection method for PCB panelization described in any of the first aspects of the present invention.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] This invention uses a method of partial imaging followed by stitching to obtain an overall image of the PCB panel, which saves costs. Secondly, this invention identifies the components in the image under test. For each component: based on the component, its soldering information and detection rules are retrieved from a preset relational database; the image of the component in the image under test is matched with its soldering information to obtain the detection area of ​​the component in the image under test; the detection area is then detected based on preset detection rules to obtain the detection result. Compared to detecting the entire PCB panel, the method of retrieving each component and performing targeted detection in this invention is more efficient and takes less time. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1This is a flowchart illustrating an automatic optical inspection method for PCB panels disclosed in an embodiment of the present invention;

[0081] Figure 2 This is a schematic diagram of an S-shaped camera imaging path in an embodiment of the present invention;

[0082] Figure 3 This is an exemplary schematic diagram of partial image stitching disclosed in this invention;

[0083] Figure 4 This is an exemplary diagram of an automatically configured bounding box disclosed in this invention;

[0084] Figure 5 This is a schematic diagram of an exemplary threshold setting process disclosed in this invention;

[0085] Figure 6 This is a schematic diagram of a splicing gap disclosed in this invention;

[0086] Figure 7 This is another schematic diagram of the splicing gap disclosed in this invention;

[0087] Figure 8 This is a schematic diagram of the structure of an automatic optical inspection device for PCB panelization disclosed in an embodiment of the present invention;

[0088] Figure 9 This is a schematic diagram of the structure of an automatic optical inspection system for PCB panelization disclosed in an embodiment of the present invention. Detailed Implementation

[0089] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0091] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0092] This invention discloses an automatic optical inspection method and apparatus for PCB panels, as well as a storage medium. It enables the acquisition of more reliable PCB panel image data through image stitching without affecting cost, thereby achieving more efficient automatic optical inspection of PCB panels. These are described in detail below.

[0093] Example 1

[0094] Please see Figure 1 , Figure 1 This is a flowchart illustrating an automated optical inspection method for PCB panelization disclosed in an embodiment of the present invention. Figure 1 The described automatic optical inspection method for PCB panels can be applied to an automatic optical inspection device for PCB panels, which can be integrated into a cloud server or a local server. This invention is not limited in its embodiments. Figure 1 As shown, the automatic optical inspection method for PCB panels may include the following operations:

[0095] Step 101: Control the camera to capture multiple local positions of the PCB panel under test to obtain multiple local images; stitch the multiple local images together to obtain the image under test.

[0096] In this embodiment of the invention, a method of partial shooting followed by stitching is used to obtain an overall image of the PCB panel, which saves costs. Optionally, in this embodiment, controlling the camera to capture multiple partial images of the PCB panel under test can include:

[0097] Generate a pure black background image (RGB:0,0,0) based on the camera resolution (e.g., 2048×2048 pixels) and the size of the PCB panel to be tested.

[0098] The camera is controlled to move relative to the PCB panel under test along a preset path and capture multiple partial images during the movement. Any two adjacent partial images must have at least a partial overlap. Optionally, such as... Figure 2 As shown, an S-shaped path is used to control the movement of an industrial camera (such as a 50-megapixel CCD) to cover the surface of the panel. The overlap rate of the shooting area is 10%-20%, and the stepping distance is dynamically adjusted according to the size of the panel. Optionally, the stepping distance is 5mm-10mm.

[0099] Furthermore, the image to be tested is obtained by stitching together multiple local images, including:

[0100] The camera coordinates at each shooting position are obtained, and the camera calibration parameters are obtained. Based on the camera coordinates and calibration parameters, each local image is mapped onto a pure black background. The camera calibration parameters are obtained in advance. The calibration method includes: using a standard calibration board (such as a checkerboard with a grid size of 1mm×1mm) to correct camera distortion, generating a transformation matrix (3×3 affine matrix), and correcting radial and tangential distortion with an error of <0.01mm.

[0101] Based on a preset image stitching strategy, all local images on a pure black background are stitched together to obtain the image to be tested. The image stitching strategy can be selected according to the actual situation. Optionally, based on calibration parameters and camera coordinates, the local images are mapped to the background image, and feature point registration algorithms (such as SIFT, scale-invariant feature transform) and fusion processing (such as weighted average fusion) are applied to eliminate stitching gaps, thereby achieving sub-pixel-level stitching accuracy (error <0.01mm). An example stitching effect is shown below. Figure 3 As shown.

[0102] In this embodiment of the invention, a pure black background image is used for stitching assistance, which can greatly improve stitching efficiency and accuracy when there are many images to be stitched.

[0103] Step 102: Identify the components on the image to be tested. For each component: read the component's welding information and detection rules from the preset relational database; match the component image on the image to be tested with the component's welding information to obtain the detection area of ​​the component on the image to be tested; perform detection on the detection area based on the preset detection rules to obtain the detection result.

[0104] The relational database is obtained by analyzing the design information of the PCB panel to be tested. It can be in JSON or SQL format. The relational database at least stores component soldering information and testing rules.

[0105] In an optional embodiment, the relational database also includes a detection area mapping table, which is obtained by analyzing the PCB panel under test and is used to represent the correspondence between pads and pins.

[0106] Furthermore, based on preset detection rules, the detection area is detected to obtain detection results, including:

[0107] The pads in the detection area are detected based on preset detection rules, and the detection results of the pins corresponding to each pad are obtained.

[0108] Based on the above optional embodiments, step 102 may include:

[0109] Read CAD and test pins:

[0110] CAD data: Parse pad information from CAD files such as Gerber and DXF, including position (x, y coordinates, accuracy 0.01mm), size (length and width, range 0.1mm-10mm), and shape (circle, rectangle, etc.).

[0111] Test data (such as angle and pad data): Test component information, such as body, pads, pin type, and number of pins (2-200).

[0112] The processing flow is as follows:

[0113] Parse CAD files to generate structured data tables (such as JSON or SQL format).

[0114] Establish the correspondence between pads and pins, and generate a detection area mapping table.

[0115] Automatically generate detection rules, including:

[0116] Automatically configure the bounding box (error <0.05mm) based on the shape and size of the pads. An example bounding box is shown below. Figure 4 As shown. Specific defect detection algorithms (two-pin and multi-pin) are assigned based on component type and part number. Defect thresholds are set (e.g., minimum solder threshold and maximum bridging threshold).

[0117] Algorithm library:

[0118] It includes template matching, feature extraction, and machine learning modules, and supports 6 common defects (offset, wrong component, cold solder joint, lifting lead, bridging solder, missing component).

[0119] Database storage and dynamic updates:

[0120] Storage: Detection rules, pad data, and component information are stored in a relational database (such as MySQL) to support fast queries.

[0121] Dynamic updates: The rules are automatically updated when the PCB design changes.

[0122] Data structure: Includes fields such as pin, pad, coordinate, defect type, and detection parameters.

[0123] Interactive verification:

[0124] User interface: Provides a graphical interface that displays pad locations, detection areas, and rule parameters.

[0125] Features: Supports manual threshold adjustment (e.g., sensitivity 0.1-1.0), adding custom rules; an example threshold setting process is as follows. Figure 5 As shown. For example, the red threshold is set to 0.8, the green threshold is set to 0.37, and the blue threshold is set to 0.31.

[0126] Verification process: Preview detection results in real time, mark potential false alarms / missed detections, and save the adjusted rules.

[0127] Read the detection rules from the database and extract pad and component information, including:

[0128] Image registration algorithms (such as SIFT-based feature points) are used to calculate image offset and accurately align the detection area (error < 0.05 mm).

[0129] Customized defect detection:

[0130] Based on the characteristics of the pads and components, a customized algorithm is applied:

[0131] – Cold solder joint: HSV threshold analysis + target detection.

[0132] – Lian Xi: HSV threshold analysis.

[0133] –Missing component: Template matching + Target detection.

[0134] – Wrong component: SSIM+ target detection.

[0135] Performance: Defect identification rate >99%, false alarm rate <5%, and false negative rate <2%.

[0136] Interactive error correction panel:

[0137] Interface functions: Displays detection results, marks defect locations, and provides parameter adjustment sliders.

[0138] Error correction process: The user confirms the false alarm / missed detection, adjusts the detection threshold, and saves it to the database.

[0139] As can be seen, in this embodiment of the invention, the overall image of the PCB panel is obtained by stitching together partial images, which saves costs. Secondly, this embodiment identifies the components on the image to be tested. For each component: based on the component, the component's soldering information and detection rules are read from a preset relational database; the image of the component on the image to be tested is matched with the component's soldering information to obtain the detection area of ​​the component on the image to be tested; the detection area is then detected based on the preset detection rules to obtain the detection result. Compared to detecting the entire PCB panel, the method of reading each component and performing targeted detection in this embodiment is more efficient and takes less time.

[0140] In this embodiment of the invention, the method for capturing local images using a camera can be that the camera moves relative to the PCB panel, the PCB panel moves relative to the camera, or both move relative to each other. For example, the camera moves along the X-direction, and the PCB panel moves along the Y-direction. During the camera's movement relative to the PCB panel, how to obtain the most favorable local image for detection, and how to stitch the images with high precision, become problems that need to be solved.

[0141] In an optional embodiment, controlling a camera to move relative to the PCB panel under test along a preset path and capturing multiple partial images during the movement includes:

[0142] The camera is controlled to move relative to the PCB panel under test along a preset path so that the camera passes directly above multiple components. When the camera moves directly above the preset multiple components, it takes multiple partial images. Any two adjacent partial images have at least a partial overlap area.

[0143] Furthermore, based on a preset image stitching strategy, all local images in the pure black background image are stitched together to obtain the image to be tested, including:

[0144] Identify the overlapping region between any two adjacent local images. For each overlapping region:

[0145] The edge detection algorithm is used to detect all edge lines in the overlapping area. Several first splicing lines with the fewest intersections with all edge lines are obtained from the overlapping area. The target splicing line with the shortest overlap distance with all edge lines is selected from all first splicing lines.

[0146] By stitching together all corresponding overlapping areas according to all target stitching lines, the image to be tested is obtained.

[0147] Image stitching is a crucial factor affecting detection accuracy. Due to varying shooting conditions for different local images, the detection performance of an image obtained by stitching together the left and right halves of element A is unlikely to match that of a standalone image of element A. This is because completely eliminating the seams during stitching requires a significant amount of computation, leading to reduced efficiency. Therefore, this invention avoids stitching elements as much as possible during the stitching process; that is, the stitching seams in this embodiment of the invention should not pass through the element images, thereby minimizing the impact of stitching seams on detection accuracy.

[0148] In this optional embodiment, an edge detection algorithm is used to detect all edge lines within the overlapping area. Edge lines refer to lines where the grayscale value changes abruptly in the image, such as the outline of each component or cracks in a PCB panel. Edge lines are crucial for subsequent detection; therefore, edge lines should be avoided during splicing. This optional embodiment uses an edge detection algorithm to detect all edge lines within the overlapping area and obtains a first splicing line with the fewest intersections with all edge lines from the overlapping area. The splicing gap obtained by a commonly used straight-line gap splicing method is shown below. Figure 3 or Figure 6 , Figure 6 The seams in the splicing process penetrate many components, resulting in some component images being composed of different parts, which reduces detection accuracy. The first splice after screening is as follows: Figure 7 As shown in the splicing gap, the splicing line avoids the components as much as possible, so that the component images are not spliced ​​together, thereby improving the detection accuracy.

[0149] In this optional embodiment, the first splicing line is simply the one with the fewest intersections with the edge lines. There can be many such first splicing lines, and the first splicing line often coincides with some edge lines. For example, the splicing line may be along the edge of a component, or the edge line may be along a conductor. The aforementioned first splicing line will cause the conductor or edge position to become a splicing gap during splicing, which will reduce the accuracy of the detection of the conductor itself and the edge of the component. Therefore, this optional embodiment further selects the target splicing line with the shortest overlap distance with all edge lines from the first splicing lines. The target splicing line should not pass through the surface of the component as much as possible, nor should it coincide with the edge line as much as possible, so as to minimize the impact of the splicing process on the accuracy of the subsequent detection process.

[0150] In this embodiment of the invention, the method of capturing partial images and then stitching them together means that different partial images may be captured by the camera from different angles. The final stitched image is not actually captured entirely from directly above. On the other hand, during defect detection, some defects are difficult to detect from a single angle and require multiple angles to determine their presence. Therefore, in another optional embodiment, image stitching can be used to stitch images from different angles. That is, a portion of each partial image at a fixed angle is stitched together to obtain multiple stitched images from different shooting angles. For example, each partial image can be divided into three regions: left, center, and right. The left region is captured by the camera at the upper right of the imaging object space, the center region is captured by the camera directly above the imaging object, and the right region is captured by the camera at the upper left of the imaging object space. During stitching, the left regions of all partial images are stitched together to obtain one stitched image; the right regions of all partial images are stitched together to obtain one stitched image; and the center regions of all partial images are stitched together to obtain one stitched image. In this way, the three stitched images are obtained by the camera taking pictures of the component under test at different spatial positions, which can provide richer and more reliable image data for subsequent detection steps.

[0151] Based on the above inventive concept, specifically, in this optional embodiment, the image to be tested may include N stitched images; optionally, for each of the N stitched images, the operation in step 102 is performed on each image, thereby enabling detection of each element after being captured from different angles.

[0152] To stitch images captured from the same angle into a single stitched image, in this optional embodiment, camera calibration parameters are used to calculate the positional relationship between different pixels in the image and camera coordinates, thereby determining different positional partitions in each local image. Specifically, stitching together multiple local images to obtain the image to be tested can include:

[0153] Obtain the camera coordinates at each shooting position and obtain the camera calibration parameters;

[0154] For each local image, based on the corresponding camera coordinates and camera calibration parameters, N local image regions located in different spatial ranges relative to the corresponding camera coordinates are determined. In each local image region, the position of all pixels relative to the corresponding camera coordinates is within a preset spatial range.

[0155] In this optional embodiment, further optionally, for each local image, based on the corresponding camera coordinates and camera calibration parameters, N local image regions located in different spatial ranges relative to the corresponding camera coordinates are determined, which may include:

[0156] For each local image, based on the corresponding camera coordinates and camera calibration parameters, the local image regions located at a first angle, a second angle, and a third angle relative to the corresponding camera coordinates are determined. Here, the first, second, and third angles are preset spatial angles. The spatial angle is defined as the angle between the line connecting a point within the nth local image region to the camera coordinates and the line connecting the camera coordinates perpendicular to the image plane in three-dimensional space.

[0157] For example, for a local image A, its corresponding camera coordinates (x, y) are used to determine N local image regions in local image A that are located in different spatial ranges relative to the corresponding camera coordinates (x, y). The line connecting the spatial coordinates of a pixel within each local image region to the camera coordinates (x, y) lies within a preset spatial angle. Taking N=3 as an example, the local image is divided into three local image regions: left, center, and right, i.e., the local image region at the first angle, the local image region at the second angle, and the local image region at the third angle. The line connecting each point in the left local image region to the camera coordinates (x, y) lies within a preset left spatial angle, ensuring that each point in the left local image region is captured by the camera from a certain angle on the right.

[0158] Based on N local image regions located in different spatial ranges on all local images, N stitched images are obtained according to a preset image stitching strategy.

[0159] In this optional embodiment, knowing the positional relationship of each pixel relative to the camera coordinates is sufficient to quickly divide the local image into different local image regions, thereby enabling rapid filtering of image regions under different shooting angles.

[0160] In another optional embodiment, the camera is controlled to capture multiple local locations corresponding to the PCB panel under test, resulting in multiple local images, including:

[0161] The camera is controlled to capture M local positions corresponding to the PCB panel under test, resulting in M ​​local images of the same size, where M is a positive integer greater than or equal to N. In this embodiment of the invention, since only a part of each local image is extracted for stitching during image stitching, there must be parts in the leftmost or rightmost image that are not involved in the stitching. Therefore, the number of local images captured must be greater than the number of stitched images. For example, if there are 3 stitched images, then at least 4 parallel local images need to be captured for stitching.

[0162] Furthermore, based on N local image regions located in different spatial ranges across all local images, and using a preset image stitching strategy, N stitched images are obtained, including:

[0163] Based on the local image regions located at the first angle in all local images, a first stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the second angle in all local images, a second stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the third angle in all local images, a third stitched image is obtained according to a preset image stitching strategy.

[0164] In this embodiment of the invention, images obtained from different shooting angles correspond to different detection rules. For example, for a certain component, the image content of an image taken from directly above, an image taken from a certain angle on the left, and an image taken from a certain angle on the right are different, and the detection schemes used to detect various defects and faults are slightly different. In this embodiment of the invention, this difference is reflected by detection parameters, wherein the detection parameters include at least a threshold parameter for measuring whether a defect exists. Therefore, in an optional embodiment, the component's welding information and detection rules are read from a preset relational database based on the component, including:

[0165] For each component in each stitched image, the welding information and initial detection rules of the component are read from the preset relational database based on the component. Based on the spatial range relationship between the component, the stitched image and the corresponding camera coordinates, the detection parameters corresponding to the initial detection rules are set to obtain the detection rules.

[0166] Furthermore, based on preset detection rules, the detection area is detected to obtain detection results, including:

[0167] For each component, the component image on each stitched image is matched with the component welding information to obtain N detection areas of the component on N stitched images; for each detection area, the corresponding target detection rule is determined according to the spatial range relationship between the detection area and the corresponding camera coordinates, and the detection area is detected based on the target detection rule to obtain the area detection result;

[0168] For each component, the detection result corresponding to that component is obtained based on the detection results of all areas corresponding to that component.

[0169] As can be seen, this optional embodiment can implement different detection strategies for stitched images from different shooting angles. It only needs to read the component detection rules from the preset relational database, and then set the detection parameters corresponding to the initial detection rules based on the spatial range relationship between the component, the stitched image and the corresponding camera coordinates to obtain the detection rules, thereby achieving targeted detection and further improving detection accuracy.

[0170] Example 2

[0171] Please see Figure 8 Embodiment 2 of the present invention discloses an automatic optical inspection device for PCB panelization, which may include:

[0172] The control module 201 is used to control the camera to capture multiple local positions corresponding to the PCB panel under test, and obtain multiple local images; and to stitch together the multiple local images to obtain the image under test.

[0173] The detection module 202 is used to identify components on the image to be tested. For each component, the following operations are performed:

[0174] Based on the component, the welding information and detection rules of the component are read from the preset relational database; the image of the component on the image to be tested is matched with the welding information of the component to obtain the detection area of ​​the component on the image to be tested; the detection area is detected based on the preset detection rules to obtain the detection result;

[0175] Among them, the relational database is obtained by analyzing the design information of the PCB panel to be tested. The relational database at least stores component soldering information and testing rules.

[0176] In an optional embodiment, the relational database may further include a detection area mapping table, which is obtained by analyzing the PCB panel under test and is used to represent the correspondence between pads and pins.

[0177] Furthermore, the specific operation method by which the detection module 202 detects the detection area based on preset detection rules and obtains the detection results may include:

[0178] The pads in the detection area are detected based on preset detection rules, and the detection results of the pins corresponding to each pad are obtained.

[0179] In another optional embodiment, the specific operation method of the control module 201 controlling the camera to capture multiple local positions corresponding to the PCB panel under test to obtain multiple local images may include:

[0180] A pure black background image is generated based on the camera resolution and the size of the PCB panel to be tested.

[0181] The camera is controlled to move relative to the PCB panel under test along a preset path and captures multiple local images during the movement. Any two adjacent local images have at least a partial overlap.

[0182] Furthermore, the specific operation method by which the detection module 202 stitches together multiple local images to obtain the image to be tested may include:

[0183] Obtain the camera coordinates at each shooting position, obtain the camera calibration parameters, and map each local image onto a pure black background image based on the camera coordinates and calibration parameters;

[0184] Based on a preset image stitching strategy, all local images in a pure black background image are stitched together to obtain the image to be tested.

[0185] In another optional embodiment, the specific operation method of the control module 201 controlling the camera to move relative to the PCB panel under test along a preset path and taking multiple partial images during the movement may include:

[0186] The camera is controlled to move relative to the PCB panel under test along a preset path so that the camera passes directly above multiple components. When the camera moves directly above the preset multiple components, it takes multiple partial images. Any two adjacent partial images have at least a partial overlap area.

[0187] Furthermore, the specific operation method by which the detection module 202 stitches together all local images in a pure black background image based on a preset image stitching strategy to obtain the image to be tested may include:

[0188] Identify the overlapping region between any two adjacent local images. For each overlapping region:

[0189] The edge detection algorithm is used to detect all edge lines in the overlapping area. Several first splicing lines with the fewest intersections with all edge lines are obtained from the overlapping area. The target splicing line with the shortest overlap distance with all edge lines is selected from all first splicing lines.

[0190] By stitching together all corresponding overlapping areas according to all target stitching lines, the image to be tested is obtained.

[0191] In another optional embodiment, the image to be tested may include N stitched images; and the specific operation method by which the detection module 202 stitches together multiple local images to obtain the image to be tested may include:

[0192] Obtain the camera coordinates at each shooting position and obtain the camera calibration parameters;

[0193] For each local image, based on the corresponding camera coordinates and camera calibration parameters, N local image regions in the local image that are located in different spatial ranges relative to the corresponding camera coordinates are determined. Among them, the positions of all pixels in each local image region are within a preset spatial range relative to the corresponding camera coordinates.

[0194] Based on N local image regions located in different spatial ranges on all local images, N stitched images are obtained according to a preset image stitching strategy.

[0195] In another optional embodiment, the specific operation method of the control module 201 controlling the camera to capture multiple local positions corresponding to the PCB panel under test to obtain multiple local images may include:

[0196] Control the camera to capture M local positions corresponding to the PCB panel under test, and obtain M local images of the same size, where M is a positive integer greater than or equal to N;

[0197] Furthermore, for each local image, the detection module 202 determines the specific operation method of N local image regions located in different spatial ranges relative to the corresponding camera coordinates in that local image based on the corresponding camera coordinates and camera calibration parameters. This may include:

[0198] For each local image, based on the corresponding camera coordinates and camera calibration parameters, the local image regions located at the first angle, the second angle, and the third angle relative to the corresponding camera coordinates are determined, where the first angle, the second angle, and the third angle are preset spatial angles.

[0199] Furthermore, the specific operation method by which the detection module 202 obtains N stitched images based on N local image regions located in different spatial ranges on all local images and a preset image stitching strategy may include:

[0200] Based on the local image regions located at the first angle in all local images, a first stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the second angle in all local images, a second stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the third angle in all local images, a third stitched image is obtained according to a preset image stitching strategy.

[0201] In another optional embodiment, the specific operation method by which the detection module 202 reads the component's welding information and detection rules from a preset relational database based on the component may include:

[0202] For each component in each stitched image, the welding information and initial detection rules of the component are read from a preset relational database based on the component. Based on the spatial range relationship between the component, the stitched image and the corresponding camera coordinates, the detection parameters corresponding to the initial detection rules are set to obtain the detection rules. The detection parameters may include at least a threshold parameter for measuring whether a defect exists.

[0203] Furthermore, the specific operation method by which the detection module 202 detects the detection area based on preset detection rules and obtains the detection results may include:

[0204] For each component, the component image on each stitched image is matched with the component welding information to obtain N detection areas of the component on N stitched images; for each detection area, the corresponding target detection rule is determined according to the spatial range relationship between the detection area and the corresponding camera coordinates, and the detection area is detected based on the target detection rule to obtain the area detection result;

[0205] For each component, the detection result corresponding to that component is obtained based on the detection results of all areas corresponding to that component.

[0206] Example 3

[0207] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an automatic optical inspection system for PCB panelization disclosed in an embodiment of the present invention. Figure 9 As shown, the PCB panel automated optical inspection system may include:

[0208] Memory 301 storing executable program code;

[0209] Processor 302 coupled to memory 301;

[0210] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the automatic optical inspection method for PCB panelization described in Embodiment 1 of the present invention.

[0211] Example 4

[0212] This invention discloses a computer storage medium storing computer instructions. When these computer instructions are invoked, they are used to execute the steps in the PCB panel automatic optical inspection method described in Embodiment 1 of this invention.

[0213] Example 5

[0214] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the PCB panel automatic optical inspection method described in Embodiment 1.

[0215] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0216] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0217] Finally, it should be noted that the embodiments disclosed in this invention are merely preferred embodiments of the invention and are only used to illustrate the technical solutions of the invention, not to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this invention.

Claims

1. An automatic optical inspection method for PCB panelization, characterized in that, The method includes: The camera is controlled to capture multiple local positions of the PCB panel under test, resulting in multiple local images; Obtain the camera coordinates at each shooting position and obtain the calibration parameters of the camera; For each local image, based on the corresponding camera coordinates and the camera calibration parameters, N local image regions in the local image that are located in different spatial ranges relative to the corresponding camera coordinates are determined, wherein the positions of all pixels in each local image region are within a preset spatial range relative to the corresponding camera coordinates. Based on N local image regions located in different spatial ranges on all local images, N stitched images are obtained based on a preset image stitching strategy; the image to be tested includes N stitched images; Identify the components on the image to be tested. For each component: Based on the component, the welding information and detection rules of the component are read from a preset relational database; the image of the component on the image to be tested is matched with the welding information of the component to obtain the detection area of ​​the component on the image to be tested; the detection area is detected based on the preset detection rules to obtain the detection result; The relational database is obtained by analyzing the design information of the PCB panel under test, and the relational database stores at least component soldering information and testing rules.

2. The automatic optical inspection method for PCB panelization according to claim 1, characterized in that, The relational database also includes a detection area mapping table, which is obtained by analyzing the PCB panel under test and is used to represent the correspondence between pads and pins. And, the step of detecting the detection area based on preset detection rules to obtain detection results includes: The pads in the detection area are detected based on preset detection rules to obtain the detection results of the pins corresponding to each pad.

3. The automatic optical inspection method for PCB panels according to claim 1, characterized in that, The controlled camera captures multiple local locations corresponding to the PCB panel under test, obtaining multiple local images, including: A pure black background image is generated based on the camera resolution and the size of the PCB panel to be tested. The camera is controlled to move relative to the PCB panel under test along a preset path and captures multiple local images during the movement. Any two adjacent local images have at least a partial overlap.

4. The automatic optical inspection method for PCB panelization according to claim 3, characterized in that, The controlled camera moves relative to the PCB panel under test along a preset path and captures multiple partial images during the movement, including: The camera is controlled to move relative to the PCB panel under test along a preset path so that the camera passes directly above multiple components. When the camera moves directly above the preset multiple components, it takes multiple partial images, and any two adjacent partial images have at least a partial overlap area.

5. The automatic optical inspection method for PCB panelization according to claim 1, characterized in that, The controlled camera captures multiple local locations corresponding to the PCB panel under test, obtaining multiple local images, including: Control the camera to capture M local positions corresponding to the PCB panel under test, and obtain M local images of the same size, where M is a positive integer greater than or equal to N; Furthermore, for each local image, based on the corresponding camera coordinates and the camera's calibration parameters, determining N local image regions located in different spatial ranges relative to the corresponding camera coordinates includes: For each local image, based on the corresponding camera coordinates and the camera calibration parameters, the local image regions located at the first angle, the second angle, and the third angle relative to the corresponding camera coordinates are determined, where the first angle, the second angle, and the third angle are preset spatial angles. And, the step of obtaining N stitched images based on N local image regions located in different spatial ranges on all local images, according to a preset image stitching strategy, includes: Based on the local image regions located at the first angle in all local images, a first stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the second angle in all local images, a second stitched image is obtained according to a preset image stitching strategy; based on the local image regions located at the third angle in all local images, a third stitched image is obtained according to a preset image stitching strategy.

6. The automatic optical inspection method for PCB panelization according to claim 1 or 5, characterized in that, The step of retrieving the component's soldering information and inspection rules from a preset relational database based on the component includes: For each component in each stitched image, the welding information and initial detection rules of the component are read from a preset relational database based on the component. Based on the spatial range relationship between the component, the stitched image and the corresponding camera coordinates, the detection parameters corresponding to the initial detection rules are set to obtain the detection rules. The detection parameters include at least a threshold parameter for measuring whether a defect exists. And, the step of detecting the detection area based on preset detection rules to obtain detection results includes: For each component, the component image on each stitched image is matched with the component welding information to obtain N detection areas of the component on N stitched images; for each detection area, the corresponding target detection rule is determined according to the spatial range relationship between the detection area and the corresponding camera coordinates, and the detection area is detected based on the target detection rule to obtain the area detection result; For each of the components, the detection result corresponding to that component is obtained based on the detection results of all the regions corresponding to that component.

7. An automatic optical inspection device for PCB panelization, characterized in that, The device includes: The control module is used to control the camera to capture multiple local positions corresponding to the PCB panel under test, obtaining multiple local images; to obtain the camera coordinates at each shooting position, and to obtain the camera calibration parameters; for each local image, based on the corresponding camera coordinates and the camera calibration parameters, to determine N local image regions located in different spatial ranges relative to the corresponding camera coordinates, wherein the positions of all pixels in each local image region are within a preset spatial range relative to the corresponding camera coordinates; based on the N local image regions located in different spatial ranges on all local images, and based on a preset image stitching strategy, to obtain N stitched images; the image under test includes N stitched images; The detection module is used to identify components on the image to be tested, and for each component, performs the following operations: Based on the component, the welding information and detection rules of the component are read from a preset relational database; the image of the component on the image to be tested is matched with the welding information of the component to obtain the detection area of ​​the component on the image to be tested; the detection area is detected based on the preset detection rules to obtain the detection result; The relational database is obtained by analyzing the design information of the PCB panel under test, and the relational database stores at least component soldering information and testing rules.

8. An automatic optical inspection system for PCB panelization, characterized in that, The system includes: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the automatic optical inspection method for PCB panel as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the PCB panel automatic optical inspection method as described in any one of claims 1-6.

Images