Defect detection method, device and equipment of printed circuit board and storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for defect detection of printed circuit boards (PCBs), relating to the field of defect detection technology. The method includes: acquiring an initial global image of the PCB; determining an image acquisition path based on the initial global image; controlling a camera movement platform to acquire images of the PCB along the image acquisition path, resulting in multiple image sequence sub-images; extracting local image features from the image sequence sub-images; stitching the image sequence sub-images into a panoramic image based on the local image features; identifying the difference information between the image sequence sub-images and a preset standard template sub-image; determining the defect detection result for each image sequence sub-image based on the difference information; and annotating the defect detection result in the panoramic image. This application can reduce the cost of PCB defect detection.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a method, apparatus, device, and storage medium for defect detection of printed circuit boards. Background Technology

[0002] In printed circuit board (PCB) defect inspection, fixed-focus cameras, due to their limited field of view, cannot simultaneously achieve complete coverage of large-sized boards and clear imaging of minute defects in a single shot. Traditional methods rely on expensive mobile platforms or multi-camera arrays, resulting in high costs. Therefore, reducing the cost of PCB defect inspection remains a problem that needs to be solved.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and storage medium for detecting defects in printed circuit boards, aiming to solve the technical problem of how to reduce the cost of defect detection in printed circuit boards.

[0005] To achieve the above objectives, this application proposes a defect detection method for printed circuit boards, the method comprising: An initial global image of the printed circuit board is acquired, an image acquisition path is determined based on the initial global image, and a camera moving platform is controlled to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images. Extract local image features from the sub-images of the image sequence; The image sequence sub-images are stitched together into a panoramic image based on the local image features; Identify the differences between the image sequence sub-images and the preset standard template sub-images, and determine the defect detection result for each image sequence sub-image based on the differences. The defect detection results are annotated in the panoramic image.

[0006] In one embodiment, the step of determining the image acquisition path based on the initial global image includes: The size information of the printed circuit board is determined based on the initial global image; Calculate the minimum number of shots required to completely cover the printed circuit board based on the size information; Multiple center point coordinates are determined based on the minimum number of shots, and the image acquisition path is generated according to the center point coordinates.

[0007] In one embodiment, the step of extracting local image features from the sub-images of the image sequence includes: The scale space of the image sequence sub-images is constructed using nonlinear diffusion filtering; Detect local extreme points in the scale space and determine the local extreme points as feature points; A binary descriptor is generated for each feature point, and the binary descriptor is determined as a local image feature.

[0008] In one embodiment, the step of stitching the image sequence sub-images into a panoramic image based on the local image features includes: Local image features of adjacent image sequence sub-images are matched to obtain matched feature point pairs; Based on the feature point pairs, calculate the spatial transformation relationship between adjacent image sequence sub-images; Based on the spatial transformation relationship, the image sequence sub-images are projected onto the same global coordinate system to generate a panoramic image.

[0009] In one embodiment, the step of identifying the difference information between the image sequence sub-image and the preset standard template sub-image includes: Obtain the local features of the preset standard template sub-image; The template space transformation relationship is calculated based on the local image features and the local template features. The image sequence sub-images are calibrated according to the template space transformation relationship to obtain the target image sequence sub-images; The target image sequence sub-image is compared with the preset standard template sub-image to obtain difference information.

[0010] In one embodiment, the step of determining the defect detection result for each image sequence sub-image based on the difference information includes: The overlap is obtained by calculating the intersection-union ratio based on the difference information. The defect detection result is determined based on the overlap and the preset defect judgment rules.

[0011] In one embodiment, the step of annotating the defect detection result in the panoramic image includes: Obtain the coordinate transformation relationship between the sub-images of the image sequence and the panoramic image; The global coordinate information of the defect detection result is determined based on the coordinate transformation relationship; The defect detection results are marked at the corresponding positions in the panoramic image based on the global coordinate information.

[0012] Furthermore, to achieve the above objectives, this application also proposes a defect detection device for printed circuit boards, the defect detection device for printed circuit boards comprising: The acquisition module is used to acquire an initial global image of the printed circuit board, determine an image acquisition path based on the initial global image, and control the camera moving platform to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images. The extraction module is used to extract local image features from the sub-images of the image sequence; The stitching module is used to stitch the image sequence sub-images into a panoramic image based on the local image features; The recognition module is used to identify the difference information between the image sequence sub-images and the preset standard template sub-images, and to determine the defect detection result of each image sequence sub-image based on the difference information; The annotation module is used to annotate the defect detection results in the panoramic image.

[0013] Furthermore, to achieve the above objectives, this application also proposes a defect detection device for printed circuit boards, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method for printed circuit boards as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the defect detection method for printed circuit boards as described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the printed circuit board defect detection method described above.

[0016] This application provides a method for defect detection of printed circuit boards (PCBs). The method involves acquiring an initial global image of the PCB, determining an image acquisition path based on the initial global image, and controlling a camera movement platform to acquire images of the PCB along the image acquisition path, resulting in multiple image sequence sub-images. Local image features are extracted from the image sequence sub-images. The image sequence sub-images are then stitched together into a panoramic image based on the local image features. Difference information between the image sequence sub-images and a preset standard template sub-image is identified, and the defect detection result for each image sequence sub-image is determined based on the difference information. The defect detection result is then labeled in the panoramic image. This method involves multiple partial shots of the PCB to obtain multiple sub-images, which are then used for defect detection and stitching to obtain a panoramic image of the PCB including the labeled defect detection results. Because it involves partial shooting, image clarity is ensured. Therefore, while maintaining defect detection accuracy, high-cost shooting equipment is not used, thus reducing the cost of PCB defect detection. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an embodiment of the defect detection method for printed circuit boards provided in this application; Figure 2 A flowchart illustrating Embodiment 2 of the defect detection method for printed circuit boards provided in this application; Figure 3 This is a schematic diagram of the module structure of the defect detection device for printed circuit boards according to an embodiment of this application; Figure 4 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the defect detection method of the printed circuit board in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] This application acquires an initial global image of a printed circuit board, determines an image acquisition path based on the initial global image, and controls a camera moving platform to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images; extracts local image features from the image sequence sub-images; stitches the image sequence sub-images into a panoramic image based on the local image features; identifies the difference information between the image sequence sub-images and a preset standard template sub-image, and determines the defect detection result of each image sequence sub-image based on the difference information; and marks the defect detection result in the panoramic image.

[0024] In printed circuit board (PCB) defect inspection, fixed-focus cameras, due to their limited field of view, cannot simultaneously achieve complete coverage of large-sized boards and clear imaging of minute defects in a single shot. Traditional methods rely on expensive mobile platforms or multi-camera arrays, resulting in high costs. Therefore, reducing the cost of PCB defect inspection remains a problem that needs to be solved.

[0025] This application involves taking multiple partial photos of the printed circuit board to obtain multiple sub-images. These sub-images are then used for defect detection and stitching to obtain a panoramic image of the printed circuit board that includes the defect detection results. Because the photos are taken in partial areas, the image clarity is guaranteed. Therefore, while ensuring the accuracy of defect detection, high-cost shooting equipment is not used, thus reducing the cost of defect detection for printed circuit boards.

[0026] Based on this, embodiments of this application provide a method for detecting defects in printed circuit boards, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the defect detection method for printed circuit boards according to this application.

[0027] In this embodiment, the defect detection method for the printed circuit board includes steps S10 to S50: Step S10: Obtain an initial global image of the printed circuit board, determine an image acquisition path based on the initial global image, and control the camera moving platform to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images; It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or a defect detection device for printed circuit boards capable of performing the above functions. The following description uses a defect detection device for printed circuit boards as an example to illustrate this embodiment and the subsequent embodiments.

[0028] It should be noted that this embodiment uses a camera with a fixed focal length and a limited field of view. When photographing large printed circuit boards (PCBs), such a camera cannot cover the entire large PCB motherboard in a single shot. If the resolution is reduced for global shooting, the ability to detect minute defects will be lost during subsequent defect detection, affecting the detection accuracy.

[0029] It should be noted that the initial global image refers to the image obtained by taking a low-resolution global preview shot of a large PCB motherboard using an industrial camera with a fixed focal length. Although the initial global image has low resolution, it can show the overall outline of the PCB. Then, object detection algorithms, such as deep learning-based boundary detection models or traditional image segmentation algorithms, can be used to identify the boundary contours of the initial global image. Based on the identified actual boundary dimensions and the fixed imaging field of view of the camera at a fixed focal length, the minimum number of shots required to completely cover the PCB board can be calculated. During the calculation, the principle of maintaining a 15%~25% overlap between adjacent sub-images is followed. Based on the minimum number of shots, an image acquisition path can be planned. Finally, the camera moving platform (such as an XY-axis electric displacement stage) is controlled to move sequentially to the center point of each partition on the path, triggering the camera to acquire high-resolution images, thereby obtaining a series of image sub-images covering the entire PCB board.

[0030] In one feasible approach, the step of determining the image acquisition path based on the initial global image includes: determining the size information of the printed circuit board based on the initial global image; calculating the minimum number of shots required to fully cover the printed circuit board based on the size information; determining multiple center point coordinates based on the minimum number of shots; and generating the image acquisition path according to the center point coordinates.

[0031] It should be noted that after acquiring the initial global image, the outline of the PCB board in the image is extracted using an edge detection algorithm (such as Canny edge detection). Then, the minimum bounding rectangle of the outline is calculated to obtain the actual physical size of the PCB board. Furthermore, given that the single-frame imaging field of view of the camera at a fixed focal length is known (e.g., 10mm horizontally and 8mm vertically), and the actual size of the PCB board (e.g., 100mm long and 80mm wide), it can be calculated that at least 100 / 10 = 10 shots are needed in the horizontal direction and at least 80 / 8 = 10 shots are needed in the vertical direction. Considering that 15%~25% overlap needs to be retained between adjacent sub-images for subsequent feature matching, the actual number of shots is appropriately increased from the theoretical minimum, for example, 12 shots in the horizontal direction and 12 shots in the vertical direction, for a total of 144 shots. Finally, based on the calculated number of shots in each direction and the overlap requirements, the entire area of ​​the PCB board is divided into multiple rectangular partitions, and the center point of each partition is the shooting position where the camera movement platform needs to stop. Arrange these center point coordinates into a continuous movement path to generate the final image acquisition path. The control platform moves sequentially to each center point and triggers a photo capture, thus obtaining a sequence of sub-images covering the entire board.

[0032] Step S20: Extract local image features from the sub-images of the image sequence; It should be noted that when extracting local image features, one can employ local feature extraction methods based on nonlinear scale space, such as the Accelerated-KAZE (AKAZE) algorithm; or other local feature extraction algorithms with scale invariance and rotation invariance, such as the Scale-Invariant Feature Transform (SIFT) algorithm. The extracted local image features are used for subsequent image matching and stitching.

[0033] In one feasible approach, the step of extracting local image features from the image sequence sub-image includes: constructing a scale space for the image sequence sub-image using a nonlinear diffusion filter; detecting local extrema points in the scale space and identifying the local extrema points as feature points; generating a binary descriptor for each feature point and identifying the binary descriptor as a local image feature.

[0034] It should be noted that the algorithm used in this embodiment is the AKAZE algorithm. The AKAZE algorithm constructs a scale space through nonlinear diffusion filtering. Nonlinear diffusion filtering is an adaptive image smoothing technique. Its core idea is to dynamically adjust the smoothing intensity based on the gradient changes in local image regions. In image edges or textured areas, the gradient value is large, so the smoothing intensity is weakened to maintain edge sharpness; in flat areas, the gradient value is small, so the smoothing intensity is strengthened to remove noise. By applying this adaptive smoothing operation multiple times, the image details are gradually reduced after each iteration, thus generating a series of image layers from fine to coarse. These image layers together constitute the nonlinear scale space. In the constructed nonlinear scale space, local extrema are detected for each image layer. Specifically, each pixel in the image is compared with all pixels in its surrounding neighborhood. This neighborhood includes the surrounding pixels in the same scale image, as well as the corresponding position and surrounding pixels in adjacent scale images. If the response value of a pixel (e.g., the intensity value of the pixel's grayscale value after some transformation) is greater than or less than the response values ​​of all other pixels in its neighborhood, then the pixel is marked as a local extrema. These local extrema are candidate feature points. To further improve the stability of the feature points, the candidate points need to be screened, specifically removing points with low contrast and unstable points located on the image edges. After screening, the remaining local extrema are the final feature points. For each determined feature point, a square image region is taken centered on it, and this region is divided into multiple uniform sub-regions (e.g., 16 sub-regions in 4 rows and 4 columns). Within each sub-region, the gradient information of the pixels is calculated, including the magnitude and direction of the gradient. Then, by comparing the gradient statistics of adjacent sub-regions, a binary bit string is generated. For example, the cumulative gradient value of a certain sub-region is compared with that of its right-side adjacent sub-region. If the former is greater than the latter, the corresponding bit is set to 1; otherwise, it is set to 0. Similarly, adjacent sub-regions in multiple directions, such as top and bottom, top left and bottom right, are compared. All comparison results are concatenated sequentially to form a sequence of 0s and 1s, which is the binary descriptor. This descriptor is called the Modified-Local Difference Binary (M-LDB) descriptor. Local difference binary descriptors occupy little storage space and only require bitwise XOR operations to calculate distances, making them extremely fast. At the same time, since the descriptors are constructed based on gradient information, they are highly robust to changes in image illumination, rotation, and scale.

[0035] Step S30: Stitch the image sequence sub-images into a panoramic image based on the local image features; It should be noted that during image stitching, feature matching can be performed on adjacent image sequence sub-images based on extracted local image features to find matching feature point pairs. Hamming distance or Euclidean distance can be used as a similarity metric for feature matching. Then, the spatial transformation relationship between adjacent sub-images is estimated based on the matched feature point pairs. When estimating the spatial transformation relationship, the Random Sample Consensus (RANSAC) algorithm can be used to eliminate erroneous matching point pairs and robustly estimate the homography matrix. Alternatively, other robust estimation methods can be used, such as the Least Median of Squares (LMedS) method or the Progressive Sample Consensus (PROSAC) algorithm. Finally, based on these spatial transformation relationships, all sub-images are projected onto the same global coordinate system, and image fusion methods, such as fade-in / fade-out weighted average fusion, multi-band fusion, or Laplacian pyramid fusion, are used to eliminate stitching seams, generating a complete, high-resolution panoramic image of the PCB board.

[0036] Step S40: Identify the difference information between the image sequence sub-image and the preset standard template sub-image, and determine the defect detection result of each image sequence sub-image based on the difference information; It should be noted that for a defect-free standard template PCB board, a preset standard template sub-image can be obtained by following the same acquisition path and parameters. The acquisition steps for the template PCB board can be pre-acquired, and the acquisition path and parameters can be set to various commonly used standards to obtain various preset standard template sub-images. When using them, you only need to select the corresponding preset standard template sub-image. Furthermore, the local features of the preset standard template sub-image can also be stored in the feature library in advance for use.

[0037] Understandably, for each image sequence sub-image to be detected, its local features can be matched with the local features of the corresponding standard template sub-image, and the spatial transformation relationship between the two, i.e., the registration parameter, can be calculated. The registration parameter can be calculated using feature point matching methods, such as RANSAC to estimate the homography matrix, or region matching methods, such as Normalized Cross Correlation (NCC) or Mutual Information (MI). Then, the sub-image to be detected undergoes spatial correction, such as perspective or affine transformation, to align it with the template sub-image. The corrected sub-image is then compared with the standard template sub-image to identify the difference regions. Deep learning-based object detection algorithms or region-based convolutional neural network algorithms can be used, which can directly locate defect candidate boxes and assign categories in the difference images; alternatively, traditional pixel-level comparison methods, such as image differencing or intersection-union (IoU) calculation, can be employed. After identifying the difference region, the presence of a defect in the sub-image is determined by combining the preset defect judgment rules (such as the area of ​​the difference region, aspect ratio, and other thresholds), thereby obtaining the defect detection result of each image sequence sub-image.

[0038] In one feasible approach, the step of identifying the difference information between the image sequence sub-image and the preset standard template sub-image includes: obtaining the template local features of the preset standard template sub-image; calculating the template space transformation relationship based on the local image features and the template local features; calibrating the image sequence sub-image according to the template space transformation relationship to obtain the target image sequence sub-image; and comparing the target image sequence sub-image with the preset standard template sub-image to obtain the difference information.

[0039] It should be noted that the local features of the preset standard template sub-image are already stored in the feature library and can be directly obtained. For the current image sequence sub-image to be detected, the local features of the corresponding standard template sub-image (including feature point coordinates and binary descriptors) are first read from the template feature library. Then, the feature descriptors of the sub-image to be detected are matched with the feature descriptors of the template sub-image, specifically using Hamming distance or nearest neighbor ratio methods to obtain matched feature point pairs. Next, the RANSAC algorithm is used to estimate the spatial transformation relationship from the sub-image to the template sub-image, i.e., the homography matrix. The homography matrix reflects the geometric deformation of the sub-image to be detected relative to the standard template due to factors such as platform positioning errors and PCB board tilt. Applying the homography matrix to the sub-image to be detected, i.e., performing a perspective transformation, aligns the sub-image with the standard template sub-image at the pixel level in geometric position. The transformed sub-image is the target image sequence sub-image. Bilinear interpolation can be used during the calibration process to ensure smooth, jagged images. The calibrated target image sequence sub-images are then compared with standard template sub-images to identify potential defect regions. Specifically, a defect detection model can be pre-trained, taking the difference features between the target image sequence sub-images and the standard template sub-images as input, and directly outputting the location and category of the defect. One feasible implementation is to superimpose or subtract the calibrated target image sequence sub-images from the standard template sub-images to generate a difference map, and then input this difference map into a trained object detection network, such as the YOLO (You Only Look Once) series of algorithms (e.g., YOLOv5, YOLOv8, etc.) or the Region-based Convolutional Neural Network (R-CNN) series of algorithms. These object detection algorithms can locate candidate boxes of suspected defects in the difference map and simultaneously provide the defect category to each candidate box.

[0040] In one feasible approach, the step of determining the defect detection result of each image sequence sub-image based on the difference information includes: calculating the intersection-union ratio based on the difference information to obtain the overlap; and determining the defect detection result based on the overlap and a preset defect judgment rule.

[0041] It should be noted that after obtaining the difference information, in order to determine whether a difference region is a real defect or a false alarm, it needs to be compared with the expected features of the corresponding region in the standard template. Specifically, for each difference region, the intersection-union ratio (IUR) with the corresponding position (same coordinate region) in the standard template sub-image is calculated. The IUR is defined as the area of ​​the intersection of the two regions divided by the area of ​​the union. For example, if the area of ​​the difference region is A, and the corresponding position in the standard template has a preset allowable tolerance area area of ​​B, then the overlap = (A∩B area) / (A∪B area). This overlap reflects the degree of overlap between the difference region and the acceptable deviation region. Then, the defect detection result is determined according to the overlap and the preset defect judgment rules. The preset defect judgment rules include multiple thresholds and conditions. For example: if the overlap is greater than 0.8, the difference region is judged as an acceptable process deviation (not a defect); if the overlap is between 0.3 and 0.8, and the area of ​​the difference region is greater than 100 pixels, it is judged as a potential defect and further inspection is required. If the overlap is less than 0.3, it is considered a defect. Furthermore, the geometric features of the differing regions (such as aspect ratio and roundness) can be matched with a defect type database. Finally, for each image sequence sub-image, a list of defect detection results is output, with each result containing information such as defect location, defect area, and confidence level.

[0042] Step S50: Mark the defect detection results in the panoramic image.

[0043] It should be noted that during the panoramic image processing, the coordinate transformation relationship from each image sequence sub-image to the panoramic image has already been obtained, i.e., the mapping matrix from the local coordinate system of the sub-image to the global coordinate system of the panoramic image. This matrix can be obtained by accumulating the homography matrix chain multiplication between adjacent sub-images. Therefore, the defect location on each detected sub-image, i.e., the local coordinates, can be converted into the global coordinates in the panoramic image. The coordinate transformation can be implemented using the Direct Linear Transformation (DLT) method or matrix multiplication based on homogeneous coordinates. Then, defect markers are drawn at the corresponding coordinate locations in the panoramic image (e.g., using colored rectangles, ellipses, or highlighted outlines to mark the defect area), and a defect type label is attached.

[0044] In one feasible approach, the step of annotating the defect detection result in the panoramic image includes: obtaining the coordinate transformation relationship between the image sequence sub-image and the panoramic image; determining the global coordinate information of the defect detection result based on the coordinate transformation relationship; and annotating the defect detection result at the corresponding position in the panoramic image based on the global coordinate information.

[0045] It should be noted that during the stitching of panoramic images, the cumulative spatial transformation matrix from each image sequence sub-image to the global coordinate system, i.e., the homography matrix chain, has already been calculated. These matrices constitute the coordinate transformation relationships. For each defect detection result, the coordinates of the four corner points of the defect rectangle can be transformed to the global coordinate system using these coordinate transformation relationships. Furthermore, different display methods can be used for different types of defects. For example, a red rectangle is drawn for short-circuit defects, a blue rectangle for open-circuit defects, and a yellow circle or highlighted outline for burr defects. Simultaneously, text labels are added next to the rectangles, indicating the defect type and size information, generating the final visualization image. This image can be displayed directly on the screen or saved as a file for subsequent quality analysis.

[0046] This embodiment acquires an initial global image of the printed circuit board (PCB), determines an image acquisition path based on the initial global image, and controls a camera movement platform to acquire images of the PCB along the image acquisition path, resulting in multiple image sequence sub-images. Local image features are extracted from the image sequence sub-images. The image sequence sub-images are then stitched together into a panoramic image based on the local image features. Difference information between the image sequence sub-images and a preset standard template sub-image is identified, and the defect detection result for each image sequence sub-image is determined based on the difference information. The defect detection result is then labeled in the panoramic image. This embodiment performs multiple partial shots of the PCB to obtain multiple sub-images, which are then used for defect detection and stitching to obtain a panoramic image of the PCB including the labeled defect detection results. Because it involves partial shooting, image clarity is ensured. Therefore, while maintaining defect detection accuracy, high-cost shooting equipment is not used, reducing the cost of PCB defect detection.

[0047] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S30 also includes steps S301 to S303: Step S301: Match the local image features of adjacent image sequence sub-images to obtain matched feature point pairs; It should be noted that two adjacent image sequences are taken as sub-images, and local image features, such as sets of binary descriptors, are obtained respectively. Hamming distance is used as a similarity metric. The Hamming distance between the descriptor of each feature point in the first sub-image and the descriptors of all feature points in the second sub-image is calculated, which is the number of different bits in the binary string. For each feature point in the first sub-image, the two feature points with the closest and second-closest Hamming distances are found in the second sub-image. If the ratio of the closest distance to the second-closest distance is less than a preset threshold (e.g., 0.7), the matching pair is considered valid. This process is repeated for all feature points to obtain a preliminary set of matched feature point pairs.

[0048] Step S302: Calculate the spatial transformation relationship between adjacent image sequence sub-images based on the feature point pairs; It should be noted that, due to the possibility of erroneous matches in the obtained matching point pairs, the RANSAC algorithm is used for robust estimation. Four pairs of matching points are randomly selected, and a candidate homography matrix is ​​calculated. This matrix describes the perspective transformation relationship from the second sub-image to the coordinate system of the first sub-image. Then, the number of interior points in all matching point pairs that conform to this transformation is counted. This process is repeated multiple times, and the homography matrix with the largest number of interior points is selected as the final spatial transformation relationship. This matrix includes parameters for translation, rotation, scaling, affine transformation, and perspective distortion.

[0049] Step S303: Project the image sequence sub-images onto the same global coordinate system according to the spatial transformation relationship to generate a panoramic image.

[0050] It should be noted that a reference coordinate system is selected, typically the coordinate system of the first sub-image (e.g., the first image in the top left corner) as the global coordinate system. Then, using the calculated homography matrix between adjacent sub-images, subsequent sub-images are transformed to the global coordinate system in turn. Finally, a complete, high-resolution panoramic image of the PCB board is output.

[0051] This embodiment matches local image features of adjacent image sequence sub-images to obtain matched feature point pairs; calculates the spatial transformation relationship between adjacent image sequence sub-images based on the feature point pairs; and projects the image sequence sub-images onto the same global coordinate system according to the spatial transformation relationship to generate a panoramic image. This embodiment, through local image feature matching of adjacent sub-images, calculation of spatial transformation relationships, and global projection fusion, achieves the generation of seamless and accurate panoramic images without the need for high-precision mechanical positioning, improving stitching efficiency and image quality, and providing a complete and high-resolution foundation for subsequent defect detection.

[0052] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the defect detection method of the printed circuit board of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0053] This application also provides a defect detection device for printed circuit boards, please refer to... Figure 3 The defect detection device for the printed circuit board includes: The acquisition module 10 is used to acquire an initial global image of the printed circuit board, determine an image acquisition path based on the initial global image, and control the camera moving platform to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images. Extraction module 20 is used to extract local image features of the image sequence sub-images; The stitching module 30 is used to stitch the image sequence sub-images into a panoramic image based on the local image features; The identification module 40 is used to identify the difference information between the image sequence sub-images and the preset standard template sub-images, and to determine the defect detection result of each image sequence sub-image based on the difference information; The annotation module 50 is used to annotate the defect detection results in the panoramic image.

[0054] This application acquires an initial global image of a printed circuit board (PCB), determines an image acquisition path based on the initial global image, and controls a camera moving platform to acquire images of the PCB along the image acquisition path, resulting in multiple image sequence sub-images. Local image features are extracted from the image sequence sub-images. The image sequence sub-images are then stitched together into a panoramic image based on the local image features. Difference information between the image sequence sub-images and a preset standard template sub-image is identified, and the defect detection result for each image sequence sub-image is determined based on the difference information. The defect detection result is then labeled in the panoramic image. This application performs multiple partial shots of the PCB to obtain multiple sub-images, which are then used for defect detection and stitching, resulting in a panoramic image of the PCB including the labeled defect detection results. Because it involves partial shooting, image clarity is ensured. Therefore, while maintaining defect detection accuracy, high-cost shooting equipment is not used, reducing the cost of PCB defect detection.

[0055] In one embodiment, the acquisition module 10 is further configured to determine the size information of the printed circuit board based on the initial global image; calculate the minimum number of shots required to fully cover the printed circuit board based on the size information; determine multiple center point coordinates based on the minimum number of shots; and generate the image acquisition path according to the center point coordinates.

[0056] In one embodiment, the extraction module 20 is further configured to construct the scale space of the image sequence sub-image using nonlinear diffusion filtering; detect local extrema points in the scale space and determine the local extrema points as feature points; generate a binary descriptor for each feature point and determine the binary descriptor as a local image feature.

[0057] In one embodiment, the stitching module 30 is further configured to match local image features of adjacent image sequence sub-images to obtain matched feature point pairs; calculate the spatial transformation relationship between adjacent image sequence sub-images based on the feature point pairs; and project the image sequence sub-images onto the same global coordinate system according to the spatial transformation relationship to generate a panoramic image.

[0058] In one embodiment, the recognition module 40 is further configured to acquire template local features of a preset standard template sub-image; calculate a template space transformation relationship based on the local image features and the template local features; calibrate the image sequence sub-image according to the template space transformation relationship to obtain a target image sequence sub-image; and compare the target image sequence sub-image with the preset standard template sub-image to obtain difference information.

[0059] In one embodiment, the identification module 40 is further configured to perform cross-union ratio calculation based on the difference information to obtain the overlap; and determine the defect detection result according to the overlap and a preset defect judgment rule.

[0060] In one embodiment, the annotation module 50 is further configured to obtain the coordinate transformation relationship between the image sequence sub-image and the panoramic image; determine the global coordinate information of the defect detection result based on the coordinate transformation relationship; and annotate the defect detection result at the corresponding position in the panoramic image based on the global coordinate information.

[0061] The printed circuit board defect detection device provided in this application, employing the printed circuit board defect detection method in the above embodiments, can solve the technical problem of how to reduce the cost of printed circuit board defect detection. Compared with the prior art, the beneficial effects of the printed circuit board defect detection device provided in this application are the same as those of the printed circuit board defect detection method provided in the above embodiments, and other technical features in the printed circuit board defect detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0062] This application provides a defect detection device for printed circuit boards. The defect detection device for printed circuit boards includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the defect detection method for printed circuit boards in the above embodiment 1.

[0063] The following is for reference. Figure 4 This document illustrates a structural schematic diagram of a defect detection device for printed circuit boards suitable for implementing embodiments of this application. The defect detection device for printed circuit boards in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The defect detection device for printed circuit boards shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application.

[0064] like Figure 4As shown, a printed circuit board defect detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the printed circuit board defect detection device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the printed circuit board defect detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show printed circuit board defect detection equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0065] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0066] The printed circuit board defect detection device provided in this application, employing the printed circuit board defect detection method described in the above embodiments, can solve the technical problem of how to reduce the cost of printed circuit board defect detection. Compared with the prior art, the beneficial effects of the printed circuit board defect detection device provided in this application are the same as those of the printed circuit board defect detection method provided in the above embodiments, and other technical features of this printed circuit board defect detection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0067] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0068] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0069] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the defect detection method for printed circuit boards in the above embodiments.

[0070] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0071] The aforementioned computer-readable storage medium may be included in a defect detection device for printed circuit boards; or it may exist independently and not be assembled into a defect detection device for printed circuit boards.

[0072] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a defect detection device for a printed circuit board, the defect detection device for the printed circuit board causes the following actions: acquiring an initial global image of the printed circuit board; determining an image acquisition path based on the initial global image; controlling a camera moving platform to acquire images of the printed circuit board along the image acquisition path, thereby obtaining multiple image sequence sub-images; extracting local image features from the image sequence sub-images; stitching the image sequence sub-images into a panoramic image based on the local image features; identifying the difference information between the image sequence sub-images and a preset standard template sub-image; determining the defect detection result for each image sequence sub-image based on the difference information; and marking the defect detection result in the panoramic image.

[0073] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0076] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described defect detection method for printed circuit boards, thereby solving the technical problem of how to reduce the cost of defect detection for printed circuit boards. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the defect detection method for printed circuit boards provided in the above embodiments, and will not be repeated here.

[0077] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the printed circuit board defect detection method described above.

[0078] The computer program product provided in this application solves the technical problem of how to reduce the cost of printed circuit board defect detection. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the printed circuit board defect detection method provided in the above embodiments, and will not be repeated here.

[0079] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method of defect detection of a printed circuit board, characterized by, The method includes: An initial global image of the printed circuit board is acquired, an image acquisition path is determined based on the initial global image, and a camera moving platform is controlled to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images. Extract local image features from the sub-images of the image sequence; The image sequence sub-images are stitched together into a panoramic image based on the local image features; Identify the differences between the image sequence sub-images and the preset standard template sub-images, and determine the defect detection result for each image sequence sub-image based on the differences. The defect detection results are annotated in the panoramic image.

2. The method as described in claim 1, characterized in that, The step of determining the image acquisition path based on the initial global image includes: The size information of the printed circuit board is determined based on the initial global image; Calculate the minimum number of shots required to completely cover the printed circuit board based on the size information; Multiple center point coordinates are determined based on the minimum number of shots, and the image acquisition path is generated according to the center point coordinates.

3. The method as described in claim 1, characterized in that, The step of extracting local image features from the sub-images of the image sequence includes: The scale space of the image sequence sub-images is constructed using nonlinear diffusion filtering; Detect local extreme points in the scale space and determine the local extreme points as feature points; A binary descriptor is generated for each feature point, and the binary descriptor is determined as a local image feature.

4. The method as described in claim 1, characterized in that, The step of stitching the image sequence sub-images into a panoramic image based on the local image features includes: Local image features of adjacent image sequence sub-images are matched to obtain matched feature point pairs; Based on the feature point pairs, calculate the spatial transformation relationship between adjacent image sequence sub-images; Based on the spatial transformation relationship, the image sequence sub-images are projected onto the same global coordinate system to generate a panoramic image.

5. The method as described in claim 1, characterized in that, The step of identifying the difference information between the image sequence sub-image and the preset standard template sub-image includes: Obtain the local features of the preset standard template sub-image; The template space transformation relationship is calculated based on the local image features and the local template features. The image sequence sub-images are calibrated according to the template space transformation relationship to obtain the target image sequence sub-images; The target image sequence sub-image is compared with the preset standard template sub-image to obtain difference information.

6. The method as described in claim 1, characterized in that, The step of determining the defect detection result for each image sequence sub-image based on the difference information includes: The overlap is obtained by calculating the intersection-union ratio based on the difference information. The defect detection result is determined based on the overlap and the preset defect judgment rules.

7. The method as described in claim 1, characterized in that, The step of annotating the defect detection results in the panoramic image includes: Obtain the coordinate transformation relationship between the sub-images of the image sequence and the panoramic image; The global coordinate information of the defect detection result is determined based on the coordinate transformation relationship; The defect detection results are marked at the corresponding positions in the panoramic image based on the global coordinate information.

8. A defect detection device for printed circuit boards, characterized in that, The device includes: The acquisition module is used to acquire an initial global image of the printed circuit board, determine an image acquisition path based on the initial global image, and control the camera moving platform to acquire images of the printed circuit board along the image acquisition path to obtain multiple image sequence sub-images. The extraction module is used to extract local image features from the sub-images of the image sequence; The stitching module is used to stitch the image sequence sub-images into a panoramic image based on the local image features; The recognition module is used to identify the difference information between the image sequence sub-images and the preset standard template sub-images, and to determine the defect detection result of each image sequence sub-image based on the difference information; The annotation module is used to annotate the defect detection results in the panoramic image.

9. A defect detection device for printed circuit boards, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method for a printed circuit board as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the defect detection method for printed circuit boards as described in any one of claims 1 to 7.

Images