Bga packaging drawing pin coordinate adaptive extraction method based on pin detection yolo network

Abstract

The application discloses a BGA packaging drawing pin coordinate adaptive extraction method based on a pin detection YOLO network, and the method comprises the following steps: obtaining an original image of a BGA packaging bottom view; performing chip region adaptive extraction on the original image to obtain a chip main body region image, which comprises the following steps: performing preprocessing of watermark removal and interference filtering on the original image, performing outer contour extraction based on the preprocessed image, and extracting a chip outer frame by adopting a double-track projection method combined with a multi-level fault tolerance mechanism to obtain the chip main body region image; embedding a PSDWA module at the end of a backbone network of a YOLO baseline network to construct a PIN-YOLO pin detection network; inputting the chip main body region image into the PIN-YOLO pin detection network which is pre-trained to perform pin detection and coordinate extraction. The application constructs a chip region adaptive extraction framework and a PIN-YOLO pin detection network to realize high-precision detection and coordinate extraction of BGA pins.

Description

Technical Field

[0001] This invention relates to the fields of electronic design automation, integrated circuit package drawing parsing, computer vision, object detection, deep learning, and image processing, specifically to an adaptive extraction method for pin coordinates of BGA package drawings based on a pin detection YOLO network. Background Technology

[0002] In the field of Electronic Design Automation (EDA), Ball Grid Array (BGA) packaging has become the mainstream packaging form for high-end chips due to its high-density interconnects and excellent electrical performance. Accurately obtaining BGA pin coordinates is an essential prerequisite for critical stages such as PCB layout and routing, package verification, and reverse engineering. However, mainstream industry datasheets provide pin distribution information in the form of unstructured two-dimensional engineering drawings, lacking machine-readable formats. Pin coordinates heavily rely on manual point-by-point verification and annotation. This manual extraction method is not only time-consuming, labor-intensive, and inefficient, but also susceptible to fatigue and subjective judgment, leading to problems such as incorrect package size matching and pin positioning deviations. These issues are significant sources of EDA design errors, severely hindering the shortening of design cycles and the improvement of automation levels.

[0003] Existing pin coordinate extraction technologies are mainly divided into two categories: traditional image processing methods and deep learning methods. Traditional methods often rely on manually designed features such as threshold segmentation, edge detection, contour analysis, Hough transform, and roundness screening, which have certain effects on real-world images or X-ray images. However, these methods essentially depend on fixed geometric priors and manual parameter tuning, resulting in weak generalization ability and difficulty in adapting to heterogeneous engineering drawing scenarios. Actual drawings generally contain complex interference such as watermarks, copyright text, dimension lines, and direction markings, and the drawing specifications of different manufacturers vary greatly. Pin shapes include various types such as solid circles, hollow circles, cross marks, and irregular ellipses. Pins are small and dense, with blurred edges and severe adhesion, making traditional algorithms prone to missed detections, false detections, and positioning drift, resulting in insufficient robustness.

[0004] In recent years, deep learning models, represented by YOLO, have made breakthroughs in the field of object detection, with some studies applying them to pin defect detection or physical image localization. However, most existing deep learning solutions are geared towards clean backgrounds and regular pin patterns in real-world images or X-ray images, exhibiting extremely poor adaptability when directly transferred to engineering drawings composed of lines. Issues such as high overlap between dense lines and pin edges, heterogeneous shapes, small scale, and compression distortion in drawings lead to feature confusion, semantic discrimination failure, and a sharp decline in detection accuracy. Furthermore, existing models lack targeted feature enhancement mechanisms, making it difficult to simultaneously perceive the global arrangement of small targets and preserve local details, resulting in significant performance bottlenecks in high-density pin scenarios.

[0005] In addition, existing technologies generally lack adaptive preprocessing processes for heterogeneous drawings, and cannot automatically adapt to various chip outline shapes such as regular, chamfered, dashed, and frameless, making it difficult to suppress background interference from the source; the network structure also lacks feature enhancement modules for small and distorted pins, making it difficult to balance detection accuracy and robustness.

[0006] Therefore, the industry urgently needs a technical solution that can adapt to heterogeneous drawings from different manufacturers, automatically remove complex interference, and extract BGA pin coordinates with high precision and stability. This solution can replace manual operation, reduce costs, improve efficiency, and at the same time ensure the accuracy and consistency of coordinate extraction, meeting the actual application requirements of EDA library construction and industrial production. Summary of the Invention

[0007] To address the issues of existing technologies in processing BGA bottom package views, where watermarks, indicator lines, and text interference lead to numerous false positives and missed detections due to direct identification, and the diverse heterogeneous chip outlines making precise location of the core chip area difficult, this invention provides an adaptive pin coordinate extraction method for BGA package drawings based on a pin detection YOLO network. It proposes an adaptive chip region extraction framework that effectively filters complex background interference and accurately trims the effective chip area. Furthermore, to address the problems of heterogeneous pin shapes from different manufacturers, densely packed pins, and the susceptibility to false positives and missed detections and insufficient accuracy of traditional detection methods, a pin detection YOLO network (PIN-YOLO) is proposed to enhance the feature extraction capability of tiny pins and suppress line interference. Through these two core technologies, this invention successfully achieves end-to-end accurate extraction from the bottom view of heterogeneous BGAs to pin coordinates, significantly improving detection accuracy, reducing false positives and missed detections, and enhancing cross-vendor adaptability and industrial robustness.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides an adaptive extraction method for pin coordinates of BGA package drawings based on a pin detection YOLO network, the method comprising: Obtain the original image of the bottom view of the BGA package; Adaptive chip region extraction is performed on the original image to obtain the chip main region image. This includes: preprocessing the original image by removing watermarks and filtering interference, extracting the outer contour based on the preprocessed image, and using a dual-track projection method combined with a multi-level fault tolerance mechanism to extract the chip outer frame to obtain the chip main region image. A PSDWA module is embedded at the end of the backbone network of the YOLO baseline network to construct a PIN-YOLO pin detection network; The image of the main area of ​​the chip is input into a pre-trained PIN-YOLO pin detection network to perform pin detection and coordinate extraction.

[0009] In a possible embodiment of the present invention, the preprocessing of the original image for watermark removal and interference filtering includes: Watermark Removal: Removes the watermark from the original image to obtain a watermark-free image; External interference filtering: Convert the watermark-removed image to a grayscale image, then perform inverse binarization to convert the grayscale image to a black and white binary image. Use a connected component analysis algorithm to remove small connected components with an area smaller than a preset threshold in the black and white binary image. Perform a bitwise AND operation between the mask after removing the small connected components and the watermark-removed image to complete the initial cropping and obtain the external interference-removed image. Internal interference filtering: Gaussian low-pass filtering is applied to the image to eliminate external interference, and global binarization is performed using the adaptive Otsu method. The optimal threshold is determined by maximizing the inter-class variance. After global binarization, morphological closing operation is performed to complete the internal interference filtering.

[0010] In a possible embodiment of the present invention, the outer contour is extracted based on the preprocessed image, and the chip outline is extracted using a dual-track projection method combined with a multi-level fault-tolerant mechanism to obtain an image of the chip's main body region, including: All closed boundaries are extracted from the preprocessed image. Based on the prior knowledge that the chip outline is the largest closed region in the image, candidate outer contours of the chip body are selected. If the candidate outer contour meets the preset area constraint, the first outline of the candidate outer contour is extracted based on the projection method of the outer contour to obtain the chip main body region image. If no closed boundary is extracted, or the candidate outer contour does not meet the preset area constraint conditions, or the first-track outer frame extraction fails, the second-track outer frame extraction is performed on the preprocessed image based on the projection method of the original image to obtain the chip main body region image.

[0011] In a possible embodiment of the present invention, the extraction of the first track outline includes: Morphological filtering optimization is performed on the image of the candidate region for the outer contour. The row and column projection features of the image after morphological filtering optimization are calculated. The horizontal and vertical projections are calculated separately, and the projection results reflect the pixel distribution features of the contour region in the horizontal and vertical directions. A dynamic threshold is set based on the projection peak value, and a spatial distance clustering algorithm is used to remove duplicate boundaries to obtain candidate outer frame boundaries; The vertical boundary is selected as the left and right boundary based on the peak value of the column integral projection. If the optimal vertical boundary spacing is less than the preset width threshold, the extreme value mode is switched to select the outermost edge line as the left and right boundary. Horizontal projection is performed within the left and right boundary constraints, and the outermost horizontal line is selected as the upper and lower boundaries according to the principle of maximum inclusiveness. Based on the determined upper and lower boundaries and left and right boundaries, the chip main body area image is cropped from the watermark-removed image.

[0012] Furthermore, the chip main body region image extracted from the first outline is optimized for anti-black border cropping, including: By combining local window grayscale features with global connected region area information, explicit black borders or implicit dark areas in images can be identified and removed. Scan line by line or column by column from the image edge inwards to find the first pixel that exceeds the effective content threshold as the new cropping boundary, and perform adaptive boundary cropping. One-dimensional projection is performed based on a center avoidance strategy. The cropping result is optimized by utilizing the sharp drop in the peak value of the chip boundary projection, resulting in the final image of the chip's main body region.

[0013] In a possible embodiment of the present invention, the extraction of the second track outline includes: Set a black tolerance threshold and convert the preprocessed image into a dark feature binary map; The dark feature binary image is projected horizontally and vertically. A dynamic threshold is set based on the projection peak value. The spatial distance clustering algorithm is used to remove duplicate boundaries to obtain candidate outer frame boundaries. Define a vertical center avoidance zone, and select the straight line closest to the edge in the upper and lower subdomains as the upper and lower boundaries respectively; Extract a set of candidate vertical lines from the binary image of dark features, and perform multi-criteria verification on the candidate vertical lines: when there are two vertical lines, verify whether the minimum width constraint is met; when there are three or more vertical lines, introduce a symmetry axis consistency detection, and take the two outer lines that meet the preset pixel-level tolerance as the left and right boundaries. If the multi-criteria verification fails to output a valid boundary, a two-stage fault-tolerant verification is initiated: the local vertical projection is re-executed within the strip of the determined upper and lower boundary constraints, and the vertical boundary is determined in conjunction with the horizontal center avoidance logic; if it is still ineffective, the center avoidance logic constraint is removed, and the second track outline extraction is re-executed.

[0014] In a possible embodiment of the present invention, the PIN-YOLO pin detection network includes a backbone network, a PSDWA module, a feature pyramid network, and a detection head; The PSDWA module employs a dual-path parallel architecture, including an identity path and an enhanced path: The input feature map is split into two parallel sub-feature maps along the channel dimension: the identity path sub-feature map and the enhanced path sub-feature map; The identity path does not perform any transformation on the identity path sub-feature map and directly passes it on to the next step; The enhancement path processes the sub-feature maps sequentially through a deep convolutional weighted attention submodule and a feedforward network: the deep convolutional weighted attention submodule extracts the spatial structure of the features through depthwise separable convolutions and learns channel interaction through global average pooling and a fully connected network; the feedforward network uses a lightweight structure to enhance the features. The feature maps of the identity path and the enhancement path are fused to output the final enhanced feature map.

[0015] In a possible embodiment of the present invention, pin detection and coordinate extraction are performed, including: Input the image of the main chip area into the PIN-YOLO pin detection network to output pin candidate boxes; Filter the candidate pin boxes according to a preset confidence threshold; Calculate the center point coordinates of each retained pin candidate box as the pixel position of the corresponding pin, and organize the center point coordinates of all pins into a structured pin coordinate list.

[0016] In a possible embodiment of the present invention, the training steps of the PIN-YOLO pin detection network include: Collect bottom view images of different BGA packages as raw samples; For images with valid outlines, the chip body area image is obtained through adaptive extraction of the chip region, and the pin positions are manually marked; for images without outlines or with invalid outlines, the pin positions are manually marked directly on the original image. Data augmentation strategies are employed to expand labeled samples and construct a pin recognition dataset; wherein, the data augmentation strategies include one or more of rotation, scaling, cropping, noise addition, blurring, and contrast adjustment; The pin identification dataset is input into the constructed PIN-YOLO pin detection network, and a dynamic learning rate adaptive strategy is used for training to obtain a pre-trained PIN-YOLO pin detection network.

[0017] In a second aspect, the present invention provides a computer device / apparatus / system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0018] Thirdly, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0019] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This invention adopts a heterogeneous collaborative architecture of "chip region adaptive extraction + PIN-YOLO detection", which is different from single traditional image processing or deep learning methods. It designs processing schemes for different morphological characteristics of chip outline and pins, making the extraction decision more targeted and effectively solving the problem of poor adaptability of existing technologies to heterogeneous drawings.

[0020] Secondly, this invention, through dual-track projection and multi-level fault-tolerant mechanisms, can adaptively adapt to various heterogeneous scenarios such as complete outer frame, dashed outer frame, unclosed outer frame and no outer frame, and can complete the extraction of the main chip area without manual intervention. At the same time, it optimizes the removal of background interference by anti-black edge clipping, providing high-quality input for pin detection and significantly reducing the probability of missed detection and false detection.

[0021] Furthermore, this invention embeds a PSDWA dual-path parallel module into the YOLO baseline network. This module preserves the underlying geometric details of the pins through the identity path and amplifies the key feature responses through the enhancement path. This solves the shortcomings of existing deep learning models in terms of insufficient accuracy and loss of details in detecting small and dense pins, and improves the accuracy of pin localization and coordinate extraction.

[0022] Furthermore, in the preferred embodiment, the present invention achieves end-to-end automated integration, provides standardized input and output interfaces, and can directly connect to EDA tools without the need for manual annotation or parameter adjustment, thereby reducing the cost of engineering applications. At the same time, through operation log recording, it is convenient to track the extraction process, debug and optimize methods, and improve the feasibility and verifiability of the project.

[0023] Furthermore, in the dependent embodiments, the generalization ability of the method can be further improved by means of dataset enhancement and expansion, network structure optimization, and threshold adaptive tuning, so as to adapt to BGA package drawings of different quality and different manufacturers. At the same time, it can be extended to the extraction of pin coordinates of other packaged devices, thus broadening the scope of application of the method. Attached Figure Description

[0024] Figure 1 This is an overall flowchart of the BGA package image pin detection process according to an embodiment of the present invention; Figure 2 This is a flowchart of an image preprocessing process according to an embodiment of the present invention; Figure 3 This is a flowchart of the dual-track outline extraction process according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the dataset construction and model training process according to an embodiment of the present invention. Figure 5 This is a schematic diagram of a pin detection YOLO network structure according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the internal structure of the PSDWA module according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided by this invention without inventive effort are within the scope of protection of this invention.

[0026] Obviously, the accompanying drawings described below are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, modifications to design, manufacturing, or production based on the technical content disclosed in this invention are merely conventional technical means and should not be construed as insufficient disclosure of the present invention.

[0027] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention may be combined with other embodiments without conflict.

[0028] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "a," "an," "an," "the," and similar words used in this invention do not indicate quantity limitation and may indicate singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms "connected," "linked," "coupled," and similar words used in this invention are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "A plurality" used in this invention refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships may exist; for example, "A and / or B" can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects have an "or" relationship. The terms "first," "second," and "third" used in this invention are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0029] The core solution of this invention addresses the challenge of extracting pin coordinates from heterogeneous schematics in BGA package datasheets. It constructs a technical architecture that integrates a chip region adaptive extraction framework with a PIN-YOLO network. First, through morphological operations, one-dimensional projection analysis, and multi-level fault-tolerant mechanisms, interference is removed from the original schematics, and the main chip region (pin array core area) is accurately extracted. Then, using a PIN-YOLO network embedded with a PSDWA module, the semantic discrimination capability and the fidelity of underlying geometric details of the micro-pins are simultaneously enhanced, achieving high-precision detection and coordinate extraction of BGA pins. This core solution does not rely on fixed prior morphology, manual annotation, or specific manufacturer's schematic specifications, and is adaptable to BGA package schematics from different manufacturers and with different configurations.

[0030] This invention provides an adaptive pin coordinate extraction method for BGA package drawings based on PIN-YOLO. It is applicable to pin coordinate extraction scenarios in Electronic Design Automation (EDA) workflows for BGA package datasheets, and also suitable for pin coordinate extraction tasks of other packaged devices with pin array characteristics, background interference, and strong drawing heterogeneity (such as QFN, SOP, etc.). Preferably, this invention can be deployed as an end-to-end automated extraction process; in some dependent embodiments, dataset expansion, model pre-training, and parameter tuning can serve as auxiliary paths for system performance optimization and generalization capability improvement.

[0031] The "heterogeneous drawings" referred to in this invention are bottom view images of BGA package datasheets provided by different electronic device manufacturers. Their heterogeneity is specifically manifested in the diverse shapes of the chip outline (conventional rectangle, chamfered rectangle, dashed frame, open frame or no outline), different pin shapes (hollow circle, solid circle, with cross mark, irregular ellipse), complex background interference (dimension indicator lines, row and column characters, watermarks), and image degradation (jagged edges, pixel loss). The above heterogeneous characteristics make it difficult for traditional extraction methods to achieve robust automated coordinate analysis.

[0032] The “chip body area” referred to in this invention refers to the core area containing the pin array in the original BGA package drawing, excluding peripheral dimension indicator lines, characters, watermarks and other interfering elements. This area is the basic input area for achieving accurate pin coordinate extraction. The “PSDWA module”, namely the parallel split depth convolution weighted attention module, is the core enhancement module of the PIN-YOLO network. It adopts a dual-path parallel architecture to simultaneously preserve the underlying geometric details of the pins and enhance semantic discrimination capabilities.

[0033] Compared with existing technologies, this invention does not employ a single traditional image processing or deep learning method, but rather a heterogeneous collaborative architecture of "traditional methods for extracting macroscopic regions + deep learning for recognizing microscopic pins": Given the clear rectangular geometric priors and enumerable shapes of the chip outline, a training-free traditional image processing method is used to achieve lightweight, cross-vendor main region extraction; considering the small, dense, diverse, and easily interfered-prone characteristics of pins, a PIN-YOLO network is used to achieve high-precision detection. This avoids the shortcomings of traditional methods, such as sensitivity to interference and high false positive / false negative rates, and also solves the problems of poor adaptability to heterogeneous drawings and loss of low-level details in existing deep learning models. Without relying on manual intervention, it significantly improves the accuracy and robustness of BGA pin coordinate extraction.

[0034] In a preferred embodiment of the present invention, the system is developed based on the PyTorch deep learning framework, integrating modules such as image preprocessing, chip main region extraction, PIN-YOLO model training and inference, and coordinate output. It provides standardized input interfaces (supporting formats such as BGA package drawing screenshots and PDF exported images) and output interfaces (supporting structured pin coordinate list export), enabling the system to directly interface with EDA tools for automated pin coordinate parsing and reuse. In this preferred embodiment, the system at least supports offline batch processing mode, online single-image detection mode, and adjustable model parameter mode, providing a foundation for engineering deployment rather than merely remaining at the laboratory verification stage.

[0035] Figure 1This is a schematic diagram illustrating the overall process of the PIN-YOLO-based adaptive extraction method for BGA package schematics provided by this invention. Figure 1 As shown, this method includes two core steps: adaptive chip region extraction and PIN-YOLO pin detection. The overall implementation process includes the following steps: S100: Acquire original BGA package schematic images. Collect bottom view images from BGA device datasheets of different manufacturers. The original images can be in formats such as lossless screenshots or exported PDF images. The images contain pin arrays, chip outlines (or no outline), and background interference elements (size indicators, characters, watermarks, etc.). Some images may exhibit degradation phenomena such as jagged edges or pixel loss. In some embodiments, the original images can be format-unified by converting all images into RGB images of a preset size for easier subsequent unified processing.

[0036] S200: Perform adaptive chip region extraction to obtain a standardized image of the main chip region. This step is the core execution process of the adaptive chip region extraction framework, used to remove background interference and lock the core area of ​​the pin array, such as... Figure 3 As shown, the specific steps include the following: S201: Image preprocessing. For example... Figure 2 As shown, the input raw RGB image undergoes multi-step preprocessing to eliminate structural noise. The specific process is as follows: S2011: Watermark Removal. A high-threshold truncation algorithm is used to remove high-brightness watermarks from the image, avoiding interference from the watermark on subsequent outline extraction and pin detection; S2012: External Interference Filtering. The watermark-removed RGB image is reduced to grayscale, and then inverse binarization is performed to convert it into a black and white binary image, highlighting the difference between the target areas such as pins and outlines and the background. Connected component analysis (CCA) is used to remove tiny connected components with an area smaller than a preset threshold from the binary image, eliminating redundant interference such as small characters at the edges and fine indicator lines. A bitwise AND operation is performed between the mask after removing tiny connected components and the original image to complete the initial cropping, narrowing the processing range.

[0037] S2013: Internal interference filtering. Gaussian low-pass filtering is introduced to weaken the gradient at the pin edges and reduce the interference of high-frequency textures on global geometric features. Adaptive Otsu's method is used for global binarization. The optimal threshold is determined by maximizing the inter-class variance. After binarization, morphological closing operation is performed to fill the gaps between pins and smooth the pin area into an internally connected foreground block, further eliminating high-frequency texture interference.

[0038] S2014: Outer contour extraction. The Suzuki topology tracking algorithm is used to extract all closed boundaries in the preprocessed image. Based on the prior knowledge that "the chip outline is the largest closed region in the image", candidate physical outlines of the chip body are selected.

[0039] If no closed contour is extracted, or the area of ​​the extracted closed contour is abnormal, it is determined that the outer frame is not closed, is a dashed line, or is missing, and proceed to step S204; if a valid closed contour is extracted normally, otherwise proceed to step S202.

[0040] S202: First-track outline extraction (projection method based on outer contour). Applicable to cases with a complete closed outline. The specific process is as follows: S2021: Morphological filtering optimization. Morphological operations are performed on the input outer contour image to remove small-region noise and discontinuous edges, smoothing the outer contour. S2022: Row and column projection feature calculation. For the selected largest contour region, calculate the horizontal and vertical projections respectively, and reflect the pixel distribution characteristics of the contour region in the horizontal and vertical directions through the projection results; S2023: Linear deduplication filtering. A dynamic threshold is set based on the projected peak value to lock the candidate bounding box boundaries, and a spatial distance clustering algorithm is used to remove duplicate boundaries; S2024: Vertical line selection rules. Prioritize filtering the vertical boundaries (left and right boundaries) based on the peak value of the column integral projection. If the spacing between the optimal vertical boundary pairs is too narrow, switch to extreme value mode and select the outermost edge line as the left and right boundaries. S2025: Lateral detection of constrained region: Perform lateral projection within the longitudinal boundary constraint, and select the outermost horizontal line as the upper and lower boundaries according to the principle of maximum inclusiveness. S2026: External interference cancellation image cropping: The external interference cancellation image obtained in S2012 is cropped based on the obtained chip boundary information to remove edge interference elements and obtain a preliminary chip main body area image; If the main chip area is not extracted, or the area of ​​the initially extracted main chip area is abnormal (much smaller than the normal chip area), the outer frame extraction is determined to have failed, and the process proceeds to step S204; if the extraction result is normal, the process proceeds to step S203.

[0041] S203: Anti-black border cropping optimization. Removes edges or residual black interference areas from the initial chip main area image, improving the accuracy of the chip main area. The specific process is as follows: S2031: Combine local window grayscale features with global connected region area information to identify explicit black borders or implicit dark areas in an image; S2032: Adaptive boundary cropping: Scan row by row (column) inward from the image edge to find the first pixel that exceeds the effective content threshold as the new cropping boundary; S2033: Projection Peak Re-optimization: One-dimensional projection is performed based on the center avoidance strategy. The clipping result is further optimized by utilizing the sharp drop characteristic of the chip boundary projection peak, so as to obtain more accurate information on the main chip area.

[0042] S204: Second-track outline extraction (projection method based on the original image). Specifically designed for cases where the outline is not closed, has a dashed line, or is missing. It directly processes the original image with a white background and black lines. The specific process is as follows: S2041: Dark Feature Extraction: Set a black tolerance threshold and convert the original image into a binary image; S2042: Row and column projection line detection. For the selected largest contour region, calculate the horizontal and vertical projections respectively, and reflect the pixel distribution characteristics of the contour region in the horizontal and vertical directions through the projection results; S2043: Deduplication and threshold filtering. A dynamic threshold is set based on the projected peak value to lock the candidate bounding box boundaries, and a spatial distance clustering algorithm is used to remove duplicate boundaries. S2044: Horizontal line layered filtering. Define a vertical center avoidance area, and select the line closest to the edge as the upper and lower boundaries in the upper and lower subdomains respectively; S2045: Vertical Boundary Extraction. Extract a set of candidate vertical lines from the binary image and perform multi-criteria verification on the candidate vertical lines: for a set of binary vertical lines, the minimum width constraint must be met; for a set of ternary or higher vertical lines, a symmetry axis consistency check is introduced. If the preset pixel-level tolerance requirement is met, the two outer lines are determined to be the real physical borders. S2046: Two-stage fault-tolerant verification. If none of the geometric rules are triggered, an adaptive backoff mechanism is initiated. Within the defined horizontal strip constraints, the local vertical projection is re-executed, and in conjunction with the horizontal center avoidance logic, the vertical boundary is precisely locked. If this is still ineffective, the center avoidance constraint is revoked, and step S204 is re-executed to relax the conditions and re-detect the outer frame.

[0043] S205: Cropping based on the watermark-free image. The extracted bounding box parameters are mapped back to the original watermark-free image, and fine segmentation is performed to obtain the final image of the chip's main body region.

[0044] S300: Construction of PIN-YOLO pin detection network. Using YOLOv8 as a baseline, a PIN-YOLO network is constructed by introducing the PSDWA module. The specific process is as follows: S301: Network construction. For example... Figure 5As shown, a PIN-YOLO network is constructed, which includes a backbone network, a PSDWA module, a feature pyramid network (Neck), and a detection head (Head). The core change is to embed the PSDWA module after the SPPF module at the end of the backbone network. S302: PSDWA module deployment. For example... Figure 6 As shown, the PSDWA module adopts a dual-path parallel architecture, and its specific working process is as follows: S3021: Feature splitting. After expanding the input feature map through a 1×1 convolution, it is uniformly split into two parallel sub-feature maps A (identity path) and B (enhanced path) along the channel dimension. S3022: Identity path processing. Identity path A does not undergo any transformation, directly preserving the original features to ensure that low-level geometric details such as pin outlines are transferred to subsequent layers without loss, preventing feature degradation; S3023: Enhancement Path Processing. Enhancement path B is processed sequentially through a Deep Convolutional Weighted Attention (DWA) submodule and a Feedforward Network (FFN): The DWA submodule extracts the spatial structure of the features using deep separable convolutions, and amplifies the key feature responses by learning channel interaction through global average pooling and a fully connected network; the FFN adopts a lightweight structure to further enhance the features; S3024: Feature Fusion. The identity path feature A and the enhanced path feature are concatenated along the channel dimension, then fused through a 1×1 convolutional layer to restore the original number of channels, and the final enhanced feature map is output.

[0045] S400: Construct a BGA pin identification dataset and complete model training. For example... Figure 4 As shown, based on the chip body area image extracted in step S200, a BGA pin recognition dataset is constructed for the training, validation, and testing of the PIN-YOLO network. The specific process is as follows: S401: Raw Sample Collection. Collecting raw bottom view images from BGA device datasheets of different manufacturers; S402: Sample annotation. For images with an outer frame, first extract the main chip area using step S200, then manually annotate the pin positions; for images without an outer frame, directly annotate the pin positions on the original image manually. S403: Data Augmentation: Employ data augmentation strategies such as rotation, scaling, and cropping to expand the original sample size and create an augmented dataset.

[0046] S404: Model Training. The expanded dataset is divided into training, validation, and test sets proportionally and input into the PIN-YOLO network. The dynamic learning rate adaptive strategy built into the YOLO framework is used for optimization. After setting the training hyperparameters, the network parameters are adjusted in real time using the validation set. When the upper limit of the number of training rounds is reached, training is stopped, and the trained PIN-YOLO pin detection model is obtained.

[0047] S500: Pin Detection and Coordinate Extraction. The standardized chip body region image obtained in step S205 is input into the trained PIN-YOLO network to perform pin detection and coordinate extraction. The specific process is as follows: S501: Model Inference. The image of the main chip area is input into the PIN-YOLO network. After processing by the backbone network, PSDWA module, and feature pyramid network (Neck), the detection head outputs candidate pin boxes. The confidence threshold of the candidate boxes is set to a preset value. Candidate boxes with confidence values ​​lower than the preset threshold are filtered out, false detection boxes are removed, and candidate boxes corresponding to the real pins are retained. S502: Coordinate Transformation. Calculate the center point coordinates of each filtered candidate box, use the center point coordinates as the pixel position of the corresponding pin, and organize the center point coordinates of all pins into a structured pin coordinate list; S600: Method terminated. Output the coordinate pin detection results to the specified storage location or directly interface with an EDA tool to complete the automated extraction of pin coordinates.

[0048] In some dependent embodiments of the present invention, the original image in S100 can be directly from a screenshot or PDF exported from a BGA device datasheet, or it can be from a scanned paper datasheet image. For scanned images, image denoising and tilt correction processing can be added before step S201 to eliminate noise and geometric distortion generated during scanning, and further improve the accuracy of subsequent region extraction and pin detection.

[0049] In a further preferred embodiment of the present invention, steps S200 to S500 of the image inference application stage can be automated and integrated end-to-end without manual intervention; the model training stage can be completed by staff to configure parameters and control the training process, and the optimal model parameters can be fixed after training. In practical applications, only the original image needs to be input, and the system can automatically complete preprocessing, chip main area extraction, pin detection, and coordinate output. The extracted structured pin coordinates can be directly imported into EDA tools such as PCB layout and routing, and package verification, achieving seamless connection from drawing analysis to engineering application. Furthermore, a running log can be recorded in each step of the automated execution process. The log includes at least the input image information, preprocessing parameters, outline extraction results, number of detected pins, coordinate extraction results, and anomaly information to support subsequent debugging, effect verification, and continuous algorithm optimization.

[0050] In possible embodiments of the present invention, various thresholds in the preprocessing, outline extraction and pin detection processes can be adjusted according to actual image features, manufacturer drawing specifications or extraction accuracy requirements, and do not constitute a limitation on the scope of protection of the present invention.

[0051] In a possible embodiment of the present invention, the multi-level fault tolerance mechanism in S200 can be further optimized: an outer frame shape recognition module is added to classify the extracted outer frame candidates into shapes (regular rectangles, chamfered rectangles, dashed frames, etc.), and the projection analysis parameters and clipping strategies are adjusted according to different shape types to further improve the robustness of chip main area extraction and adapt to more heterogeneous drawing scenarios.

[0052] In a possible embodiment of the present invention, the PIN-YOLO network in S300 can be further optimized: a multi-scale feature fusion module is introduced into the feature pyramid network (Neck) to enhance the detection capability of pins of different sizes; a bounding box refinement algorithm is introduced into the detection head to improve the positioning accuracy of the pin candidate box, thereby improving the extraction accuracy of the pin center point coordinates.

[0053] In a possible embodiment of the present invention, the data augmentation strategy in S403 can be further extended to add operations such as noise addition, blurring, and contrast adjustment to simulate the degradation phenomenon that occurs during image compression, distribution, and scanning, thereby improving the generalization ability of the PIN-YOLO network and enabling it to adapt to more BGA package drawings with large quality differences.

[0054] This invention is not limited to pin coordinate extraction from BGA package drawings. It can be applied to any scenario involving small, dense targets, background interference, highly heterogeneous drawings, and the need to extract target coordinates from 2D engineering drawings. This includes, but is not limited to, pin coordinate extraction for other packaged devices such as QFN and SOP, as well as coordinate analysis tasks for other small targets.

[0055] Compared with existing technologies, the present invention has the following advantages and beneficial effects in the method for extracting pin coordinates from BGA package drawings: First, this invention adopts a heterogeneous collaborative architecture of "chip region adaptive extraction + PIN-YOLO detection", which is different from single traditional image processing or deep learning methods. It designs processing schemes for different morphological characteristics of chip outline and pins, making the extraction decision more targeted and effectively solving the problem of poor adaptability of existing technologies to heterogeneous drawings.

[0056] Secondly, this invention, through dual-track projection and multi-level fault-tolerant mechanisms, can adaptively adapt to various heterogeneous scenarios such as complete outer frame, dashed outer frame, unclosed outer frame and no outer frame, and can complete the extraction of the main chip area without manual intervention. At the same time, it optimizes the removal of background interference by anti-black edge clipping, providing high-quality input for pin detection and significantly reducing the probability of missed detection and false detection.

[0057] Furthermore, this invention embeds a PSDWA dual-path parallel module into the YOLOv8 baseline network. This module preserves the underlying geometric details of the pins through the identity path and amplifies the key feature responses through the enhancement path. This solves the shortcomings of existing deep learning models in terms of insufficient accuracy and loss of details in detecting small and dense pins, and improves the accuracy of pin localization and coordinate extraction.

[0058] Furthermore, in the preferred embodiment, the present invention achieves end-to-end automated integration, provides standardized input and output interfaces, and can directly connect to EDA tools without the need for manual annotation or parameter adjustment, thereby reducing the cost of engineering applications. At the same time, through operation log recording, it is convenient to track the extraction process, debug and optimize methods, and improve the feasibility and verifiability of the project.

[0059] Furthermore, in the dependent embodiments, the generalization ability of the method can be further improved by means of dataset enhancement and expansion, network structure optimization, and threshold adaptive tuning, so as to adapt to BGA package drawings of different quality and different manufacturers. At the same time, it can be extended to the extraction of pin coordinates of other packaged devices, thus broadening the scope of application of the method.

[0060] Furthermore, the present invention also provides a computer device / apparatus / system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0061] The present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0063] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more blocks in a block diagram.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more flowcharts and / or one or more blocks in a block diagram.

[0065] It should be noted that, depending on the implementation needs, the various steps / components described in this invention can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0066] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for adaptive extraction of pin coordinates from BGA package drawings based on a pin detection YOLO network, characterized in that, The method includes: Obtain the original image of the bottom view of the BGA package; Adaptive chip region extraction is performed on the original image to obtain the chip main region image. This includes: preprocessing the original image by removing watermarks and filtering interference, extracting the outer contour based on the preprocessed image, and using a dual-track projection method combined with a multi-level fault tolerance mechanism to extract the chip outer frame to obtain the chip main region image. A PSDWA module is embedded at the end of the backbone network of the YOLO baseline network to construct a PIN-YOLO pin detection network; The image of the main area of ​​the chip is input into a pre-trained PIN-YOLO pin detection network to perform pin detection and coordinate extraction.

2. The adaptive extraction method for pin coordinates of BGA package drawings based on pin detection YOLO network according to claim 1, characterized in that, Preprocessing of the original image, including watermark removal and interference filtering, includes: Watermark Removal: Removes the watermark from the original image to obtain a watermark-free image; External interference filtering: Convert the watermark-removed image to a grayscale image, then perform inverse binarization to convert the grayscale image to a black and white binary image. Use a connected component analysis algorithm to remove small connected components with an area smaller than a preset threshold in the black and white binary image. Perform a bitwise AND operation between the mask after removing the small connected components and the watermark-removed image to complete the initial cropping and obtain the external interference-removed image. Internal interference filtering: Gaussian low-pass filtering is applied to the image to eliminate external interference, and global binarization is performed using the adaptive Otsu method. The optimal threshold is determined by maximizing the inter-class variance. After global binarization, morphological closing operation is performed to complete the internal interference filtering.

3. The adaptive extraction method for pin coordinates of BGA package drawings based on a pin detection YOLO network according to claim 1, characterized in that, Based on the preprocessed image, the outer contour is extracted, and the chip outline is extracted using a dual-track projection method combined with a multi-level fault-tolerant mechanism to obtain the chip main body region image, including: All closed boundaries are extracted from the preprocessed image. Based on the prior knowledge that the chip outline is the largest closed region in the image, candidate outer contours of the chip body are selected. If the candidate outer contour meets the preset area constraint, the first outline of the candidate outer contour is extracted based on the projection method of the outer contour to obtain the chip main body region image. If no closed boundary is extracted, or the candidate outer contour does not meet the preset area constraint conditions, or the first-track outer frame extraction fails, the second-track outer frame extraction is performed on the preprocessed image based on the projection method of the original image to obtain the chip main body region image.

4. The adaptive extraction method for pin coordinates of BGA package drawings based on pin detection YOLO network according to claim 3, characterized in that, Extracting the first outline, including: Morphological filtering optimization is performed on the image of the candidate region for the outer contour. The row and column projection features of the image after morphological filtering optimization are calculated. The horizontal and vertical projections are calculated separately, and the projection results reflect the pixel distribution features of the contour region in the horizontal and vertical directions. A dynamic threshold is set based on the projection peak value, and a spatial distance clustering algorithm is used to remove duplicate boundaries to obtain candidate outer frame boundaries; The vertical boundary is selected as the left and right boundary based on the peak value of the column integral projection. If the optimal vertical boundary spacing is less than the preset width threshold, the extreme value mode is switched to select the outermost edge line as the left and right boundary. Horizontal projection is performed within the left and right boundary constraints, and the outermost horizontal line is selected as the upper and lower boundaries according to the principle of maximum inclusiveness. Based on the determined upper and lower boundaries and left and right boundaries, the chip main body area image is cropped from the watermark-removed image.

5. The adaptive extraction method for pin coordinates of BGA package drawings based on pin detection YOLO network according to claim 3 or 4, characterized in that, The image of the chip's main body region obtained from the first outline extraction is optimized for anti-black border cropping, including: By combining local window grayscale features with global connected region area information, explicit black borders or implicit dark areas in images can be identified and removed. Scan line by line or column by column from the image edge inwards to find the first pixel that exceeds the effective content threshold as the new cropping boundary, and perform adaptive boundary cropping. One-dimensional projection is performed based on a center avoidance strategy. The cropping result is optimized by utilizing the sharp drop in the peak value of the chip boundary projection, resulting in the final image of the chip's main body region.

6. The adaptive extraction method for pin coordinates of BGA package drawings based on pin detection YOLO network according to claim 3, characterized in that, Extracting the second outline, including: Set a black tolerance threshold and convert the preprocessed image into a dark feature binary map; The dark feature binary image is projected horizontally and vertically. A dynamic threshold is set based on the projection peak value. The spatial distance clustering algorithm is used to remove duplicate boundaries to obtain candidate bounding boxes. Define a vertical center avoidance zone, and select the straight line closest to the edge in the upper and lower subdomains as the upper and lower boundaries respectively; Extract a set of candidate vertical lines from the binary image of dark features, and perform multi-criteria verification on the candidate vertical lines: when there are two vertical lines, verify whether the minimum width constraint is met; when there are three or more vertical lines, introduce a symmetry axis consistency detection, and take the two outer lines that meet the preset pixel-level tolerance as the left and right boundaries. If the multi-criteria verification fails to output a valid boundary, a two-stage fault-tolerant verification is initiated: the local vertical projection is re-executed within the strip of the determined upper and lower boundary constraints, and the vertical boundary is determined in conjunction with the horizontal center avoidance logic; if it is still ineffective, the center avoidance logic constraint is removed, and the second track outline extraction is re-executed.

7. The adaptive extraction method for pin coordinates of BGA package drawings based on a pin detection YOLO network according to claim 1, characterized in that, The PIN-YOLO pin detection network includes a backbone network, a PSDWA module, a feature pyramid network, and a detection head; The PSDWA module employs a dual-path parallel architecture, including an identity path and an enhanced path: The input feature map is split into two parallel sub-feature maps along the channel dimension: the identity path sub-feature map and the enhanced path sub-feature map; The identity path does not perform any transformation on the identity path sub-feature map and directly passes it on to the next step; The enhancement path processes the sub-feature maps sequentially through a deep convolutional weighted attention submodule and a feedforward network: the deep convolutional weighted attention submodule extracts the spatial structure of the features through depthwise separable convolutions and learns channel interaction through global average pooling and a fully connected network; the feedforward network uses a lightweight structure to enhance the features. The feature maps of the identity path and the enhancement path are fused to output the final enhanced feature map.

8. The adaptive extraction method for pin coordinates of BGA package drawings based on pin detection YOLO network according to claim 1, characterized in that, Perform pin detection and coordinate extraction, including: Input the image of the main chip area into the PIN-YOLO pin detection network to output pin candidate boxes; Filter the candidate pin boxes according to a preset confidence threshold; Calculate the center point coordinates of each retained pin candidate box as the pixel position of the corresponding pin, and organize the center point coordinates of all pins into a structured pin coordinate list; The training steps for the PIN-YOLO pin detection network include: Collect bottom view images of different BGA packages as raw samples; For images with valid outlines, the chip body area image is obtained through adaptive extraction of the chip region, and the pin positions are manually marked; for images without outlines or with invalid outlines, the pin positions are manually marked directly on the original image. Data augmentation strategies are employed to expand labeled samples and construct a pin recognition dataset; wherein, the data augmentation strategies include one or more of rotation, scaling, cropping, noise addition, blurring, and contrast adjustment; The pin identification dataset is input into the constructed PIN-YOLO pin detection network, and a dynamic learning rate adaptive strategy is used for training to obtain a pre-trained PIN-YOLO pin detection network.

9. A computer device / equipment / system, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

Images