Multi-bom batch labeling method and system for PCB board

Abstract

This invention discloses a method and system for batch annotation of multiple Bill of Materials (BOMs) on PCB boards. The method includes: acquiring a template PCB image and inputting it into a pre-trained component detection model; acquiring a PCB image to be annotated, and when the PCB image to be annotated consists of multiple local sub-images, stitching the multiple local sub-images together; extracting feature points and feature descriptors from both the template PCB image and the PCB image to be annotated; performing feature matching based on the feature descriptors, and filtering the matching results by combining the component categories and component positional relationships in the template BOM annotation data; calculating the transformation matrix between the template PCB image and the PCB image to be annotated; transforming the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and mapping the transformed annotation boxes to the PCB image to be annotated; and removing duplicate annotation boxes mapped to overlapping areas according to component categories and intersection-union ratio (IU). This invention balances annotation completeness, accuracy, and processing efficiency during batch annotation.

Description

Technical Field

[0001] This invention relates to the field of batch annotation technology for PCB boards, and in particular to a method, system, computer equipment, and storage medium for batch annotation of multiple BOMs on PCB boards. Background Technology

[0002] In the manufacturing process of printed circuit boards (PCBs), it is usually necessary to label the various components on the board with Bill of Materials (BOM) to facilitate subsequent production verification, inspection and analysis, and assembly management. Current PCB BOM labeling methods mostly employ manual labeling or automatic matching based on fixed templates. Manual labeling suffers from low efficiency, error-proneness, and poor consistency. Automatic labeling based on fixed templates often relies on pre-configured board templates, making it less adaptable to different layouts, sizes, or batches of PCBs. When the board type changes, it is usually necessary to recreate or adjust the template, which is insufficient to meet the efficiency and flexibility requirements of batch processing scenarios.

[0003] Furthermore, for larger PCBs, the limited field of view and imaging conditions of industrial cameras often make it difficult to acquire a complete high-resolution image at once. In practical applications, it is usually necessary to acquire multiple partial images and then stitch them together. During this process, relying solely on conventional image matching or simple annotation mapping methods can easily lead to annotation position shifts, duplicate annotations, or omissions due to differences in board type, viewing angle deviations, or repetition of local areas. This further reduces the stability and accuracy of batch annotation. Therefore, there is a common overall deficiency: when dealing with PCBs of different types and sizes, it is difficult to achieve efficient and stable batch BOM annotation while ensuring the completeness and accuracy of the annotations. Summary of the Invention

[0004] The purpose of this application is to propose a method, system, computer equipment, and storage medium for batch annotation of multiple BOMs on PCB boards, in order to solve the technical problem of how to accurately convert template BOM annotations into PCB images to be annotated in application scenarios with different board types and large-size PCB boards, while taking into account annotation completeness, accuracy, and processing efficiency during batch annotation.

[0005] To address the aforementioned technical problems, this application provides a method for batch annotation of multiple BOMs on PCB boards, employing the following technical solution: A template PCB image is obtained and input into a pre-trained component detection model to generate template BOM annotation data. The template BOM annotation data includes at least component category, component location, and annotation box coordinates. A PCB image to be labeled is obtained. When the PCB image to be labeled is composed of multiple local sub-images, the multiple local sub-images are stitched together to obtain a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to be labeled. Extract feature points and feature descriptors from the template PCB image and the PCB image to be labeled, respectively; Feature matching is performed based on the feature descriptors, and the matching results are filtered by combining the component categories and component positional relationships in the template BOM annotation data to obtain matching point pairs; Calculate the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pairs; The coordinates of the annotation boxes in the template BOM annotation data are transformed according to the transformation matrix, and the transformed annotation boxes are mapped to the PCB image to be annotated to obtain the target BOM annotation data corresponding to the PCB image to be annotated; When the PCB image to be labeled is obtained by stitching together multiple local sub-images, the label boxes mapped to the overlapping areas are deduplicated according to component category and intersection-union ratio.

[0006] To address the aforementioned technical problems, this application also provides a multi-BOM batch annotation system for PCB boards, employing the following technical solution: The first acquisition module is used to acquire a template PCB image and input the template PCB image into a pre-trained component detection model to generate template BOM annotation data. The template BOM annotation data includes at least component category, component position and annotation box coordinates. The second acquisition module is used to acquire the PCB image to be labeled. When the PCB image to be labeled is composed of multiple local sub-images, the multiple local sub-images are stitched together to obtain a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to be labeled. The extraction module is used to extract feature points and feature descriptors from the template PCB image and the PCB image to be labeled, respectively. The filtering module is used to perform feature matching based on the feature descriptor and to filter the matching results by combining the component category and component position relationship in the template BOM annotation data, so as to obtain matching point pairs; The calculation module is used to calculate the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pairs; The transformation module is used to transform the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and map the transformed annotation boxes to the PCB image to be annotated, so as to obtain the target BOM annotation data corresponding to the PCB image to be annotated; The deduplication module is used to deduplicate the annotation boxes mapped to the overlapping areas according to the component category and intersection-union ratio when the PCB image to be annotated is obtained by stitching together multiple local sub-images.

[0007] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution: A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the multi-BOM batch annotation method for PCB boards as described above.

[0008] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below: A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the multi-BOM batch annotation method for PCB boards as described above.

[0009] Compared with the prior art, the embodiments of this application have the following main advantages: The multi-BOM batch annotation method for PCB boards disclosed in this application generates template BOM annotation data through template PCB images, stitches together large-size PCB images to be annotated, extracts feature points and feature descriptors from the template and the images to be annotated, filters matching point pairs based on component categories and component positional relationships, calculates the transformation matrix between images, performs coordinate transformation of annotation boxes, and removes duplicate annotation boxes in overlapping areas. This enables the component annotations on the template to be accurately transferred to the PCB images to be annotated. It is applicable to both ordinary PCB boards that can be imaged in a single step and large-size PCB boards that require stitching together multiple local sub-images, thereby achieving efficient and accurate batch BOM annotation for PCB boards of different types and sizes. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1This is a flowchart of an embodiment of the multi-BOM batch annotation method for PCB boards according to this application; Figure 2 This is a schematic diagram of a structure of an embodiment of the multi-BOM batch annotation system for PCB boards according to this application; Figure 3 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] refer to Figure 1 A flowchart illustrating an embodiment of the multi-BOM batch annotation method for PCB boards according to this application is shown. The multi-BOM batch annotation method for PCB boards includes the following steps: Step S101: Obtain a template PCB image and input the template PCB image into a pre-trained component detection model to generate template BOM annotation data. The template BOM annotation data includes at least component category, component location, and annotation box coordinates.

[0014] In one specific embodiment, a template PCB image is first acquired and input into a pre-trained component detection model to generate template BOM annotation data. Here, the template PCB image can be understood as a pre-selected standard image for a specific type of PCB board, used to carry reference information for subsequent annotation conversion. The component detection model can be a deep learning target detection model trained on multiple PCB types, such as a detection network based on the YOLO architecture. Its function is to automatically identify component targets such as resistors, capacitors, integrated circuits, and connectors from the template PCB image and output template BOM annotation data corresponding to each component target. The template BOM annotation data includes at least component category, component location, and bounding box coordinates. The component category characterizes the type of the corresponding device, the component location characterizes the spatial distribution of the device in the template PCB image, and the bounding box coordinates define the boundary range of the device in the image. For example, for a template PCB image, the component detection model can identify a resistor located in the upper left region, a capacitor located in the center region, and a chip located in the lower right region, and generate corresponding category information, location parameters, and rectangular bounding boxes, thereby forming template BOM annotation data available for subsequent conversion. This processing method avoids the need for manual selection and input of template images one by one, improves the efficiency of template construction, and provides a unified data foundation for the batch annotation of different PCB images to be annotated.

[0015] Step S102: Obtain the PCB image to be labeled. When the PCB image to be labeled is composed of multiple local sub-images, stitch the multiple local sub-images together to obtain a panoramic image of the PCB to be labeled, and use the panoramic image of the PCB to be labeled as the PCB image to be labeled.

[0016] After obtaining the template BOM annotation data, the PCB image to be annotated is acquired. When the PCB image to be annotated consists of multiple local sub-images, these local sub-images are stitched together to obtain a panoramic image of the PCB to be annotated. This panoramic image is then used as the PCB image to be annotated in subsequent processing. Here, the PCB image to be annotated refers to the target image for which BOM annotation needs to be performed. For small PCBs that can be captured completely in one go, the PCB image to be annotated can be a single complete image. For large PCBs that cannot be imaged in one go due to the limitations of the field of view of industrial cameras, multiple local sub-images can be acquired relative to the PCB by moving a platform or using a camera to capture different areas of the PCB. A local sub-image is a high-resolution image covering a local area of ​​the PCB. It is preferable to retain a certain overlap between adjacent local sub-images, such as 15% to 25%, to facilitate the extraction of common features and stable stitching. The panoramic image of the PCB to be annotated obtained after image stitching is essentially a complete board image formed by mapping multiple local sub-images to a unified coordinate system, thus enabling subsequent annotation transformations to be performed uniformly on the entire large-size PCB. For example, for a long server motherboard, multiple local sub-images can be continuously captured along the length of the board, and then the sub-images can be merged into a panoramic image based on the common component textures and pad structures of adjacent areas, so as to avoid the loss of subsequent annotations due to incomplete single shooting.

[0017] Step S103: Extract feature points and feature descriptors from the template PCB image and the PCB image to be labeled, respectively.

[0018] After determining the template PCB image and the PCB image to be labeled, feature points and feature descriptors are extracted from both images. Here, feature points can be understood as local locations in the image with strong discriminative power and stability, such as component edge corners, abrupt changes in pad contours, or salient points of silkscreen characters. Feature descriptors are feature vectors encoded with the neighborhood texture information of feature points, used to measure whether local regions in different images correspond to the same physical location. In a preferred embodiment, the AKAZE algorithm can be used to extract feature points and binary feature descriptors. The AKAZE algorithm is an algorithm for extracting local features in a nonlinear scale space, balancing computational efficiency with adaptability to rotation, scale, and certain lighting changes. Therefore, it is well-suited for image matching tasks with small components and dense textures in PCB scenarios. For example, in the template PCB image and the PCB image to be labeled, the same chip pin area, pad array area, or silkscreen character area can usually form a relatively stable local texture structure. The AKAZE algorithm can extract corresponding feature point sets from these areas, providing a basis for subsequent image alignment.

[0019] Step S104: Perform feature matching based on the feature descriptor, and filter the matching results by combining the component categories and component positional relationships in the template BOM annotation data to obtain matching point pairs.

[0020] After extracting feature points and feature descriptors from two images, feature matching is performed based on the feature descriptors. The matching results are then filtered using the component categories and positional relationships in the template BOM annotation data to obtain matching point pairs. Feature matching refers to comparing the similarity between feature descriptors in the template PCB image and those in the PCB image to be annotated, to determine feature point combinations that may correspond to the same physical region. In one embodiment, Hamming distance can be used as a similarity metric for fast matching of binary feature descriptors. Hamming distance reflects the number of different bits between two binary codes; the smaller the distance, the more similar the two descriptors are. Since relying solely on descriptor similarity may lead to false matches, this application further combines component categories and positional relationships in the template BOM annotation data for filtering. The component positional relationship can be understood as the relative spatial layout of multiple components in the template PCB image, such as a capacitor located to the left of a specific chip, a resistor located above a connector, or multiple devices arranged in an array along the same direction. By overlaying category consistency and relative positional constraints on descriptor matching, mismatched points that are similar only in local texture but inconsistent in overall component layout can be eliminated, resulting in more reliable matching point pairs. For example, if a feature point in the template image is located in the resistor region, while the corresponding candidate feature point in the image to be labeled is located in the chip region, even if their descriptors are similar, they can be excluded based on the inconsistency in component categories. Similarly, if multiple matching points fall into the same category of components, but their distance and orientation deviate significantly from the original layout in the template, they can also be considered abnormal matches and eliminated. The resulting matching point pairs not only preserve local texture similarity but also maintain the consistency of the PCB board component layout, which is more conducive to the stable solution of the subsequent transformation matrix.

[0021] Step S105: Calculate the transformation matrix between the template PCB image and the PCB image to be annotated based on the matching point pairs.

[0022] After obtaining the matching point pairs, the transformation matrix between the template PCB image and the PCB image to be labeled is calculated based on the matching point pairs. The transformation matrix describes the spatial mapping relationship between the two images, enabling the position coordinates in the template PCB image to be transformed to the corresponding positions in the PCB image to be labeled; in a specific embodiment, this transformation matrix can be a homography matrix. The homography matrix is ​​suitable for describing the mapping relationship of planar targets under different shooting angles, scales, or slight perspective changes, which is more in line with the application scenario where the PCB board surface is basically on the same plane. To improve the solution accuracy, the Random Sampling Consensus Algorithm (RANSAC) can be used to remove outliers and mismatched points from the initial matching point pairs, and then estimate the homography matrix based on the retained valid matching point pairs. The role of RANSAC is to construct candidate models through multiple random samplings and select the transformation result that can obtain the most interior point support, thereby avoiding a small number of erroneous matches from causing significant interference to the overall transformation solution. For example, when the PCB image to be labeled has slight rotation, scaling or shooting angle deviation relative to the template PCB image, the matching point pairs after RANSAC filtering can still support the calculation of a relatively stable transformation matrix, so that the overall distribution relationship of the components in the template image is mapped to the image to be labeled.

[0023] Step S106: Transform the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and map the transformed annotation boxes to the PCB image to be annotated, so as to obtain the target BOM annotation data corresponding to the PCB image to be annotated.

[0024] After obtaining the transformation matrix, the coordinates of the bounding boxes in the template BOM annotation data are transformed according to the transformation matrix, and the transformed bounding boxes are mapped to the PCB image to be annotated, so as to obtain the target BOM annotation data corresponding to the PCB image to be annotated. The bounding box coordinate transformation can be understood as performing a geometric transformation on the vertex coordinates or boundary coordinates of the bounding boxes of each component in the template PCB image according to the transformation matrix, so that they form the target bounding boxes at the corresponding positions in the PCB image to be annotated. After the transformation, the bounding boxes that originally identified a resistor, capacitor or chip in the template PCB image can be projected onto the corresponding device area in the PCB image to be annotated, thereby obtaining the target BOM annotation data containing the component category, component position and target bounding boxes. For example, a capacitor bounding box located in the central area of ​​the template PCB image may show a positional shift and slight perspective distortion in the PCB image to be annotated due to changes in the shooting angle. After the coordinate transformation of the bounding box by the transformation matrix, it can still be accurately mapped to the actual capacitor position in the image to be annotated, realizing the automatic migration of template annotations to the target image. By repeatedly performing the above process of feature extraction, matching and filtering, transformation matrix calculation and annotation box mapping, multiple PCB images of the same or similar board types can be automatically annotated in batches, thereby significantly reducing the workload of manual annotation and improving annotation efficiency and consistency.

[0025] Step S107: When the PCB image to be annotated is obtained by stitching together multiple local sub-images, the annotation boxes mapped to the overlapping area are deduplicated according to the component category and intersection-union ratio.

[0026] When a PCB image to be labeled is obtained by stitching together multiple local sub-images, it is also necessary to deduplicate the label boxes mapped to the overlapping area according to component category and intersection-union ratio (IU). The overlapping area refers to the common area of ​​the same board surface corresponding to two or more local sub-images during the stitching process; the IU is the ratio between the overlapping area of ​​two label boxes and their union area, used to measure the degree of spatial overlap between the two label boxes. Since the same physical component may appear simultaneously in adjacent local sub-images, duplicate labels are easily formed in the overlapping area of ​​the panoramic image after each sub-image participates in stitching and label mapping, thus requiring deduplication. Specifically, it can be first determined whether the component categories of two or more label boxes in the overlapping area are consistent, and only the label boxes with consistent categories are further calculated with the IU; when the IU exceeds a preset threshold, such as 0.5, these label boxes can be identified as corresponding to the same physical component, and the process of merging or retaining a unique label is performed, thereby avoiding the same device being repeatedly included in the BOM labeling results. For example, if a connector is located at the boundary of two adjacent local sub-images, each sub-image will have a connector bounding box after mapping. If the two bounding boxes are of the same category and their intersection-union ratio (IU) reaches a preset threshold, they can be merged into a single final bounding box. Conversely, if the IU is low or the bounding box is of different categories, it indicates that the connector may correspond to different devices, and it should not be simply deleted to avoid omissions. This deduplication mechanism ensures that the annotation results of large-size PCB boards remain complete and unique even after being stitched together into a panoramic image.

[0027] This application generates template BOM annotation data from template PCB images, stitches together large-size PCB images to be annotated, extracts feature points and feature descriptors from the template and the images to be annotated, filters matching point pairs based on component categories and component positional relationships, calculates the transformation matrix between images, performs coordinate transformation of annotation boxes, and removes duplicate annotation boxes in overlapping areas. This enables component annotations on the template to be accurately transferred to the PCB images to be annotated. It is applicable to both ordinary PCBs that can be imaged in a single step and large-size PCBs that require stitching together multiple local sub-images, thus achieving efficient and accurate batch BOM annotation for PCBs of different board types and sizes.

[0028] In some optional implementations of this embodiment, the above-mentioned inputting the template PCB image into a pre-trained component detection model to generate template BOM annotation data includes: The standard template PCB image is input into the component detection model trained based on multi-board type PCB samples to detect each component target in the standard template PCB image; Based on the detection results, template BOM annotation data corresponding to each component target is generated. The template BOM annotation data includes at least the component category, component position, and annotation box coordinates.

[0029] In this embodiment, a sample set of PCB images containing various board types and component types is pre-collected, and a deep learning object detection model is trained based on this sample set. The deep learning object detection model can employ a detection network based on the YOLOv8 architecture, combined with transfer learning and data augmentation to improve its adaptability to different PCB board types. Data augmentation can include rotation, scaling, brightness perturbation, and local cropping to improve the model's robustness to changes in actual shooting conditions. After training, a standard template PCB image is selected as input for a specific target board type. The component detection model outputs the detection results for each component target, and then generates template BOM annotation data based on the detection results. The template BOM annotation data includes at least the component category, component location, and bounding box coordinates. For example, for a server motherboard template image, the detection model can identify resistors, capacitors, integrated circuits, and connectors, and automatically output the category label, center position or region position, and bounding box coordinates for each component, thereby forming the template BOM annotation data required for subsequent annotation conversion. In this way, the template annotation foundation can be quickly established without manually selecting and inputting each template image.

[0030] This application trains a component detection model with generalization capabilities on PCB samples of multiple board types, which can reduce the cost of repeatedly creating templates for different board types and can automatically generate template BOM annotation data containing component category, component location and annotation box coordinates, thereby improving template construction efficiency and providing a unified and reliable data foundation for subsequent batch annotation.

[0031] In some optional implementations of this embodiment, the above-mentioned feature matching based on the feature descriptor, and the filtering of the matching results in combination with the component categories and component positional relationships in the template BOM annotation data, to obtain matching point pairs, includes: AKAZE feature points and binary feature descriptors are extracted from the template PCB image and the PCB image to be labeled, respectively, and candidate matching point pairs are obtained based on Hamming distance; The candidate matching point pairs are filtered according to the consistency of their component categories and the relative positional relationship of the corresponding components to obtain matching point pairs used to calculate the transformation matrix.

[0032] In this embodiment, AKAZE feature extraction is performed on the template PCB image and the PCB image to be labeled to obtain their respective feature points and binary feature descriptors. The AKAZE algorithm is an algorithm for extracting local salient features in a nonlinear scale space, which is more suitable for the detection and representation of fine-grained structures such as pad edges, silkscreen characters, and component outlines in PCB images. Subsequently, Hamming distance is used to perform similarity matching on the binary feature descriptors of the two images to obtain candidate matching point pairs. In order to reduce the false matching caused by relying solely on local texture similarity, the candidate matching point pairs are further filtered by combining the component categories and component positional relationships in the template BOM annotation data. The component positional relationship can be the relative distance relationship, orientation relationship, or array distribution relationship between two components. For example, when a feature point in the template image corresponds to a capacitor area, and the matching candidate point in the image to be labeled falls into a chip area, even if there is a similarity in local texture, it can be eliminated based on the inconsistency of component categories; or, if multiple candidate matching points are located in the same type of device, but their spatial layout deviates significantly from the original layout relationship in the template, they can also be regarded as abnormal matches and excluded. After the above screening, the remaining matching point pairs simultaneously meet the requirements of local feature similarity and device layout consistency, making them more suitable as input for subsequent transformation matrix solving.

[0033] This application combines AKAZE feature matching with component category constraints and component positional relationship constraints. This not only retains the advantages of AKAZE algorithm, such as high computational efficiency and adaptability to scale and rotation changes, but also significantly reduces mismatched point pairs and improves the reliability of matching results, thus laying the foundation for accurate solution of the subsequent transformation matrix.

[0034] In some optional implementations of this embodiment, the above-mentioned calculation of the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pairs includes: The random sampling consensus algorithm is used to remove mismatched point pairs from the matched point pairs; The homography matrix between the template PCB image and the PCB image to be labeled is estimated based on the valid matching point pairs after removing mismatched point pairs, and is used as the transformation matrix.

[0035] In this embodiment, the Random Sampling Consensus Algorithm (RANSAC) is first used to estimate interior points and remove outliers from the selected matching point pairs. Specifically, several sets of samples can be randomly selected from all matching point pairs to construct candidate models, and the number of interior points that meet the error threshold under each candidate model is counted. The model with the most interior point support is selected as the optimal geometric relationship. Based on this, the homography matrix between the template PCB image and the PCB image to be labeled is estimated based on the retained valid matching point pairs, which serves as the transformation matrix. The homography matrix is ​​used to characterize the perspective transformation relationship between two planar images and can describe rotation, translation, scaling, and a certain degree of perspective distortion. For example, even if the PCB image to be labeled is slightly tilted or offset relative to the template PCB image, after RANSAC removes mismatched point pairs, a stable homography matrix can still be obtained based on the valid matching point pairs, thereby mapping the device positions in the template image to the image to be labeled.

[0036] This application reduces the interference of abnormal matching on the geometric solution results by first using RANSAC to remove erroneous point pairs and then using valid matching point pairs to estimate the transformation matrix. This improves the stability and accuracy of the spatial mapping relationship between the template PCB image and the PCB image to be annotated, thereby enhancing the accuracy of subsequent annotation and conversion.

[0037] In some optional implementations of this embodiment, the above-mentioned transformation of the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and mapping the transformed annotation boxes to the PCB image to be annotated, to obtain the target BOM annotation data corresponding to the PCB image to be annotated, includes: Perform perspective transformation on the vertex coordinates of each annotation box to obtain the corresponding vertex coordinates in the PCB image to be annotated; The target bounding box is generated based on the transformed corresponding vertex coordinates, and the target BOM annotation data is generated based on the component category and component position of the target bounding box.

[0038] In this embodiment, the vertex coordinates or boundary coordinates of each annotation box in the template BOM annotation data are first read, and the obtained transformation matrix is ​​used to perform perspective transformation on the vertex coordinates to obtain their corresponding vertex coordinates in the PCB image to be annotated. Then, target annotation boxes are generated based on the transformed corresponding vertex coordinates, and the target annotation boxes are associated with the component category and component position in the original template BOM annotation data to form the target BOM annotation data corresponding to the PCB image to be annotated. For example, a capacitor located in the central area of ​​the template image corresponds to a rectangular bounding box. After homography matrix perspective transformation, this rectangular box may appear as a target box with a slight positional shift and shape distortion in the image to be annotated. By transforming its vertex coordinates, the target annotation box of the device can be accurately recovered in the image to be annotated, while retaining its capacitor category and corresponding position information. By repeatedly performing the above transformation and mapping on each component annotation box in the template, the target BOM annotation data of the PCB image to be annotated can be generated in batches.

[0039] This application performs geometric transformation on the coordinates of the annotation boxes in the template BOM annotation data and maps them to the PCB image to be annotated. This can directly complete the automatic migration of template annotations to the target image, avoid manual repetitive annotation, and improve the efficiency, consistency and accuracy of BOM annotation conversion between different PCB images.

[0040] In some optional implementations of this embodiment, the above-mentioned image stitching of multiple local sub-images to obtain a panoramic image of the PCB to be annotated includes: The image acquisition device is controlled to acquire multiple local sub-images, and the adjacent local sub-images have an overlap area of ​​15% to 25%. Multiple local sub-images are stitched together to form a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to perform feature extraction, feature matching, transformation matrix calculation and label box mapping.

[0041] In this embodiment, when the size of the PCB board to be labeled exceeds the single effective field of view of the industrial camera, the moving platform or the relative displacement between the camera and the PCB board is controlled to capture images of the target PCB board in sections, thereby obtaining multiple local sub-images. To ensure that the subsequent stitching has sufficient common feature areas, an overlap of 15% to 25% is preferably set between adjacent local sub-images. After the acquisition is completed, the multiple local sub-images are sequentially input into the stitching process. The geometric correspondence between adjacent sub-images is determined by the common features between them, and the local sub-images are stitched together to form a panoramic image of the PCB to be labeled. After obtaining the panoramic image, it is used as the PCB image to be labeled, and subsequent feature extraction, feature matching, transformation matrix calculation, and annotation box mapping processing are performed. For example, for a long communication motherboard, several high-resolution local sub-images can be continuously acquired along the length of the board, with an overlap of about 20% between adjacent images, and then the local sub-images are stitched together to form a complete board image, so as to ensure that all components on the large-sized PCB board are included in the subsequent labeling range.

[0042] This application overcomes the limitations of industrial camera field of view by acquiring and stitching multiple local sub-images of large-size PCBs that cannot be captured in a single complete image. This allows for obtaining a complete panoramic image of the PCB to be annotated, avoiding annotation loss due to incomplete single imaging, and providing a complete image basis for batch annotation of large-size PCBs.

[0043] In some optional implementations of this embodiment, the above-mentioned stitching of multiple local sub-images into a panoramic image of the PCB to be labeled, and the deduplication of the label boxes mapped to the overlapping areas according to component category and intersection-union ratio, includes: AKAZE feature points and binary feature descriptors of adjacent local sub-images are extracted respectively. Feature matching is performed based on Hamming distance. The homography matrix between adjacent local sub-images is calculated using the random sampling consensus algorithm. After projecting each local sub-image onto a unified global coordinate system, image fusion is performed. Calculate the intersection-union ratio (IUR) for bounding boxes that are located within overlapping areas and have the same component category, and merge or retain a unique bounding box when the IUR is greater than a preset threshold.

[0044] In this embodiment, AKAZE feature points and binary feature descriptors of adjacent local sub-images are first extracted, and feature matching between adjacent sub-images is performed based on Hamming distance. Then, RANSAC is used to calculate the homography matrix between adjacent local sub-images, and each local sub-image is progressively projected onto a unified global coordinate system. Image fusion is then performed in the unified coordinate system to generate a complete panoramic image of the PCB. After stitching and annotation mapping are completed, the repetition of multiple annotation boxes located in the overlapping area is checked. Specifically, the component categories of corresponding annotation boxes are first compared. Only when the component categories are the same, the intersection-union ratio (IUGR) between corresponding annotation boxes is calculated. The IUGR is the ratio between the overlapping area of ​​two annotation boxes and their union area, which reflects the degree of spatial overlap between the two annotation boxes. When the IUGR exceeds a preset threshold, such as 0.5, it can be determined that the corresponding annotation boxes point to the same physical component, and merging processing is then performed or only a unique annotation is retained. For example, if a connector spans the boundary area of ​​two adjacent local subgraphs, two connector boxes may appear in the overlapping area after splicing and labeling conversion. If the two are of the same category and the crossover ratio reaches the threshold, one of them can be retained or merged into a final label box to eliminate duplicate labeling. If the categories are different or the crossover ratio is low, it means that the two may correspond to different devices, and they should not be simply deleted to prevent omission of labeling.

[0045] This application achieves stable fusion by using AKAZE feature matching and RANSAC geometric estimation during the sub-graph stitching process, and performs deduplication by combining component category and cross-union ratio in the overlapping area. This can eliminate duplicate annotations while ensuring the integrity of annotations on large-size PCB boards, thereby improving the uniqueness, accuracy and usability of the final BOM annotation results.

[0046] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0047] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0048] Further reference Figure 2 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of a multi-BOM batch annotation system for PCB boards. This system embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0049] like Figure 2 As shown, the PCB board multi-BOM batch annotation system 200 described in this embodiment includes: a first acquisition module 201, a second module 202, an extraction module 203, a filtering module 204, a calculation module 205, a conversion module 206, and a deduplication module 207. Wherein: The first acquisition module 201 is used to acquire a template PCB image and input the template PCB image into a pre-trained component detection model to generate template BOM annotation data. The template BOM annotation data includes at least component category, component position and annotation box coordinates. The second acquisition module 202 is used to acquire the PCB image to be labeled. When the PCB image to be labeled is composed of multiple local sub-images, the multiple local sub-images are stitched together to obtain a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to be labeled. Extraction module 203 is used to extract feature points and feature descriptors from the template PCB image and the PCB image to be labeled, respectively; The filtering module 204 is used to perform feature matching based on the feature descriptor and filter the matching results by combining the component category and component position relationship in the template BOM annotation data to obtain matching point pairs; Calculation module 205 is used to calculate the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pair; The conversion module 206 is used to convert the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and map the converted annotation boxes to the PCB image to be annotated, so as to obtain the target BOM annotation data corresponding to the PCB image to be annotated; The deduplication module 207 is used to deduplicate the annotation boxes mapped to the overlapping area according to the component category and intersection-union ratio when the PCB image to be annotated is obtained by splicing multiple local sub-images.

[0050] The PCB board multi-BOM batch annotation system provided in this embodiment of the invention can realize all the processes of the PCB board multi-BOM batch annotation method of the above embodiments. The functions and technical effects of each module in the device are the same as the functions and technical effects of the PCB board multi-BOM batch annotation method of the above embodiments, and will not be repeated here.

[0051] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0052] The computer device 3 includes a memory 31, a processor 32, and a network interface 33 that are interconnected via a system bus. It should be noted that only the computer device 3 with components 31-33 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0053] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0054] The memory 31 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Of course, the memory 31 may include both the internal storage unit and its external storage device of the computer device 3. In this embodiment, the memory 31 is typically used to store the operating system and various application software installed on the computer device 3, such as computer-readable instructions for multi-BOM batch annotation methods on PCB boards. In addition, the memory 31 can also be used to temporarily store various types of data that have been output or will be output.

[0055] In some embodiments, the processor 32 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 32 is typically used to control the overall operation of the computer device 3. In this embodiment, the processor 32 is used to execute computer-readable instructions stored in the memory 31 or to process data, such as executing computer-readable instructions for the multi-BOM batch annotation method of the PCB board.

[0056] The network interface 33 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 3 and other electronic devices.

[0057] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the multi-BOM batch annotation method for PCB boards as described above.

[0058] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0059] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for batch annotation of multiple BOMs on a PCB board, characterized in that, Includes the following steps: A template PCB image is obtained and input into a pre-trained component detection model to generate template BOM annotation data. The template BOM annotation data includes at least component category, component location, and annotation box coordinates. A PCB image to be labeled is obtained. When the PCB image to be labeled is composed of multiple local sub-images, the multiple local sub-images are stitched together to obtain a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to be labeled. Extract feature points and feature descriptors from the template PCB image and the PCB image to be labeled, respectively; Feature matching is performed based on the feature descriptors, and the matching results are filtered by combining the component categories and component positional relationships in the template BOM annotation data to obtain matching point pairs; Calculate the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pairs; The coordinates of the annotation boxes in the template BOM annotation data are transformed according to the transformation matrix, and the transformed annotation boxes are mapped to the PCB image to be annotated to obtain the target BOM annotation data corresponding to the PCB image to be annotated; When the PCB image to be labeled is obtained by stitching together multiple local sub-images, the label boxes mapped to the overlapping areas are deduplicated according to component category and intersection-union ratio.

2. The method according to claim 1, characterized in that, The step of inputting the template PCB image into a pre-trained component detection model to generate template BOM annotation data includes: The standard template PCB image is input into the component detection model trained based on multi-board type PCB samples to detect each component target in the standard template PCB image; Based on the detection results, template BOM annotation data corresponding to each component target is generated. The template BOM annotation data includes at least the component category, component position, and annotation box coordinates.

3. The method according to claim 1, characterized in that, The step of performing feature matching based on the feature descriptor, and filtering the matching results by combining the component categories and component positional relationships in the template BOM annotation data to obtain matching point pairs, includes: AKAZE feature points and binary feature descriptors are extracted from the template PCB image and the PCB image to be labeled, respectively, and candidate matching point pairs are obtained based on Hamming distance; The candidate matching point pairs are filtered according to the consistency of their component categories and the relative positional relationship of the corresponding components to obtain matching point pairs used to calculate the transformation matrix.

4. The method according to claim 3, characterized in that, The step of calculating the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pairs includes: The random sampling consensus algorithm is used to remove mismatched point pairs from the matched point pairs; The homography matrix between the template PCB image and the PCB image to be labeled is estimated based on the valid matching point pairs after removing mismatched point pairs, and is used as the transformation matrix.

5. The method according to claim 4, characterized in that, The step of transforming the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and mapping the transformed annotation boxes to the PCB image to be annotated, to obtain the target BOM annotation data corresponding to the PCB image to be annotated, includes: Perform perspective transformation on the vertex coordinates of each annotation box to obtain the corresponding vertex coordinates in the PCB image to be annotated; The target bounding box is generated based on the transformed corresponding vertex coordinates, and the target BOM annotation data is generated based on the component category and component position of the target bounding box.

6. The method according to claim 1, characterized in that, The step of stitching together multiple local sub-images to obtain a panoramic image of the PCB to be annotated includes: The image acquisition device is controlled to acquire multiple local sub-images, and the adjacent local sub-images have an overlap area of ​​15% to 25%. Multiple local sub-images are stitched together to form a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to perform feature extraction, feature matching, transformation matrix calculation and label box mapping.

7. The method according to claim 6, characterized in that, The step of stitching multiple local sub-images into a panoramic image of the PCB to be labeled, and deduplicating the label boxes mapped to the overlapping areas according to component category and intersection-union ratio, includes: AKAZE feature points and binary feature descriptors of adjacent local sub-images are extracted respectively. Feature matching is performed based on Hamming distance. The homography matrix between adjacent local sub-images is calculated using the random sampling consensus algorithm. After projecting each local sub-image onto a unified global coordinate system, image fusion is performed. Calculate the intersection-union ratio (IUR) for bounding boxes that are located within overlapping areas and have the same component category, and merge or retain a unique bounding box when the IUR is greater than a preset threshold.

8. A multi-BOM batch annotation system for PCB boards, characterized in that, include: The first acquisition module is used to acquire a template PCB image and input the template PCB image into a pre-trained component detection model to generate template BOM annotation data. The template BOM annotation data includes at least component category, component position and annotation box coordinates. The second acquisition module is used to acquire the PCB image to be labeled. When the PCB image to be labeled is composed of multiple local sub-images, the multiple local sub-images are stitched together to obtain a panoramic image of the PCB to be labeled, and the panoramic image of the PCB to be labeled is used as the PCB image to be labeled. The extraction module is used to extract feature points and feature descriptors from the template PCB image and the PCB image to be labeled, respectively. The filtering module is used to perform feature matching based on the feature descriptor and to filter the matching results by combining the component category and component position relationship in the template BOM annotation data, so as to obtain matching point pairs; The calculation module is used to calculate the transformation matrix between the template PCB image and the PCB image to be labeled based on the matching point pairs; The transformation module is used to transform the coordinates of the annotation boxes in the template BOM annotation data according to the transformation matrix, and map the transformed annotation boxes to the PCB image to be annotated, so as to obtain the target BOM annotation data corresponding to the PCB image to be annotated; The deduplication module is used to deduplicate the annotation boxes mapped to the overlapping areas according to the component category and intersection-union ratio when the PCB image to be annotated is obtained by stitching together multiple local sub-images.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the multi-BOM batch annotation method for PCB boards as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the multi-BOM batch annotation method for PCB boards as described in any one of claims 1 to 7.

Images