Method for analyzing number of pins and geometric coordinates of integrated circuit

Abstract

The invention relates to an integrated circuit pin number and geometric coordinate analysis method, which adopts a multi-agent framework, is realized through cooperative workflow of a chart agent, a planning agent and a parameter agent, and comprises the following steps of: receiving an integrated circuit data manual image by the chart agent; a chip packaging graph area in the image is accurately identified and positioned, and a cut packaging graph image is output; a planning intelligent agent receives the cut packaging graph image, identifies the type of chip packaging, and generates a planning instruction for pin coordinate extraction and analysis according to the packaging type; and the parameter intelligent agent receives the cut packaging graph image and the planning instruction, executes geometric parameter analysis according to the planning instruction, and generates a standardized list containing pin numbers and corresponding geometric coordinates thereof. Compared with the prior art, the method has the advantages that the coordinate recognition precision is remarkably improved, and the processing efficiency is remarkably improved.

Description

Technical Field

[0001] This invention relates to the intersection of electronic design automation (EDA) and artificial intelligence, and in particular to a method for analyzing the number of integrated circuit pins and geometric coordinates based on a multimodal large language model, which is especially suitable for intelligent analysis and packaging modeling of IC drawings in the process of printed circuit board (PCB) design. Background Technology

[0002] As electronic products rapidly evolve towards high-density interconnectivity and miniaturization, integrated circuit packaging technology has undergone iterative evolution from DIP to QFP, BGA, and then to CSP. Currently, the package types defined by the JEDEC standard have expanded to 21 major categories and 186 subcategories, and the pin pitch has been reduced from the traditional 0.65mm to the 0.3mm level, which has placed unprecedented precision requirements on PCB design automation technology.

[0003] However, traditional PCB design faces three major technical bottlenecks: (1) High reliance on manual labor: Engineers need to manually analyze the dimension annotations in the datasheet, and on average, each QFN package requires comparison of 32 dimension parameters, taking about 45 minutes per component. (2) Errors in understanding drawings: Industry surveys show that 68% of PCB rework is due to misreading of package parameters, especially in BGA packages, a 0.1mm deviation in pad position can lead to a 10% soldering defect rate. (3) Data heterogeneity: There are 12 types of differences in annotation systems in the datasheets of mainstream manufacturers. For example, there is a ±15% difference in the pin dimension annotations between TI and ST.

[0004] Meanwhile, existing integrated circuit packaging drawing analysis technologies have the following limitations: (1) Optical character recognition (OCR) schemes: such as ABBYY FineReader, the character recognition accuracy on technical drawings is only 72.3%, and it cannot handle nested annotations and lead indicators. (2) Traditional computer vision: OpenCV template matching method is sensitive to rotation, and the recall rate is less than 40% when processing SOIC and other multi-angle packages. (3) General large language models: models such as GPT-4 and Deepseek have poor prediction ability in IC drawing understanding tasks, and there are pad counting deviations and size misjudgments.

[0005] Chinese patent CN116090391A discloses a pin extraction method and system based on GDS files. It parses the GDS file to obtain D structures by using the first-layer label and first-data-type label of the pin's layer specified by the user. Each structure includes multiple geometric groups. When the second-layer label and second-data-type label of all geometric shapes in the same geometric group are the same as the first-layer label and first-data-type label specified by the user, the corresponding geometric group is determined to be a pin, and the structure including that geometric group is considered a component. The process iterates through all geometric groups, determining whether each group is a pin, and regenerating new pads corresponding to the pins. Pins with the same geometry reference the corresponding new pads. This solves the technical problem of importing GDS files into currently known tools, which only provides graphic information but lacks component, pin, and pad information. However, this method mainly processes GDS files, relies on the user-specified pin layer labels for parsing, has a single data source, and its processing method is mainly based on extracting and matching existing geometric information in the file, resulting in weak adaptability and generalization ability.

[0006] Therefore, there is an urgent need for an automatic PCB design drawing recognition method with engineering semantic understanding capabilities to improve recognition accuracy, efficiency, and applicability. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies, such as low efficiency of manual operation, insufficient accuracy of traditional optical character recognition (OCR) methods, and poor interpretability of "black box" reasoning based on a single large language model (LLM). This invention provides a method for analyzing the number and geometric coordinates of integrated circuit pins. By explicitly simulating the pin analysis workflow of an expert engineer, this method achieves high-precision, high-efficiency, and high-interpretability automatic extraction of the number and geometric coordinates of pins from integrated circuit datasheets.

[0008] The objective of this invention can be achieved through the following technical solutions: A method for analyzing the number of integrated circuit pins and their geometric coordinates, employing a multi-agent framework and implemented through a collaborative workflow involving a graph agent, a planning agent, and a parameter agent, includes the following steps: Chart detection: The chart agent receives an image of an integrated circuit datasheet and accurately identifies and locates the chip package chart area in the image, and outputs a cropped package image. Packaging planning: The planning agent receives the cropped packaging image, identifies the type of chip packaging, and generates planning instructions for pin coordinate extraction and parsing based on the packaging type; Parameter reasoning and generation: The parameter agent receives the cropped package image and the planning instructions, performs geometric parameter parsing according to the planning instructions, and generates a standardized list containing pin numbers and their corresponding geometric coordinates.

[0009] The chart agent is a finely tuned multimodal large language model. Its training objective is to accurately identify and segment the chart areas in the datasheet that describe pin layout and size information, and remove irrelevant text and background information.

[0010] The planning instructions include at least a list of names of the key geometric parameters to be extracted.

[0011] The planning agent receives the cropped package image and matches it with a predefined package type database, classifying it into two-sided pins, four-sided pins, grid arrays, or other irregular types based on pin arrangement. The planning agent queries a parameter list specific to the IC type and generates planning instructions suitable for that type based on the identified package type. For two-sided pins, the key geometric parameters to be extracted include pin pitch and pin row / column spacing; for four-sided pins, the key geometric parameters to be extracted include the number of pins on each side, pin end spacing, and same-side pin spacing; for grid arrays, the key geometric parameters to be extracted include the number of rows and columns, pin pitch, and radius.

[0012] The parameter agent performs the following steps: Based on the planning instructions, identify and interpret the key geometric parameters in the encapsulation diagram image; Based on the identified key geometric parameters, extract the absolute coordinates or relative offsets, and calculate the pin coordinates; The pin coordinates are standardized and converted into a text-based, structured output format.

[0013] The parameter agent calculates the pin coordinates based on the identified key geometric parameters through the following steps: 1) Determine the origin: Take the geometric center of the package as the origin of the coordinate system; 2) Pin Positioning: For each pin, calculate its coordinates using a formula, where, For dual-sided pins, the calculation formula is as follows: , ,in, , indicates row index, , indicates column index, For the number of pin rows, Number of pin columns For the first i Line number j The coordinates of the column pins; , These represent the row spacing and column spacing of the pins, respectively. For a quad-pin package, the calculation formula is as follows: Let the number of pins on each side be n, and the center-to-center distance between the two sides be the pin tip distance, then the half-pitch h = center-to-center distance / 2. ,in, For the first i The offset of each pin relative to the center line of its side. The coordinates of the pin spacing on the same side are: For the pin on the left side, its coordinates are: For the pin on the right, its coordinates are... For the upper pin, its coordinates are For the lower pin, its coordinates are... .

[0014] The planning instructions also include: geometric constraints or derivation formulas for calculating missing coordinates or dimensions.

[0015] When identifying key geometric parameters, if there are parameters that are not explicitly marked in the diagram but must be used to calculate coordinates, the parameter agent applies the geometric constraints or derivation formulas in the planning instructions to perform calculations and inferences. Specifically, for the four side pins, if the pad width is missing and only the pin spacing exists, the parameter agent deduces the width of the outer pad as the pin spacing based on the pin spacing. If the pin end spacing is missing, the parameter agent deduces the pin end spacing by the sum of the distance between the inner edges on both sides and the pad width, or the difference between the distance between the outer edges on both sides and the pad width. If there is a center pad in the package, its size parameters are represented by length and width. If either the length parameter or the width parameter is missing, the missing parameter value is set to another known parameter value. The parameter agent automatically generates the center pad coordinates (0,0) and corresponding dimensions based on this constraint, realizing dynamic reasoning and completion of the center pad.

[0016] The parameter agent also generates pad diagrams: for each pin, it calculates its relative position and draws a pad diagram for each pin according to the pin spacing and package type. For circular pads, the row pin spacing is used as the pad diameter, and for rectangular pads, the row pin spacing and column pin spacing are used as the length and width of the rectangle, respectively. The generated pad layout diagram and pin coordinate information are output together to form an IC package diagram that conforms to physical standards, and are saved together with other data.

[0017] The graph agent, planning agent, and parameter agent are trained using a multi-stage sequential fine-tuning strategy. This training strategy specifically includes: a first stage: fine-tuning the graph agent using a dataset with graph region bounding box annotations; a second stage: fine-tuning the planning agent using a dataset with encapsulation type and planning path annotations; and a third stage: fine-tuning the parameter agent using a dataset with pin coordinate results and intermediate inference steps annotations.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) The graph agent, planning agent, and parameter agent of this invention work together to improve the accuracy of the recognition results. Among them, the graph agent focuses on a high-level semantic task—the chip package graph area, which is more accurate than letting the model directly learn the pin location. The core contribution of the planning agent is to identify the package type and construct the corresponding planning instructions based on the determined package type. After receiving the explicit planning instructions, the parameter agent can call the most suitable dedicated parsing algorithm instead of using a "general" algorithm to handle all cases, thereby greatly improving the accuracy of the final recognition results. In the ICGEO8K benchmark test, the average Euclidean distance dpin of 1.11 is achieved, which is significantly improved compared with other mainstream LLM models.

[0019] (2) The three intelligent agents constitute a complete, end-to-end automated decision-making pipeline. From the input image to the final structured coordinate list, no human intervention is required. This greatly reduces the time engineers spend on repetitive tasks and improves recognition efficiency. The present invention can parse a 144-pin BGA package in just 15 seconds, which is 58 times more efficient than manual work, and supports the automatic separation of multi-page datasheet package diagrams.

[0020] (3) The multiple agents in this invention are modular. If it is necessary to improve the performance of a certain link, the corresponding agent can be updated or trained separately without modifying other agents. This design greatly improves development efficiency and system maintainability.

[0021] (4) The parameter agent performs directional analysis under the guidance of planning instructions, which can concentrate computing resources, thereby improving the analysis speed and accuracy.

[0022] (5) The entire decision-making process of this invention is transparent and traceable. Each agent produces a clear, human-understandable intermediate result, which is highly interpretable. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram illustrating the dataset construction and model training process of this invention; Figure 3 This is a comparison chart of the average Euclidean distance between the predicted and actual values ​​of the thought chain method S1 (one round of dialogue, chained questioning) of the present invention and other single prediction methods S2 (one round of dialogue, asking two single questions in succession) and S3 (two rounds of dialogue, asking one question each time). Figure 4 This is a comparison chart showing the average time consumed by the present invention and several mainstream EDA tools on a single sample. Detailed Implementation

[0024] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0025] This embodiment provides a method for analyzing the number of integrated circuit pins and their geometric coordinates. This method employs a multi-agent framework, implemented through a collaborative workflow involving a Diagram Agent, a Planning Agent, and a Parameter Agent. The core functions of each agent and their inputs and outputs are shown in Table 1 below.

[0026] Table 1 The three agents are trained using a multi-stage sequential fine-tuning strategy to ensure that each agent achieves optimal performance based on the upstream output, thereby jointly completing the automated parsing of the pin geometry coordinates.

[0027] like Figure 1 As shown, the method includes the following steps: S1, Chart Detection: The chart agent receives the integrated circuit datasheet image, accurately identifies and locates the chip package chart area in the image, and outputs the cropped package image. S2, Packaging Planning: The planning agent receives the cropped packaging diagram image, identifies the type of chip packaging, and generates planning instructions for pin coordinate extraction and parsing based on the packaging type; S3, Parameter Reasoning and Generation: The parameter agent receives the cropped package image and the planning instruction, performs geometric parameter parsing according to the planning instruction, and generates a standardized list containing pin numbers and their corresponding geometric coordinates.

[0028] The chart agent is a finely tuned multimodal large language model. Its training objective is to accurately identify and segment the chart area in the datasheet that describes pin layout and size information, and remove irrelevant text and background information.

[0029] The planning agent receives the cropped package image and matches it against a predefined package type database, classifying it according to pin arrangement as 2-sides, 4-sides, grid, or other irregular types. The rules used are as follows: , , The corresponding parameters will be generated based on these types shortly. Identify the package type, and then extract the package size based on the derived formula. Pins per side indicates the number of pins or pads on a single side of an IC package. This parameter is very intuitive for square or rectangular packages (such as QFP and QFN). For example, a QFP package with 48 pins on four sides has a pins_per_side of 12 if the pins are evenly distributed. and Indicates pin pitch. and These represent the pin row and column spacing (center distance). Key parameters are extracted from the predicted text, and the package structure is inferred based on these parameters.

[0030] Based on the classification results, the planning agent queries a list of parameters specific to the IC type. According to the identified package type, the planning agent generates planning instructions applicable to that type. In one embodiment, the planning instructions include at least a list of names of key geometric parameters to be extracted, and may also include geometric constraints or derivation formulas for calculating missing coordinates or dimensions.

[0031] For dual-sided pins, the key geometric parameters to be extracted include pin spacing ( and ) and pin row and column spacing ( and For four-sided pins, the key geometric parameters to be extracted include the number of pins per side (pins_per_side), the pin-to-side distance (side_to_side_distance), and the pin-pacing-within_side (pin_pacing_within_side). The side-to-side_distance parameter measures the total distance between the ends of pins on one side and the ends of pins on the other side of the IC package. This is a crucial dimension when planning the footprint of the IC on the PCB, as it determines the width or length the IC occupies on the board. Pin-pacing-within_side is the pin pitch (center-to-center distance), also often referred to as "pin pitch." It refers to the distance between the center points of two adjacent pins on the same side. This is a core parameter for measuring the pin density of the IC. A smaller pitch means a denser pin density, which places higher demands on the PCB manufacturing process and soldering techniques. For example, common pin pitches include 1.27mm, 0.8mm, 0.65mm, and 0.5mm. For grid arrays, the key geometric parameters to be extracted include the number of rows and columns, pin spacing, and radius. These instructions are parsed from the model output using the `_parse_answer_content()` method and then parametrically inferred using the `_extract_final_parameters()` method, ultimately providing the geometric inference path for subsequent steps.

[0032] The parameterized agent performs the following steps: S31, based on the planning instructions, identify and interpret the key geometric parameters in the encapsulation diagram image.

[0033] In one preferred embodiment, when the parameter agent identifies key geometric parameters, if there are parameters that are not explicitly marked in the diagram but must be used to calculate coordinates, it applies the geometric constraints or derivation formulas in the planning instructions to perform calculation inference: For the four side pins, if the pad width (dx1) is missing, and only the pin spacing (dx, dy) exists, the parameter agent derives the value from the pin spacing, setting dx1=dx and dy1=dy. That is, the width of the outer pad is taken as the pin spacing. If the pin end distance (side_to_side_distance) is missing, the parameter agent derives the pin end distance from the sum of the distance between the inner edges (inner_side_distance) and the pad width (dx1), or the difference between the distance between the outer edges (outer_side_distance) and the pad width (dx1). That is, side_to_side_distance = inner_side_distance + dx1 or side_to_side_distance = outer_side_distance - dx1. If the package contains a center pad (Thermal... The dimensions of the pad are represented by length and width (dx2, dy2). Class-based packages are commonly found in types such as QFN and LGA. To enhance heat dissipation and conductivity, if either the length or width parameter is missing, the missing parameter value is set to another known parameter value. For example, if dy2 is not provided, then dy2 = dx2. The parameter agent automatically generates the coordinates (0,0) and corresponding dimensions of the center pad based on this constraint, realizing dynamic reasoning and completion of the center pad.

[0034] S32 extracts absolute coordinates or relative offsets based on the identified key geometric parameters and calculates the pin coordinates.

[0035] Specifically, the following steps are included: 1) Determine the origin: Take the geometric center of the package as the origin of the coordinate system; 2) Pin Positioning: For each pin, calculate its coordinates using a formula, where, For dual-sided pins, the calculation formula is as follows: , ,in, , indicates row index, , indicates column index, For the number of pin rows, Number of pin columns For the first i Line number j The coordinates of the column pins; , These represent the row spacing and column spacing of the pins, respectively. For a quad-pin package, the calculation formula is as follows: Let the number of pins on each side be n, and the center-to-center distance between the two sides be the pin tip distance, then the half-pitch h = center-to-center distance / 2. ,in, For the firsti The offset of each pin relative to the center line of its side. The coordinates of the pin spacing on the same side are: For the pin on the left side, its coordinates are: For the pin on the right, its coordinates are... For the upper pin, its coordinates are For the lower pin, its coordinates are... .

[0036] S33 standardizes the pin coordinates and converts them into a text-based, structured output format.

[0037] Specifically, the generation tool receives standardized pin coordinate data parsed by the parameter agent. This coordinate information is standardized in the `_format_conversations()` method and passed to the generation tool. The output data structure is then transformed into a text-based, structured final output format.

[0038] In a preferred embodiment, the parameter agent also generates a pad map: for each pin, its relative position is calculated, and a pad map for each pin is drawn according to the pin pitch and package type, wherein for circular pads, the row pin pitch is used as the pad diameter, and for rectangular pads, the row pin pitch and column pin pitch are used as the length and width of the rectangle, respectively. The generated pad layout diagram and pin coordinate information are output together to form an IC package diagram that conforms to physical standards, and are saved together with other data.

[0039] In this embodiment, the graph agent, planning agent, and parameter agent are trained using a multi-stage sequential fine-tuning strategy. This training strategy specifically includes: a first stage: fine-tuning the graph agent using a dataset with graph region bounding box annotations; a second stage: fine-tuning the planning agent using a dataset with encapsulation type and planning path annotations; and a third stage: fine-tuning the parameter agent using a dataset with pin coordinate results and intermediate inference step annotations.

[0040] like Figure 2 As shown, the process of dataset preparation and model training is as follows: Step S101: Multi-source data acquisition and standardization processing.

[0041] Model matching: Data is requested through the model retrieval interface of the Digi-Key API ( / Search / v3 / Products), and a wide range of chip model lists are collected. Non-IC devices are filtered based on the JEDEC standard naming rules.

[0042] EDA Resource Scheduling: For screened devices, further obtain the corresponding datasheets, including basic model information, package diagrams, mechanical package types, reference application circuits, etc. During the data request and acquisition process, whenever a missing package diagram is detected for a component, call the UltraLibrarian's GetFootprint service to obtain the IPC-7351 standard file through the manufacturer model cross-index.

[0043] Data standardization: Establish a mapping table for encapsulation types (e.g., "SOT-23" → "TO-236") to unify naming differences across different platforms.

[0044] Step S102: Design an AI-assisted tool to obtain the IC diagram from the source file and perform manual verification.

[0045] Locating the package diagram from the datasheet: The design of the AI-assisted tool uses a multimodal large model, takes the datasheet PDF as input, performs in-page area detection on each page of the PDF, sets keywords such as "Pad Layout" and "RecommendedLand Pattern" and sets the confidence threshold to 0.9, locates the IC pin diagram and obtains the image of that page for subsequent manual verification.

[0046] Obtain pad size data: The system calls the large language model API interface, using structured instruction prompts to automatically parse the chip EDA file content and extract the center coordinates, height, and width (in mm) of the pads. Considering potential errors in model predictions, the system retains a manual correction interface. Some key data requires final verification and calibration by professional engineers to ensure accuracy. The final output is structured JSON data, with verified and pending verification items labeled to support subsequent manual verification steps.

[0047] Manual verification: The selected pad diagrams and pin coordinate data are matched according to the IC number. Experts compare the pad diagrams and pin coordinate data, manually filtering out abnormal data, such as special pin numbering methods and some irregular pin layout diagrams. Through the above methods, a real dataset of 4,138 samples is constructed. The structure of each sample group is (pic real ,{(x i ,y i ,h i ,w i )} i ), where pic real This is a manually verified pad layout diagram, {(x i ,y i ,h i ,w i )} iThis provides the two-dimensional coordinates of the center of each pin, as well as the height and width of the corresponding pin pad.

[0048] Step S103: ICAGENT-INSTRUCT dataset refinement and three-level annotation Based on the massive dataset consisting of CHIPPAD-REAL and CHIPPAD-SYN (a total of 8,608 samples), a three-level annotation was performed to construct the ICAGENT-INSTRUCT dataset to meet the training requirements of multi-agent frameworks. Three subsets are constructed: The ICLABAGENT-INSTRUCT dataset contains three subsets that are strictly aligned with the agent's subtasks: Diagram Detection Dataset: Used to train a diagram agent, it labels the bounding boxes that encapsulate diagrams.

[0049] Parameter Planning Dataset: Used to train planning agents, it labels the encapsulation type (such as 2-sides, 4-sides, grid, other) and the planning instructions for parameter extraction.

[0050] Parameter Extraction Dataset: Used to train the parameter agent, it contains the final pin coordinate results and step labels for intermediate geometric parameter extraction.

[0051] CoT Integration: The data generation process strictly follows the aforementioned inference workflow. CoT training is introduced during the training phase to facilitate step-by-step inference by the model, ensuring that the model's understanding of the task is reliable and interpretable.

[0052] Step S201: Multi-Stage Sequential Fine-tuning The three agents were sequentially fine-tuned using the ICAGENT-INSTRUCT dataset. The base model for all agents was Qwen2-VL-7B, and the training parameters were consistent to ensure that the downstream agents had the best performance when receiving the output of the upstream agents.

[0053] Graph Agent Fine-tuning: Using graph detection annotations, train the model to accurately locate and segment encapsulate graph regions.

[0054] Fine-tuning of the planning agent: Using planning instructions for annotation, the model is trained to accurately classify package types based on graph images and generate highly correlated, step-by-step customized instructions. For example, for QFN packages, the instructions will explicitly require the extraction of outline dimensions and pin pitch.

[0055] Parametric agent fine-tuning: Using parametric inference annotations, the model is trained to receive planning instructions and treat them as "external knowledge" to guide image parsing. The model needs to be able to locate pins, resolve geometric parameters, perform coordinate calculations, and output the final pin numbers and coordinates based on the instructions. .

[0056] The specific training configuration and hyperparameters are shown in Table 2 below.

[0057] Table 2 Step S202: Geometric Feature Extraction - First Round of Training Initial feature learning stage: The Qwen2-VL-7B model was used as the base model and fine-tuned based on the synthetic dataset obtained in step S102. Prior knowledge of IC geometry was injected during the data loading stage, and knowledge of the electrical engineering field (symmetry / mesh layout) was encoded into the model through position encoding and constraints.

[0058] Three rounds of supervised fine-tuning (SFT) are performed on the synthetic dataset obtained in step S103. A thought process chain guides the large model, improving the fine-tuning effect of its parameters. The thought process sequence is input as a prompt: "Please reason in the following order: First, determine the location of the pad images and basic geometric information; second, determine the pin positions and revert to the center origin coordinates; third, provide the coordinate information of each pin based on the origin coordinates." Guided by this thought process chain, the model provides intermediate inference information. The basic model is then fine-tuned by combining the intermediate and final results.

[0059] Loss function and coordinate optimization: Pin counting task loss ( Cross-entropy loss (CE) is used to monitor the model for the total number of pins.

[0060] Coordinate regression task loss ( Mean Squared Error Loss (MSE) is used to monitor the regression accuracy of the model for each pin coordinate (x,y).

[0061] For a single IC plot sample, the coordinate regression loss function is defined as: in: This is the total number of pins on the chip; , These are the normalized coordinates predicted by the model for the i-th pin; , It is the normalized coordinate truth value of the i-th pin from the JSON tag; = It is MSE The training and fine-tuning process employs the low-rank adaptation (LoRA) technique, using the Adam optimizer with a truncation length of 4096 and a learning rate of 5e-5. In each training phase, 10% of the samples are randomly selected as the validation set, and the number of training epochs is set to 3.

[0062] Step S203: Geometric Feature Extraction - Second Round of Training Advanced feature adaptation stage: The core objective of the second round of training is to improve the model's robustness in predicting pin coordinates (x, y) in the presence of real-world noise (such as image blurring, label offset, and obscuring pin outlines by interfering objects). The training process is similar to S201, but the training set is replaced with a real dataset containing real-world noise. Since the real dataset contains a large amount of real-world noise and exhibits differences in image resolution, pin image type, and label text font, further training and fine-tuning the intermediate model on this complex real dataset can increase the model's ability to process and distinguish subtle image features, thereby improving its robustness in real-world scenarios.

[0063] Through two rounds of training on both synthetic and real datasets, the model simulates the analytical logic of an engineer's "overall layout → local details," offering greater interpretability compared to black-box models. Three rounds of fine-tuning based on real datasets address the issue of recognition anomalies when the model handles blurry or occluded images. The basic geometric feature analysis capabilities gained in the first round of training enable the model to identify fundamental chip geometry; the second round of training allows the model to overcome noise interference, accurately fill in missing details, and stably output high-precision pin coordinates.

[0064] The loss function is the same as in step S201. The training fine-tuning process uses the low-rank adaptation (LoRA) technique, selecting the cross-entropy loss function and employing the Adam optimizer, setting the cutoff length to 4096, and the learning rate to 5e-5. 10% of the samples are randomly selected as the validation set in each training phase, and the number of training rounds is set to 3.

[0065] The final pin count and coordinate automatic recognition model is obtained through the above two training steps.

[0066] The IC component's package diagram is taken as input, and the model trained in step S201 outputs the corresponding pin count prediction value and pin coordinate prediction value.

[0067] The design incorporates comparative experiments with various general-purpose large-scale models, outputting and recording key performance indicators such as accuracy and runtime. The model is compared with multiple large-scale models to verify the accuracy and efficiency of the pin count and coordinate automatic recognition model. Coordinate prediction accuracy is the core evaluation metric. This embodiment selects the Mean Euclidean Distance Error (MEDE) as the evaluation standard: calculating the average Euclidean distance between the predicted coordinates of all pins and their true coordinates (converted to actual physical units, mm).

[0068] To evaluate the geometric reasoning capabilities of the aforementioned pin count and coordinate auto-recognition model, this embodiment compares its performance with current mainstream large models (GPT-4o, Gemini 2.0, DeepSeek-VL2, Qwen2-VL-7B) on the ICGEOQA benchmark. All general-purpose models are called via API, and single-sample hint engineering is used to ensure output format compatibility.

[0069] In 398 sets of industry benchmark tests (ICGEOQA): This invention achieves an average Euclidean distance dpin of 1.11, which significantly surpasses other mainstream large models (GPT-4o, Gemini 2.0, DeepSeek-VL2, Qwen2-VL-7B).

[0070] like Figure 3 As shown, compared to directly asking questions and training a large model, the model using the coherent thought chain training strategy (S1) achieves better performance (mean Euclidean distance dpin value of 1.11), confirming the effectiveness of multi-task collaborative learning for geometric reasoning.

[0071] like Figure 4 As shown, single-package parsing takes 15 seconds. While ensuring accuracy, this is 58 times more efficient than manual annotation by professional engineers using mainstream EDA tools (Altium, Autodesk EAGLE, KiCAD, traditional tools take an average of 7-15 minutes per sample).

[0072] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An integrated circuit pin number and geometry coordinate resolution method, characterized by, The method adopts a multi-agent framework and is implemented through a collaborative workflow of a chart agent, a planning agent and a parameter agent, and includes the following steps: Chart detection: The chart agent receives the integrated circuit data manual image, accurately identifies and locates the chip package chart region in the image, and outputs the cropped package chart image; Package planning: The planning agent receives the cropped package chart image, identifies the type of the chip package, and generates planning instructions for pin coordinate extraction and analysis according to the package type; Parameter reasoning and generation: The parameter agent receives the cropped package chart image and the planning instructions, performs geometric parameter analysis according to the planning instructions, and generates a standardized list containing pin numbers and their corresponding geometric coordinates.

2. The method of claim 1, wherein, The chart agent is a fine-tuned multi-modal large language model, and its training target is to accurately identify and segment the chart region in the data manual that describes the pin layout and size information, and to remove irrelevant text and background information.

3. The method for analyzing the number of integrated circuit pins and geometric coordinates according to claim 1, characterized in that, The planning instructions at least include a list of names of key geometric parameters to be extracted.

4. The method of claim 3, wherein, The planning agent receives the cropped package chart image, matches it with a pre-defined package type database, classifies it into double-sided pins, four-sided pins, grid array or other irregular types according to the pin arrangement, queries the IC type-specific parameter list, and generates planning instructions suitable for the type according to the identified package type, wherein for double-sided pins, the key geometric parameters to be extracted include pin pitch and pin row-column pitch; for four-sided pins, the key geometric parameters to be extracted include the number of pins on each side, pin end pitch and same-side pin pitch; for grid array, the key geometric parameters to be extracted include the number of rows and columns, pin pitch and radius.

5. The method of claim 1, wherein: The parameter agent performs the following steps: According to the planning instructions, identify and interpret the key geometric parameters in the package chart image; Based on the identified key geometric parameters, extract absolute coordinates or relative offsets, and calculate pin coordinates; Standardize the pin coordinates and convert them into textual and structured format.

6. The method of claim 5, wherein: The parameter agent calculates the pin coordinates based on the identified key geometric parameters through the following steps: 1) Determine the origin: take the geometric center of the package as the coordinate origin; 2) Position the pins: for each pin, calculate its coordinates by the formula, wherein, For a double-sided pin, the calculation formula is as follows: , , wherein, , represents a row index, , represents a column index, is the number of pin rows, is the number of pin columns, is the coordinate of the pin in the i-th row, i is the coordinate of the pin in the j-th column; j , , respectively represent the pin row spacing and the column spacing. For four-side pin package, the calculation formula is as follows: assuming that the number of pins on each side is n, the distance between the center lines of two sides is the distance between the ends of pins, then the half distance h = the distance between the center lines of two sides / 2, wherein, is the offset of the i pin relative to the center line of the side where it is located, is the distance between pins on the same side, for the pins on the left side, the coordinates are , for the pins on the right side, the coordinates are , for the pins on the top side, the coordinates are , and for the pins on the bottom side, the coordinates are .

7. The method of claim 3, wherein the method further comprises: determining the number of pins and the geometric coordinates of the pins of the integrated circuit based on the number of pins and the geometric coordinates of the pins of the integrated circuit. The planning instructions also include geometric constraints or derivation formulas for calculating missing coordinates or dimensions.

8. The method of claim 7, wherein: When identifying key geometric parameters, if there are parameters in the chart that are not explicitly labeled but must be used to calculate coordinates, the parameter agent applies the geometric constraints or derivation formulas in the planning instructions to perform calculation and reasoning: For four-side pin, if the pad width is missing and only the pin pitch exists, the parameter agent derives the width of the peripheral pad as the pin pitch; if the pin end pitch is missing, the parameter agent derives the pin end pitch by the sum of the distance between the inner edges of the two sides and the pad width, or the difference between the distance between the outer edges of the two sides and the pad width; if there is a center pad in the package, the size parameters are expressed in length and width, and if any one of the length parameter or the width parameter is missing, the missing parameter value is set as the known parameter value, and the parameter agent automatically generates the center pad coordinates (0, 0) and the corresponding size according to the constraint to realize dynamic reasoning and completion of the center pad.

9. The method of claim 5, wherein the method further comprises: determining the number of pins and the geometric coordinates of the pins of the integrated circuit based on the number of pins and the geometric coordinates of the pins of the integrated circuit. The parameter agent also generates a pad diagram: for each pin, the relative position is calculated, and the pad diagram of each pin is drawn according to the pin pitch and the package type, wherein for a circular pad, the row pin pitch is used as the pad diameter, and for a rectangular pad, the row pin pitch and the column pin pitch are used as the length and the width of the rectangle respectively; The generated pad layout and pin coordinate information are output together to form an IC package diagram conforming to the physical standard, and are saved together with other data.

10. The method of claim 1, wherein: The chart agent, the planning agent and the parameter agent are trained by a multi-stage sequential fine-tuning strategy, which specifically includes: first stage: fine-tuning the chart agent using a data set with chart area bounding box annotation; second stage: fine-tuning the planning agent using a data set with package type and planning path annotation; third stage: fine-tuning the parameter agent using a data set with pin coordinate result and intermediate reasoning step annotation.

Images