A circuit diagram automatic analysis method

Abstract

This application discloses an automatic circuit diagram parsing method, relating to the intersection of computer-aided design and artificial intelligence technologies. By sequentially performing file format conversion, circuit diagram segmentation, text information extraction, and dataset construction on an editable circuit diagram, and using a combination of image processing and semantic parsing, it can accurately extract isolated circuit modules and their corresponding circuit semantic information without manual intervention, effectively improving the circuit diagram parsing effect and efficiency. Simultaneously, graph theory algorithms are used to perform topology optimization on the multimodal training data in the multimodal training dataset, effectively shortening the length of connection relationship text. Finally, the optimized multimodal training data is used for training and inference of a multimodal large language model, requiring only one stage of inference, significantly improving model training efficiency and inference accuracy.

Description

Technical Field

[0001] This application relates to the intersection of computer-aided design and artificial intelligence technologies, specifically to a method for automatic circuit diagram parsing. Background Technology

[0002] Existing circuit diagram connection extraction technologies directly process circuit diagram PDFs or images in multiple stages, employing deep learning models to identify components and connections. The main problems are: 1. Inability to accurately associate graphic elements with text descriptions in the image; 2. Significantly decreased accuracy in densely wired areas; 3. Loss of rich semantic information from the schematic file; 4. Weak generalization ability across different types of circuit diagrams. Summary of the Invention

[0003] The purpose of this application is to provide an automatic circuit diagram parsing method that solves the problems of existing technologies that cannot establish an accurate correspondence between image elements and text descriptions, require multi-stage training and reasoning, and are prone to error accumulation.

[0004] This application is achieved through the following technical solution:

[0005] An automatic circuit diagram parsing method, comprising:

[0006] The editable circuit diagram is converted into a file format to obtain a first target format file and a second target format file; wherein, the first target format file represents the schematic diagram structure information; and the second target format file represents a text description information file.

[0007] Based on the first target format file and the second target format file, the circuit diagram is segmented and synchronized text information is extracted to obtain the segmented target circuit diagram and its corresponding circuit description text.

[0008] A multimodal training dataset is generated based on the segmented target circuit diagram and its corresponding circuit description text; any multimodal training data in the multimodal training dataset includes a segmented circuit diagram and a pair of connection relationship labels;

[0009] A graph theory algorithm is used to perform topology optimization on the multimodal training data in the multimodal training dataset to obtain an optimized multimodal training dataset.

[0010] A multimodal large language model is trained using an optimized multimodal training dataset. The trained multimodal large language model is then used for automatic circuit diagram parsing.

[0011] In one possible implementation, the editable circuit diagram is converted to a file format to obtain a first target format file and a second target format file, including:

[0012] A multi-process parallel conversion strategy is adopted to convert the file format of multiple editable circuit diagrams, converting the schematic structure information into a first target format file and converting the text content and its precise location into a second target format file;

[0013] The editable circuit diagram is an SCH file, the first target format file is an ASCII file, and the second target format file is a PDF file.

[0014] In one possible implementation, based on the first target format file and the second target format file, the circuit diagram is segmented and synchronized text information is extracted to obtain the segmented target circuit diagram and its corresponding circuit description text, including:

[0015] The first target format file is parsed to extract device location, device type, pin information, and network connection relationships to obtain basic circuit information;

[0016] The second target format file is parsed to extract the text content and its corresponding precise location. A unified coordinate mapping mechanism is constructed to map the text data to image pixels, thereby obtaining the basic text information.

[0017] Based on the aforementioned basic circuit information, a circuit segmentation algorithm is used to segment the circuit, resulting in a pre-processed circuit diagram.

[0018] The circuit diagram after preliminary processing is enhanced and segmented using the aforementioned basic text information to obtain the segmented target circuit diagram.

[0019] Based on the aforementioned textual information, the circuit information corresponding to the segmented target circuit diagram is determined, and the circuit description text corresponding to the segmented target circuit diagram is obtained based on the circuit information; the circuit description text is structured data.

[0020] In one possible implementation, based on the aforementioned circuit fundamental information, a circuit segmentation algorithm is used to segment the circuit to obtain a pre-processed circuit diagram, including:

[0021] Based on the aforementioned basic circuit information, a binary mask image is constructed, and device regions and connection lines are marked to obtain an initial circuit diagram;

[0022] The initial circuit diagram is sequentially subjected to seed filling algorithm to identify connected regions, design area sorting and remarking, intelligent merging mechanism of adjacent connected regions, and 30-pixel boundary filling protection to obtain the circuit diagram after preliminary processing.

[0023] In one possible implementation, the pre-processed circuit diagram is enhanced and segmented using the textual basic information to obtain a segmented target circuit diagram, including:

[0024] Based on the aforementioned textual information, and combined with spatial location constraints and text similarity matching, the textual description regions of the devices in the sub-circuit diagram are added to the pre-processed circuit diagram. At the same time, regular expression technology is used to identify and process special text patterns, and electrical characteristics are identified through the device attribute semantic library to obtain the segmented target circuit diagram.

[0025] In one possible implementation, based on the aforementioned textual information, circuit information corresponding to the segmented target circuit diagram is determined, and circuit description text corresponding to the segmented target circuit diagram is obtained based on the circuit information, including:

[0026] Each segmented target circuit diagram is treated as a sub-circuit module;

[0027] Based on the aforementioned textual information, and through coordinate constraints, the complete circuit information corresponding to each sub-circuit module is determined;

[0028] The data is processed using a breadth-first search algorithm to obtain the circuit description text corresponding to the segmented target circuit diagram.

[0029] In one possible implementation, a multimodal training dataset is generated based on the segmented target circuit diagram and its corresponding circuit description text, including:

[0030] The connection relationships are determined based on the circuit description text corresponding to the segmented target circuit diagram;

[0031] The coordinates of the connection points are converted into relative coordinates relative to the center of their corresponding devices, and the relative coordinates are added to the connection relationships to obtain connection relationships with connection point coordinates.

[0032] The segmented target circuit diagram and its corresponding connection relationships with connection point coordinates are used together as multimodal training data.

[0033] In one possible implementation, the method further includes:

[0034] The multimodal training data in the multimodal training data is augmented to obtain augmented multimodal training data, and the augmented multimodal training data is added to the multimodal training data.

[0035] In one possible implementation, a graph theory algorithm is used to perform topology optimization on the multimodal training data in the multimodal training dataset to obtain an optimized multimodal training dataset, including:

[0036] The breadth-first search algorithm is used to parse the connection relationship of each segmented target circuit diagram into multiple isolated connected components, and the connected components are integrated to obtain the processed connection relationship.

[0037] The processed connection relationships are sorted by connection paths to obtain the sorted connection relationships.

[0038] The segmented target circuit diagram and the sorted connection relationships are used together as the optimized multimodal training data, and all the optimized multimodal training data are constructed as follows.

[0039] In one possible implementation, a multimodal large language model is trained using an optimized multimodal training dataset to obtain the trained multimodal large language model, including:

[0040] A multimodal large language model is constructed using a visual encoder / text encoder-language decoder.

[0041] Based on the optimized multimodal training dataset, the multimodal training dataset is trained using a frozen visual encoder, LoRa fine-tuning strategy, and progressive learning strategy to obtain the trained multimodal large language model.

[0042] Compared with the prior art, this application has the following advantages and beneficial effects:

[0043] This application provides an automatic circuit diagram parsing method. By sequentially performing file format conversion, circuit diagram segmentation, text information extraction, and dataset construction on editable circuit diagrams, and combining image processing and semantic parsing, it can accurately extract isolated circuit modules and their corresponding circuit semantic information without manual intervention, effectively improving the circuit diagram parsing effect and efficiency. At the same time, graph theory algorithms are used to perform topology optimization on the multimodal training data in the multimodal training dataset, effectively shortening the length of connection relationship text. Finally, the optimized multimodal training data is used for training and inference of a multimodal large language model, requiring only one stage of inference, significantly improving model training efficiency and inference accuracy. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0045] Figure 1This is a flowchart of an automatic circuit diagram parsing method provided in an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0047] like Figure 1 As shown, this application provides a method for automatic circuit diagram parsing, including:

[0048] S101. Convert the editable circuit diagram to a file format to obtain a first target format file and a second target format file; wherein, the first target format file represents the schematic diagram structure information; and the second target format file represents a text description information file.

[0049] S102. Based on the first target format file and the second target format file, the circuit diagram is segmented and synchronized text information is extracted to obtain the segmented target circuit diagram and its corresponding circuit description text.

[0050] S103. Generate a multimodal training dataset based on the segmented target circuit diagram and its corresponding circuit description text; any multimodal training data in the multimodal training dataset includes a segmented circuit diagram and a pair of connection relationship labels;

[0051] S104. Use graph theory algorithms to perform topology optimization on the multimodal training data in the multimodal training dataset to obtain the optimized multimodal training dataset;

[0052] S105. Train a multimodal large language model using the optimized multimodal training dataset, obtain the trained multimodal large language model, and use the trained multimodal large language model for automatic circuit diagram parsing.

[0053] In one possible implementation, the editable circuit diagram is converted to a file format to obtain a first target format file and a second target format file, including:

[0054] A multi-process parallel conversion strategy is adopted to convert the file format of multiple editable circuit diagrams, converting the schematic structure information into a first target format file and converting the text content and its precise location into a second target format file;

[0055] The editable circuit diagram is an SCH file, the first target format file is an ASCII file, and the second target format file is a PDF file.

[0056] For example, the COM interface of the SailWind software can be used to achieve automatic batch conversion of SCH files to PDF and ASCII files.

[0057] During the conversion process, multiple processes can be used for parallel conversion: the SailWindLogic application is started to load each SCH file; a multi-process architecture is adopted to achieve parallel processing and improve conversion efficiency; a timeout control strategy is designed to automatically terminate and restart the process in case of an exception; and process ID tracking is used to ensure that resources are completely released.

[0058] During the conversion process, pop-up responses can be automatically closed: macro files are generated to automatically handle font replacement pop-ups; specific dialog boxes are detected and responded to via Windows API; the entire conversion process can be completed without manual intervention.

[0059] After the conversion is complete, the file is exported: a high-precision ASCII format file is exported, preserving the complete schematic structure information; a high-quality PDF format image is generated, providing a foundation for subsequent visual processing; and the coordinate system consistency between the two data formats is maintained.

[0060] In one possible implementation, based on the first target format file and the second target format file, the circuit diagram is segmented and synchronized text information is extracted to obtain the segmented target circuit diagram and its corresponding circuit description text, including:

[0061] The first target format file is parsed to extract device location, device type, pin information, and network connection relationships to obtain basic circuit information;

[0062] The second target format file is parsed to extract the text content and its corresponding precise location. A unified coordinate mapping mechanism is constructed to map the text data to image pixels, thereby obtaining the basic text information.

[0063] Based on the aforementioned basic circuit information, a circuit segmentation algorithm is used to segment the circuit, resulting in a pre-processed circuit diagram.

[0064] The circuit diagram after preliminary processing is enhanced and segmented using the aforementioned basic text information to obtain the segmented target circuit diagram.

[0065] Based on the aforementioned textual information, the circuit information corresponding to the segmented target circuit diagram is determined, and the circuit description text corresponding to the segmented target circuit diagram is obtained based on the circuit information; the circuit description text is structured data.

[0066] In one possible implementation, based on the aforementioned circuit fundamental information, a circuit segmentation algorithm is used to segment the circuit to obtain a pre-processed circuit diagram, including:

[0067] Based on the aforementioned basic circuit information, a binary mask image is constructed, and device regions and connection lines are marked to obtain an initial circuit diagram;

[0068] The initial circuit diagram is sequentially subjected to seed filling algorithm to identify connected regions, design area sorting and remarking, intelligent merging mechanism of adjacent connected regions, and 30-pixel boundary filling protection to obtain the circuit diagram after preliminary processing.

[0069] In one possible implementation, the pre-processed circuit diagram is enhanced and segmented using the textual basic information to obtain a segmented target circuit diagram, including:

[0070] Based on the aforementioned textual information, and combined with spatial location constraints and text similarity matching, the textual description regions of the devices in the sub-circuit diagram are added to the pre-processed circuit diagram. At the same time, regular expression technology is used to identify and process special text patterns, and electrical characteristics are identified through the device attribute semantic library to obtain the segmented target circuit diagram.

[0071] In one possible implementation, based on the aforementioned textual information, circuit information corresponding to the segmented target circuit diagram is determined, and circuit description text corresponding to the segmented target circuit diagram is obtained based on the circuit information, including:

[0072] Each segmented target circuit diagram is treated as a sub-circuit module;

[0073] Based on the aforementioned textual information, and through coordinate constraints, the complete circuit information corresponding to each sub-circuit module is determined;

[0074] The data is processed using a breadth-first search algorithm to obtain the circuit description text corresponding to the segmented target circuit diagram.

[0075] For example, the SCH file and PDF file obtained above are parsed, and sub-circuit diagrams are segmented and corresponding circuit diagram description texts are generated. The core of this process includes:

[0076] 1. ASCII / PDF parsing:

[0077] 1.1 Parse the ASCII file to extract the precise location, type, pin information, and network connection relationships of the device;

[0078] 1.2 Read PDF files, convert them into high-resolution images, and extract all text content and precise locations;

[0079] 1.3 Construct a unified coordinate mapping mechanism to accurately correspond text data with image pixels;

[0080] This step primarily parses ASCII and PDF files.

[0081] 2. Sub-circuit functional module segmentation algorithm based on connected components and morphological operations:

[0082] 2.1 To prevent the circuit diagram from being too dense, it is necessary to divide it into sub-circuit modules;

[0083] 2.2 Construct a binary mask image and mark the device regions and connecting lines;

[0084] 2.3. Seed-fill algorithm is used to identify connected components;

[0085] 2.4 Design an area sorting and remarking strategy to ensure that key functional areas are processed first;

[0086] 2.5. An intelligent merging mechanism for adjacent connected components is adopted to automatically merge separated regions based on the overlap of bounding boxes;

[0087] 2.6 Implement 30-pixel boundary padding protection to prevent loss of edge information during segmentation;

[0088] For example, the circuit basic information (device coordinates and pin positions from the ASCII file) and text basic information (image pixel coordinates from the PDF file) obtained in step S102 can be used to divide the entire circuit diagram into independent functional modules. The specific implementation steps are as follows:

[0089] 2.1 To prevent the circuit diagram from being too dense, it is necessary to divide it into sub-circuit modules;

[0090] Since industrial-grade circuit diagrams often contain thousands of components, inputting the entire diagram can lead to memory overflow in multimodal models and distraction of attention mechanisms. Therefore, it is necessary to divide the circuit diagram into several independent sub-circuit modules. The division is based on the physical connectivity of the circuit: electrically connected components are usually clustered in the physical layout and connected by wires.

[0091] 2.2 Construct a binary mask image and mark the device regions and connecting lines;

[0092] Data input: Device center coordinates obtained by parsing the ASCII file in step S102. Component enclosure size And the coordinates of the pin connection points; simultaneously, the image resolution obtained from PDF parsing. Mapping ratio with the schematic logic coordinate system .

[0093] Implementation steps: Create an empty canvas with the same resolution as the PDF image and initialize it to black (pixel value 0).

[0094] Traverse all devices in the ASCII file and determine the mapping ratio. Convert logical coordinates to image pixel coordinates;

[0095] Draw a white rectangle (255 pixels) in the corresponding position on the canvas to mark the device area, and draw white lines to mark the pins and network connection lines;

[0096] Morphological dilation: To avoid connectivity disruptions caused by drawing gaps, binary images are subjected to... The morphological dilation operation of the convolution kernel ensures that physically connected devices form a connected region on the mask.

[0097] Output: An initial binary mask image containing device and interconnect information. .

[0098] 2.3. Seed-fill algorithm is used to identify connected components;

[0099] Algorithm logic: based on The Seed-Fill Algorithm is used to identify independent connected components.

[0100] Implementation steps: Initialize the label matrix Size and Consistent, initial value is 0; traversal All pixels, when encountering a value of 255 and When there are unlabeled pixels, use them as seed points;

[0101] An 8-neighbor connectivity strategy is used for recursive or stack-based filling, marking all connected white pixels as having the same connectivity ID. );

[0102] Record the set of pixels corresponding to each connected component ID. and the circumscribed rectangle boundingbox .

[0103] Output: Set of initially partitioned connected components .

[0104] 2.4 Design an area sorting and remarking strategy to ensure that key functional areas are processed first;

[0105] Logical purpose: There may be a few isolated noise points in the circuit diagram caused by drawing errors (such as tiny broken wires). By sorting by area, the main functional areas can be retained first.

[0106] Implementation steps: Calculate each connected component area (i.e., the total number of pixels contained); set the area threshold. (For example, 0.1% of the total image area) Remove The noisy connected components are identified; the remaining connected components are sorted by area from largest to smallest, and priority labels are reassigned. This step ensures that subsequent merging operations prioritize large-area functional modules, avoiding small fragments from interfering with the integrity of the main module.

[0107] 2.5. An intelligent merging mechanism for adjacent connected components is adopted to automatically merge separated regions based on the overlap of bounding boxes;

[0108] Logical purpose: Some peripheral components of a circuit may not be included in the circuit's connectivity domain and need to be incorporated into the complete circuit module.

[0109] Data usage: Combine the connected component bounding boxes obtained in step 2.4 with the netlist information obtained from ASCII parsing in step S102.

[0110] Merging condition: for the connected components of the circuit and If any of the following conditions are met, the property is considered "adjacent and functionally related":

[0111] Spatial Proximity: The Euclidean Distance Between the Boundary Boxes of Two Connected Domains (e.g., 50 pixels);

[0112] Bounding box overlap: the intersection-union ratio of two connected bounding boxes. (Applicable to cases where the edges are in contact after expansion);

[0113] Electrical connection: Query the ASCII netlist, if Device pins and If the pins of devices belong to the same netname, they will be forcibly merged.

[0114] Implementation steps: Using a union-find data structure, merge the connected component IDs that meet the above conditions, and update their bounding rectangles to the minimum bounding box that contains all sub-connected components.

[0115] 2.6 Implement 30-pixel boundary padding protection to prevent loss of edge information during segmentation.

[0116] This step primarily involves precisely dividing the sub-circuit modules and forming a segmentation mask.

[0117] 3. Further refine the segmentation of sub-circuit regions using PDF text information:

[0118] 3.1 Build a PDF text extraction module to obtain all text content and precise locations;

[0119] 3.1 Build a PDF text extraction module to obtain all text content and precise locations;

[0120] This step aims to accurately extract all text content and its pixel positions in the image coordinate system from the PDF file generated in step S101, providing basic data for subsequent "graphics-text" association and alignment. The specific implementation steps are as follows:

[0121] 1) PDF parsing engine selection and initialization;

[0122] PyMuPDF (fitz library) is used as the core parsing engine. This library supports direct access to the underlying object structure of PDF, and can accurately obtain the font, font size, rotation angle and bounding box coordinates of the text. Moreover, the parsing speed is better than traditional OCR solutions.

[0123] 2) Text block extraction and coordinate acquisition;

[0124] Data input: The high-quality PDF file output from step S101 (consistent with the ASCII file coordinate system).

[0125] Implementation steps:

[0126] Extract the structured text information from the PDF page and return a nested dictionary containing blocks, lines, and spans;

[0127] Traverse the text block: Extract the following fields for each span (the smallest unit of text):

[0128] text: Text content (e.g., "R472", "10kΩ", "GND");

[0129] box: Text bounding box (x0, y0, x1, y1), in PDF logical coordinates;

[0130] rotation: Text rotation angle (0° / 90° / 180° / 270°), used to correct vertical annotation text;

[0131] Coordinate system unification: Converting PDF logical coordinates to image pixel coordinates;

[0132] 3) Text cleaning and structured storage;

[0133] Filtering rules:

[0134] Remove tiny text (mostly noise or decorative symbols) whose bounding box area is smaller than a threshold (e.g., 50 pixels²).

[0135] Remove text blocks whose font names contain "Watermark" or "Header";

[0136] For rotated text, correct the bbox orientation based on the rotation field to ensure that the bounding box is always a horizontal rectangle;

[0137] Output format: Store the cleaned text information as a structured list.

[0138] 4) Correlation verification with ASCII parsed data;

[0139] Data input: Step 2.2 parses the ASCII file to obtain the device list devices_ascii (including device type, logical coordinates, and pin information).

[0140] Association logic:

[0141] Spatial proximity matching: For each PDF text item, calculate the Euclidean distance between its center and the centers of each device in devices_ascii. If distance < D_match (e.g., 30 pixels), it is determined to be the annotation text of that device.

[0142] Text similarity assistance: If spatial matching is ambiguous (e.g., multiple devices are close to each other), calculate the edit distance (Levenshtein Distance) between the PDF text and the ASCII device names, and prioritize matching items with similarity > 0.8;

[0143] Output: Generate a "device-text" mapping table device_text_map, in the format {device_id: {"name":"R472", "value": "10kΩ", "bbox": [...]}}.

[0144] 3.2 Perform dual verification matching on the device positions in ASCII and the corresponding PDF device text positions. Combine spatial position constraints (mask constraints for each sub-circuit) and text similarity matching (device names in PDF and similar device names in ASCII) to add the text description area of ​​the device in the sub-circuit diagram to the sub-circuit mask.

[0145] This step aims to address potential coordinate inaccuracies or missing text issues arising from a single data source. Through dual verification of "spatial location + text semantics," it achieves precise binding between devices and text. The specific implementation principle is as follows:

[0146] 1) Data Input:

[0147] ASCII device list: includes device logical ID (e.g., "U29") and logical center coordinates. Pin list;

[0148] PDF Text List: Text content extracted in step 3.1 (e.g., "U29", "10k"), pixel center coordinates ;

[0149] Sub-circuit Mask Constraint: Sub-circuit region mask generated in step 2.

[0150] 2) Spatial location constraint matching (first layer of verification):

[0151] Iterate through the ASCII device list, for each device Search the PDF text list for text items that meet the following spatial criteria. :

[0152]

[0153] in, Set to 50 pixels (based on typical schematic text spacing statistics).

[0154] Mask Filtering: Only retain text items falling within the Mask area of ​​the sub-circuit generated in step 2, excluding interfering text from headers, footers, or other sub-circuits.

[0155] 3) Text similarity matching (secondary verification):

[0156] For candidate text items that pass spatial verification, calculate the edit distance (Levenshtein Distance) between their content string and the ASCII device ID.

[0157] If similarity If the match is successful, then the match is considered successful; where, This represents the edit distance between candidate texts in the asiccii and pdf. This indicates the length of the candidate text in asiccii. This indicates the length of the candidate text in the PDF.

[0158] Error handling: If spatial matching is successful but text mismatch occurs (e.g., ASCII is "R472", PDF is "R473"), the PDF visual text takes precedence (since the PDF is the final output view) and is marked as "coordinate correction item".

[0159] 3.3. Use regular expression technology to identify and process special text patterns, such as "SWD_CLK []" and "[] SWD_CLK [*]".

[0160] 3.4 Establish a semantic library of device attributes to automatically identify electrical characteristics such as capacitance and resistance values;

[0161] This step aims to extract the electrical parameters of the devices, enriching the semantic information of the circuit description text so that it includes not only connection relationships but also component attributes.

[0162] 1) Semantic library construction:

[0163] Build a regular expression rule base, for example:

[0164] Resistance: ^\d+(\.\d+)?[kKMΩ]+$

[0165] Capacitance: ^\d+(\.\d+)?[uunpPfF]+$

[0166] Inductance: ^\d+(\.\d+)?[uμH]+$

[0167] Establish a unit mapping table: convert "Ω", "k", "uF" and other units into standard units (such as "Ohm" and "uF").

[0168] 2) Identification and Filling Process:

[0169] Iterate through the text surrounding the device matched in step 3.2;

[0170] Use a regular expression library to match the text content; if a match is successful, extract the numerical value and unit.

[0171] Information entry: Enter the identified electrical characteristics into the **circuit description text (JSON structure)** generated in step 4.

[0172] Example: If the text "10k" is detected next to the resistor "R472", then generate {"device":"R472", "type": "Resistor", "value": "10kΩ"} in the final description text, instead of just retaining the device ID.

[0173] The device description area of ​​the sub-circuit is added to the corresponding mask to obtain the segmented sub-circuit diagram.

[0174] 4. Extraction of multi-level circuit connection relationships:

[0175] 4.1. Parse the original connection information from the ASCII file and obtain the segmentation area of ​​each sub-circuit module;

[0176] 1) Obtaining segmented regions: Directly reuse the sub-circuit bounding box list [(x_min, y_min, x_max, y_max), ...] output from step 2.6. Each bounding box corresponds to an independent sub-circuit module.

[0177] 2) Use of text location: The "text content and precise location" extracted in step 3.1 is mainly used for device matching in step 3.2. In this step, the **Netlist** data obtained by ASCII parsing is mainly used because the connection relationship in ASCII is logically accurate, while the PDF text is only used for auxiliary verification.

[0178] 4.2. By using coordinate constraints, obtain complete circuit information for constructing each sub-circuit module from the ASCII file;

[0179] 1) Coordinate constraint logic: Traverse the ASCII full-map netlist, for each device pin Check its coordinates Does it fall within the current subcircuit boundary box? Inside.

[0180] 2) Filtering rules:

[0181] a) If and They belong to the same network (Net), and both are in If the connection is internal, then the connection relationship is preserved;

[0182] b) If exist Inside, but exist For external (cross-module connections), mark it as "External Interface", retain the connection relationship but label it as "Off-Page".

[0183] 3) Purpose: To ensure that the description text of each sub-circuit module is self-contained, while retaining the necessary interface information.

[0184] 4.3. A breadth-first search (BFS) algorithm is used to construct a connection graph from the sub-module circuit information;

[0185] 1) Graph construction principle:

[0186] a) Node definition: Define each device pin as a node in the graph. ;

[0187] b) Edge definition: If two pins belong to the same net name, then an undirected edge is created between them. ;

[0188] c) BFS traversal:

[0189] Initialize the set of unvisited nodes For all pins;

[0190] Choose any unvisited node As a starting point, join the queue. ;

[0191] when If not empty, pop the node. traverse all its adjacent nodes (i.e., other pins on the same network);

[0192] like Not visited, marked as visited and added Record the path at the same time ;

[0193] when An empty value indicates that a connected network has been traversed and a connection path string has been generated.

[0194] 2) Output: Each sub-circuit module corresponds to a connection relationship list, such as ["Net1: PinA-PinB-PinC","Net2: PinD-PinE"].

[0195] This step ultimately yields the circuit diagram description text corresponding to the segmented circuit diagram.

[0196] In one possible implementation, a multimodal training dataset is generated based on the segmented target circuit diagram and its corresponding circuit description text, including:

[0197] The connection relationships are determined based on the circuit description text corresponding to the segmented target circuit diagram;

[0198] The coordinates of the connection points are converted into relative coordinates relative to the center of their corresponding devices, and the relative coordinates are added to the connection relationships to obtain connection relationships with connection point coordinates.

[0199] The segmented target circuit diagram and its corresponding connection relationships with connection point coordinates are used together as multimodal training data.

[0200] In one possible implementation, the method further includes:

[0201] The multimodal training data in the multimodal training data is augmented to obtain augmented multimodal training data, and the augmented multimodal training data is added to the multimodal training data.

[0202] For example, by reading the segmented circuit diagram and its corresponding description text obtained above, a one-click generation of a standardized dataset can be achieved from the segmentation result:

[0203] 1. Precise Connection Location: Parse the description text of the segmented circuit diagram to extract precise connection relationships; convert the coordinates of the connection points to relative coordinates with respect to the center of the components; add spatial location information for each connection relationship. This step mainly obtains the connection relationships corresponding to each segmented circuit diagram and the spatial location of each connection point.

[0204] 2. Intelligent Data Augmentation: Multi-angle rotation (0° to 360°) is used to enhance data diversity; flip transformation is applied to increase data generalization ability; grayscale image versions are generated to adapt to different input requirements; spatial information of connection relationships is kept unchanged to ensure data consistency after augmentation; this step mainly performs data augmentation on segmented circuit diagram data.

[0205] 3. Structured JSON Data Generation: This step ultimately yields a standardized JSON structure containing the segmented circuit diagram and its corresponding connections.

[0206] In one possible implementation, a graph theory algorithm is used to perform topology optimization on the multimodal training data in the multimodal training dataset to obtain an optimized multimodal training dataset, including:

[0207] The breadth-first search algorithm is used to parse the connection relationship of each segmented target circuit diagram into multiple isolated connected components, and the connected components are integrated to obtain the processed connection relationship.

[0208] The processed connection relationships are sorted by connection paths to obtain the sorted connection relationships.

[0209] The segmented target circuit diagram and the sorted connection relationships are used together as the optimized multimodal training data, and all the optimized multimodal training data are constructed as follows.

[0210] For example, this application introduces graph theory algorithms to perform topology optimization on the connection relationships obtained above, significantly optimizing data length and final task performance:

[0211] 1. Connected component identification:

[0212] 1.1 Read the JSON file containing the connection relationships for each segmented circuit diagram;

[0213] 1.2 Construct a connection graph and use breadth-first search (BFS) to identify pairwise connected relationships as isolated connected components;

[0214] This step mainly involves resolving the connection relationships of each segmented circuit diagram into several isolated connected components.

[0215] 2. Integration of collinear connection points: Integrating connected components together, changing the connection relationship from pairwise connection to a whole connected path.

[0216] The core objective of this step is to compress the number of tokens, adapt to the context window limitations of large language models, and maintain the topology unchanged.

[0217] 1) Definition of "connected":

[0218] a) Two connected components are considered "connected" if and only if they share the same net name. In step 1.2, BFS has already classified the pins that are connected to each other as the same connected component (i.e., the same net).

[0219] 2) The significance of splitting and then integrating:

[0220] a) Raw data (split state): The raw ASCII netlist is usually stored as pairwise connections, such as A->B, B->C, C->D. If directly input into the model, three fragments are required for description, and there are redundant nodes (B and C are mentioned repeatedly).

[0221] b) Integrated data (integrated state): By identifying that they belong to the same connected component, they are integrated into path representations A, B, C, D.

[0222] c) Technical effects:

[0223] Length compression: for containing A network of pins, the original representation requires Each pair of connections can be combined into a single sequence, reducing text length by an average of 90%.

[0224] Semantic coherence: Path representation is more in line with human reading habits and is conducive to the model learning the semantic concept of "network connectivity" rather than simply "point-to-point".

[0225] 3) Specific operation: Arrange all node IDs in each connected component identified in step 1.2 according to the sorting rules in step 6.3, and connect them with commas to form the final string.

[0226] This step eliminates redundant representations, reducing text length by an average of over 90%; it primarily merges scattered connections into coherent and more concise path descriptions.

[0227] 3. Coordinate system sorting mechanism:

[0228] 3.1 Sort the connection points in each connected path according to their spatial location. Design a two-level sorting strategy: first sort by Y coordinate (from top to bottom), then by X coordinate (from left to right).

[0229] 3.2. The coordinates of the first connecting point in each connected path are used as the reference point;

[0230] 3.3. Sort all connected paths globally according to the coordinates of the reference point;

[0231] This step mainly involves internally sorting the connection paths corresponding to each sub-circuit diagram based on the global sorting, so that they meet a certain rule, which is beneficial for model learning.

[0232] 4. Output Structured JSON Data: Create a compact text representation for each set of connection paths obtained in the above steps, such as "A,B,C,D" instead of "A->B;B->C;C->D", and finally write it to a JSON file. This step ultimately yields a standardized JSON structure of the connection relationships corresponding to each segmented circuit diagram, forming the training dataset.

[0233] In one possible implementation, a multimodal large language model is trained using an optimized multimodal training dataset to obtain the trained multimodal large language model, including:

[0234] A multimodal large language model is constructed using a visual encoder / text encoder-language decoder.

[0235] Based on the optimized multimodal training dataset, the multimodal training dataset is trained using a frozen visual encoder, LoRa fine-tuning strategy, and progressive learning strategy to obtain the trained multimodal large language model.

[0236] For example, in this application embodiment, the above dataset is used to train a multimodal large language model specifically for understanding circuit connection relationships, and finally used for circuit diagram connection relationship reasoning:

[0237] 1. Model Architecture:

[0238] 1.1. A visual / text encoder-language decoder architecture (qwen3_VL 8B) is adopted.

[0239] 1.2 The visual encoder processes the circuit diagram image and extracts spatial features;

[0240] 1.3 The language decoder processes the description of connection relationships, learns the output order of connection relationships, and generates optimized expressions;

[0241] 1.4 Cross-modal attention mechanism enables precise alignment of images and text;

[0242] 1) Vision Encoder:

[0243] a) Input: The segmented sub-circuit image;

[0244] b) Output: Visual feature vector sequence ;

[0245] c) Technical implementation: Use a pre-trained ViT-L / 14 model and freeze the parameters.

[0246] 2) Text Encoder:

[0247] a) Input: Prompt (e.g., "Please analyze the connection relationships in this circuit diagram:");

[0248] b) Output: Text feature vector ;

[0249] 3) Cross-modal projection layer (Projector):

[0250] a) Function: Visual features Mapping to the text embedding space achieves modal alignment;

[0251] 4) Language Decoder:

[0252] a) Input: Projected visual features + text features;

[0253] b) Output: A text sequence of connection relationships (e.g., "VCC, R1, C1, GND").

[0254] Technical implementation: The architecture is based on LLaMA-7B and fine-tuned using LoRA.

[0255] 2. Training strategy:

[0256] 2.1 Low-Memory Training: Freeze the visual encoder and fine-tune only the text encoder and decoder parts to increase the text output length under limited memory conditions.

[0257] 2.2 Efficient Fine-tuning: Using LoRa for fine-tuning reduces the burden on video memory / main memory and shortens fine-tuning time;

[0258] 2.3. Adopt a progressive learning strategy, starting with simple circuits and progressing to complex circuits;

[0259] 2.4 Perform data augmentation to improve model robustness;

[0260] Based on the above technical solution, the automatic circuit diagram parsing method provided in this application has the following technical effects:

[0261] 1. No manual annotation required: Enables fully automatic conversion from raw SCH files to standard datasets without manual intervention. The processing efficiency is more than 98% higher than traditional methods, and the annotation time for a single circuit diagram is reduced from 120 minutes to 2-5 minutes.

[0262] 2. Establish a precise circuit diagram segmentation dataset: By combining image processing and semantic parsing, isolated circuit modules and their corresponding circuit semantic information can be accurately extracted. The accuracy rate of isolated circuit extraction reaches 99%, the text omission rate on the circuit diagram reaches 3%, and the accuracy rate of circuit semantic information extraction reaches 100%.

[0263] 3. Optimization of connection relationship representation: Through the topology optimization algorithm, the length of the connection relationship text is shortened by an average of 42.7% while maintaining 100% structural integrity. Only one stage of reasoning is required, which significantly improves the model training efficiency and reasoning accuracy.

[0264] 4. Improved Multimodal Model Training Performance: The multimodal model trained using the dataset generated in this application achieves 96% accuracy in understanding circuit connections in the Sailwind scenario, and also demonstrates good generalization ability to other completely different circuit scenarios. It supports understanding long sequence connections and can handle complex circuits with up to 400 different pins and 200+ connected lines. It also supports RGB and Gray image recognition.

[0265] Taking a Sailwind format circuit design provided by a certain company as an example, the design contains about 200 schematics and a total of about 770,000 connected lines.

[0266] Step A: Batch convert all SCH files using SailWind's COM interface:

[0267] Start a multi-processing framework to process 10 SCH files simultaneously;

[0268] Automatically handles font replacement pop-ups without manual intervention;

[0269] The conversion of 200 SCH files was completed within two hours, generating corresponding PDF and ASCII files;

[0270] The system automatically recovers three abnormal processes through a timeout control mechanism.

[0271] Step B, Circuit Diagram Segmentation and Information Extraction:

[0272] Parse the ASCII file to extract the location, type, and pin information of all devices;

[0273] Process PDF images to establish a precise mapping between pixel coordinates and schematic coordinates;

[0274] Through connected component analysis, approximately 9168 circuit functional modules were identified;

[0275] Extract the device text descriptions from each module and establish an accurate correspondence with 98.7% accuracy.

[0276] Identify approximately 2,820,858 pairs of interconnected relationships and construct a complete connection graph;

[0277] Step C: Structured representation of connection relationships:

[0278] Extract connection relationship descriptions from 9168 segmentation results;

[0279] Generate standard JSON format;

[0280] Generate precise coordinates and pin information for each device;

[0281] Convert the connection path into a structured sequence of path points;

[0282] Step D: Optimization of connection topology:

[0283] Identify collinear connection points in 9168 functional modules;

[0284] 213,473 connected components were constructed using breadth-first search (BFS);

[0285] Reorganize the connection order according to the Y-coordinate sorting strategy;

[0286] The original average text length of 5277 characters was optimized to 540 characters, a reduction of 90%;

[0287] Maintain 100% integrity of connection relationships with no information loss;

[0288] Original data format: U29.4>GND;U29.4>U29.3;U29.4>U29.2;U29.3>GND;U29.3>U29.2;U29.2>GND;U29.5>R474.1;U29.5>R472.1;U29.5>VCC1.8_A;U29.5>R473.1;U29.5>C454.1;U29.8>DSPA_SDA;U29.8>R472.2;R473.2>U29.7;U29.7>DSPA_SCL;U29. 6>R474.2;R474.1>R472.1;R472.1>VCC1.8_A;R472.1>R473.1;R472.1>C454.1;R472.2>DSPA_SDA;R474.1>R473.1;R47 3.1>VCC1.8_A;R473.1>C454.1;R473.2>DSPA_SCL;R474.1>VCC1.8_A;R474.1>C454.1;C454.2>GND;C454.1>VCC1.8_A-

[0289] Optimized data format:

[0290] VCC1.8_A,R472.1,R473.1,R474.1,U29.5,C454.1;R472.2,U29.8,DSPA_SDA;R 473.2,U29.7,DSPA_SCL;R474.2,U29.6;U29.2,U29.3,U29.4,GND;C454.2,GND;

[0291] In the data format, taking R472.1 as an example, "R472" represents the device name, and "1" represents the pin code.

[0292] Step E, Multimodal Large Model Training:

[0293] After deduplicating 9168 data points, 6000 valid data points were obtained. 1000 samples were randomly selected as the test set, and the remaining samples were used as the training set.

[0294] The visual-language model was trained using 5000 optimized samples;

[0295] A two-stage training strategy is adopted, first fine-tuning simple circuits and then fine-tuning complex circuits;

[0296] Achieved a connection relationship recognition accuracy of 96% on the test set;

[0297] Result evaluation:

[0298] The overall processing time for automatic annotation of Sailwind format data is approximately 6 hours (compared to approximately 2000 hours using traditional methods and approximately 20 minutes for manual annotation of a single data point).

[0299] When processing the training set, the functional module segmentation accuracy was 98.5%.

[0300] When processing the training set, the device-to-text correspondence accuracy was 97.8%.

[0301] When processing the training set, the text length optimization rate was 95%.

[0302] Dataset size: 9168 high-quality samples;

[0303] Multimodal model training results: 96% accuracy in understanding circuit connections;

[0304] This embodiment demonstrates that the present invention can efficiently and accurately extract structured connection relationships from complex circuit diagrams, providing a high-quality data foundation for training multimodal large language models. After training with multimodal large language models, the performance and efficiency of intelligent circuit analysis can be significantly improved.

[0305] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0306] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0307] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0308] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0309] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0310] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for automatic parsing of a circuit diagram, characterized by, include: The editable circuit diagram is converted into a file format to obtain a first target format file and a second target format file; wherein, the first target format file represents the schematic diagram structure information; and the second target format file represents a text description information file. Based on the first target format file and the second target format file, the circuit diagram is segmented and synchronized text information is extracted to obtain the segmented target circuit diagram and its corresponding circuit description text. A multimodal training dataset is generated based on the segmented target circuit diagram and its corresponding circuit description text; any multimodal training data in the multimodal training dataset includes a segmented circuit diagram and a pair of connection relationship labels; A graph theory algorithm is used to perform topology optimization on the multimodal training data in the multimodal training dataset to obtain an optimized multimodal training dataset. A multimodal large language model is trained using an optimized multimodal training dataset. The trained multimodal large language model is then used for automatic circuit diagram parsing.

2. The circuit diagram automatic parsing method according to claim 1, characterized in that, The editable circuit diagram is converted to a different file format to obtain a first target format file and a second target format file, including: A multi-process parallel conversion strategy is adopted to convert the file format of multiple editable circuit diagrams, converting the schematic structure information into a first target format file and converting the text content and its precise location into a second target format file; The editable circuit diagram is an SCH file, the first target format file is an ASCII file, and the second target format file is a PDF file.

3. The circuit diagram automatic parsing method according to claim 1, characterized in that, Based on the first target format file and the second target format file, the circuit diagram is segmented and synchronized text information is extracted to obtain the segmented target circuit diagram and its corresponding circuit description text, including: The first target format file is parsed to extract device location, device type, pin information, and network connection relationships to obtain basic circuit information; The second target format file is parsed to extract the text content and its corresponding precise location. A unified coordinate mapping mechanism is constructed to map the text data to image pixels, thereby obtaining the basic text information. Based on the aforementioned basic circuit information, a circuit segmentation algorithm is used to segment the circuit, resulting in a pre-processed circuit diagram. The circuit diagram after preliminary processing is enhanced and segmented using the aforementioned basic text information to obtain the segmented target circuit diagram. Based on the aforementioned textual information, the circuit information corresponding to the segmented target circuit diagram is determined, and the circuit description text corresponding to the segmented target circuit diagram is obtained based on the circuit information; the circuit description text is structured data.

4. The circuit diagram automatic parsing method according to claim 3, characterized in that, Based on the aforementioned basic circuit information, a circuit segmentation algorithm is used to segment the circuit, resulting in a pre-processed circuit diagram, including: Based on the aforementioned basic circuit information, a binary mask image is constructed, and device regions and connection lines are marked to obtain an initial circuit diagram; The initial circuit diagram is sequentially subjected to seed filling algorithm to identify connected regions, design area sorting and remarking, intelligent merging mechanism of adjacent connected regions, and 30-pixel boundary filling protection to obtain the circuit diagram after preliminary processing.

5. The circuit diagram automatic parsing method according to claim 3, wherein, The circuit diagram after preliminary processing is enhanced and segmented using the aforementioned basic text information to obtain the segmented target circuit diagram, including: Based on the aforementioned textual information, and combined with spatial location constraints and text similarity matching, the textual description regions of the devices in the sub-circuit diagram are added to the pre-processed circuit diagram. At the same time, regular expression technology is used to identify and process special text patterns, and electrical characteristics are identified through the device attribute semantic library to obtain the segmented target circuit diagram.

6. The circuit diagram automatic parsing method according to claim 3, wherein, Based on the aforementioned textual information, the circuit information corresponding to the segmented target circuit diagram is determined, and the circuit description text corresponding to the segmented target circuit diagram is obtained based on the circuit information, including: Each segmented target circuit diagram is treated as a sub-circuit module; Based on the aforementioned textual information, and through coordinate constraints, the complete circuit information corresponding to each sub-circuit module is determined; The data is processed using a breadth-first search algorithm to obtain the circuit description text corresponding to the segmented target circuit diagram.

7. The automatic circuit diagram parsing method according to claim 1, characterized in that, Based on the segmented target circuit diagram and its corresponding circuit description text, a multimodal training dataset is generated, including: The connection relationships are determined based on the circuit description text corresponding to the segmented target circuit diagram; The coordinates of the connection points are converted into relative coordinates relative to the center of their corresponding devices, and the relative coordinates are added to the connection relationships to obtain connection relationships with connection point coordinates. The segmented target circuit diagram and its corresponding connection relationships with connection point coordinates are used together as multimodal training data.

8. The automatic circuit diagram parsing method according to claim 1, characterized in that, Also includes: The multimodal training data in the multimodal training data is augmented to obtain augmented multimodal training data, and the augmented multimodal training data is added to the multimodal training data.

9. The automatic circuit diagram parsing method according to claim 1, characterized in that, A graph theory algorithm is used to perform topology optimization on the multimodal training data in the multimodal training dataset to obtain an optimized multimodal training dataset, including: The breadth-first search algorithm is used to parse the connection relationship of each segmented target circuit diagram into multiple isolated connected components, and the connected components are integrated to obtain the processed connection relationship. The processed connection relationships are sorted by connection paths to obtain the sorted connection relationships. The segmented target circuit diagram and the sorted connection relationships are used together as the optimized multimodal training data, and all the optimized multimodal training data are constructed as follows.

10. The automatic circuit diagram parsing method according to claim 1, characterized in that, A multimodal large language model is trained using the optimized multimodal training dataset to obtain the trained multimodal large language model, including: A multimodal large language model is constructed using a visual encoder / text encoder-language decoder. Based on the optimized multimodal training dataset, the multimodal training dataset is trained using a frozen visual encoder, LoRa fine-tuning strategy, and progressive learning strategy to obtain the trained multimodal large language model.

Images