A method and system for automatically generating a connector BOM for an industrial engineering drawing

Abstract

The application discloses a kind of connector BOM automatic generation method and system for industrial engineering drawings.The method comprises: obtaining industrial engineering drawings and preprocessing, using cascaded search strategy to locate the key drawing page containing connector specification parameters.Detect the connector entity and line position table in the key page, and crop the corresponding area image.Perform visual feature inference stream and text semantic analysis stream on the cropped image in parallel, obtain visual parameter set and text parameter set, and obtain fusion parameter set through consistency check and conflict resolution.Based on the spatial position of the entity area image, a local search domain is constructed, and the attribute text in the search domain is associated with the entity.Tolerance check is performed on the numerical parameters in the fusion parameter set, and the attribute text is integrated to obtain a revised parameter set.Based on the revised parameter set, a general specification model is generated, and a standard material library is matched to generate and output the BOM of the entity.The application significantly improves the generation efficiency and accuracy of the connector BOM.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing technology for industrial engineering drawings, and more specifically, relates to a method and system for automatic generation of connector BOM for industrial engineering drawings based on multimodal cascaded retrieval and dual-path mutual verification of video and text. Background Technology

[0002] In industrial equipment, data centers, communication base stations, and automotive electronics, connectors serve as crucial interconnecting components for signal and power transmission, and their selection directly impacts the physical compatibility and electrical reliability of products. Connector BOM (Bill of Materials) generation is a critical step in the product design process, from drawings to procurement and manufacturing. It involves the accurate extraction and material system mapping of specifications such as pin count, insertion direction, pull tabs, and impedance.

[0003] Currently, connector BOM generation mainly relies on manual methods. Designers need to browse through dozens or even hundreds of pages of PDF / CAD drawings, locate the page containing the specification parameter table, manually read the parameters, and then enter or query the ERP system to complete the BOM compilation. This process is highly dependent on human experience, inefficient, and prone to errors.

[0004] In recent years, the development of artificial intelligence and computer vision technologies has provided new technical means for drawing analysis. In the intelligent recognition of engineering drawings, deep learning technology has been widely applied to symbol recognition, text extraction, and connection information detection. Technical solutions based on object detection, semantic segmentation, and OCR fusion have made progress in engineering drawing recognition in fields such as communications, power, and construction. In connector drawing processing, existing research focuses on the automatic generation and verification of connector CAD drawings, as well as defect detection and model recognition based on visual positioning. Regarding automatic BOM generation, methods for extracting information from product drawings to generate BOMs have been studied. In multimodal document retrieval, multi-vector retrieval methods based on visual language models (such as ColPali) have been proposed for processing documents rich in visual structure. These methods capture multimodal information such as text, charts, and tables by encoding pages as images and generating multi-vector embeddings. In the field of engineering documents, there have been explorations of layout-aware multimodal retrieval systems and recursive multimodal retrieval methods.

[0005] Despite the aforementioned technological explorations, existing connector BOM automatic generation solutions still have the following significant drawbacks.

[0006] First, the efficiency of locating key information is low. Connector specification tables or partial views account for a very small proportion of the entire set of drawings (usually less than 5% of the total page area). General retrieval methods (such as CLIP) compress the entire page into a single global vector, resulting in a severe loss of fine-grained local details during global pooling, making it difficult to effectively distinguish between visually similar but semantically different page types such as "assembly drawings" and "definition drawings." Traditional OCR solutions first convert the document into plain text, discarding spatial relationships and visual structure, and have limited ability to process non-text elements such as tables and diagrams. Although multi-vector retrieval methods based on visual language models (such as ColPali) have emerged in recent years, these methods are mainly designed for general graphic document design. When dealing with the high-density vector lines and extremely small-scale specification tables unique to engineering drawings, there are still problems with inaccurate feature alignment, and they have not yet been systematically integrated with the automatic BOM generation process.

[0007] Second, structural feature extraction is incomplete and lacks robustness. Connectors involve multi-dimensional heterogeneous features such as the number of pins, insertion direction, mechanical accessories (locking clips, pull straps), and electrical impedance, which are presented as graphic symbols, text labels, or visual forms. A single visual model is susceptible to factors such as occlusion and scanning distortion, making it difficult to simultaneously cover multi-dimensional feature extraction. In existing systems, visual recognition and text recognition are often independent of each other, lacking an effective cross-modal collaborative mechanism, and no dedicated multi-dimensional heterogeneous feature extraction framework has been designed for the specific component of connectors.

[0008] Third, there is a lack of spatial correlation between text and graphics. In engineering drawings, connector entities and their corresponding specifications (model, impedance, number of pins) are often spatially separated, with the entity located in the center of the page and the parameters scattered in side or bottom tables. Existing technologies lack an effective cross-regional spatial correlation mechanism, leading to incorrect parameter attribution. That is, when multiple connectors exist on the same page, it is impossible to accurately determine the correspondence between each parameter and each entity. Research on text-graphic separation in the field of engineering drawing vectorization primarily serves to distinguish between graphics and text, rather than focusing on precise graphic-parameter correlation at the semantic level.

[0009] Fourth, there is a lack of a fault-tolerant closed loop from non-standard identification to standard data. AI vision models inevitably contain errors in industrial drawing recognition, such as misidentifying a 64-pin connector as a 63-pin connector (due to occlusion or undercounting), or misidentifying a 74-pin connector as a 73-pin connector. These identification results cannot be directly mapped to the discrete standardized values ​​(such as 64, 74, 98, etc.) in the database. Existing technology lacks effective fault-tolerant verification and automatic correction mechanisms, causing the automated process to break down at this stage, still requiring manual intervention for verification.

[0010] Fifth, the integration of existing solutions is insufficient. Although there are scattered technical points such as drawing recognition, OCR, and BOM generation, there is a lack of an end-to-end integrated solution from drawing input to standard ERP part number output. Data flow in each link relies on manual connection, making it difficult to form a complete automated closed loop.

[0011] In summary, existing connector BOM generation technologies suffer from problems such as difficulty in locating key information, incomplete extraction of multi-dimensional features, lack of spatial correlation between graphics and text, lack of fault tolerance mechanisms for non-standard data, and insufficient system integration. There is an urgent need for a fully automated solution that can automatically match standard materials from intelligent analysis of engineering drawings. Summary of the Invention

[0012] The purpose of this invention is to propose an automatic connector BOM generation method and system for industrial engineering drawings, addressing the technical problems in existing connector BOM generation processes, such as difficulty in locating key specification pages, incomplete extraction of multi-dimensional heterogeneous features, lack of spatial correlation between graphic and textual parameters, lack of fault-tolerant mapping mechanism between AI recognition results and standard databases, and insufficient integration of various stages. This invention achieves end-to-end automated BOM generation from input of industrial engineering drawings to output of standard ERP part numbers. It achieves accurate location of key pages through a cascading retrieval strategy, reliable extraction of multi-dimensional features through visual-text dual-path parallel reasoning and mutual verification, accurate binding of graphic entities and neighboring attribute text through spatial correlation, and automatic correction and standardized mapping of AI recognition deviations through parameter tolerance verification.

[0013] To achieve the above objectives, in a first aspect, this invention proposes a method for automatically generating connector BOMs based on industrial engineering drawings, comprising: Obtain industrial engineering drawings and preprocess them; A cascading retrieval strategy is used to locate key drawing pages containing connector specification parameters from the preprocessed industrial engineering drawings. Detect the connector entity and wire position table in the key drawing page, and crop out the connector entity area image and wire position table area image from the key drawing page; Visual feature inference stream and text semantic parsing stream are executed in parallel on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set, respectively. The connector visual parameter set and connector text parameter set are subjected to consistency verification and conflict resolution to obtain the fused connector parameter set; A local search domain is constructed based on the spatial location of the connector entity region image, and the attribute text within the local search domain is associated with the connector entity. The numerical parameters in the fusion connector parameter set are subjected to tolerance verification, and the attribute text is incorporated into the verified parameter set to obtain the corrected connector parameter set. Generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity and output it.

[0014] Optionally, the step of locating key drawing pages containing connector specification parameters from the preprocessed industrial engineering drawings using a cascading retrieval strategy includes: Based on the object detection model, basic graphic elements are detected on all preprocessed drawing pages. Pages that detect preset basic graphic elements and have a confidence level higher than a preset threshold are marked as candidate pages. Based on the multi-vector vision-language retrieval model, the images of the candidate pages are encoded into multiple local embedding vectors. By calculating the similarity between the preset query vector and the local embedding vectors of each candidate page, the page with the highest score is selected as the key drawing page.

[0015] Optionally, the step of detecting the connector entity and wire position table in the key drawing page, and cropping the connector entity area image and wire position table area image from the key drawing page, includes: Based on the target detection model, target detection is performed on the key drawing pages, and the detection boxes of the connector entities and the line position table are output. Based on the detection frame of the connector entity, the image of the connector entity area is cropped from the key drawing page; Based on the detection frame of the line position table, the line position table area image is cropped from the key drawing page.

[0016] Optionally, the parallel execution of visual feature inference and text semantic parsing streams on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set respectively includes: The image of the connector entity region is input into a first visual model, and the first visual model is used to perform structural classification on the image of the connector entity region to identify the insertion direction and pull strap accessories of the connector entity; the image of the wire position table region is input into a second visual model, and the second visual model is used to perform visual counting on the image of the wire position table region to obtain the number of first pins; Simultaneously, optical character recognition is performed on the connector entity area image to extract the direction and accessory character labels on the connector entity; optical character recognition is performed on the line position table area image to extract the first and last pin numbers and / or explicit pin number descriptions; semantic mapping is performed on the character labels according to preset rules to obtain the text direction and text pull tab of the connector entity; the second pin number is obtained by calculating or directly extracting the explicit pin number description based on the first and last pin numbers; The insertion direction, the zipper attachment, and the number of the first pins identified by the visual feature inference stream are used as the connector visual parameter set. The text direction, text pull, and number of the second pins parsed from the text semantic parsing stream are used as the connector text parameter set.

[0017] Optionally, the step of performing consistency verification and conflict resolution on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set includes: The insertion direction is compared with the text direction, the pull strap attachment is compared with the text pull strap, and the number of the first pins is compared with the number of the second pins; If the insertion direction is consistent with the text direction, the pull strap attachment is consistent with the text pull strap, and the number of the first pins is consistent with the number of the second pins, then the connector visual parameter set or the connector text parameter set is directly output as the fused connector parameter set. If the insertion direction is inconsistent with the text direction and / or the pull strap attachment is inconsistent with the text pull strap and / or the number of the first pin is inconsistent with the number of the second pin, then a conflict resolution is performed according to a preset modal priority strategy, and the resolution result is used as the fusion connector parameter set.

[0018] Optionally, the preset modal priority strategy includes: When the text direction or text pull strip contains strong semantic keywords and the corresponding optical character recognition confidence level is higher than a preset threshold, the corresponding parameter value that is inconsistent with the connector visual parameter set in the connector text parameter set is adopted. When the optical character recognition confidence level of all text content in the connector text parameter set is not higher than a preset threshold, the corresponding parameter value in the connector visual parameter set that is inconsistent with the connector text parameter set is adopted. When the deviation between the number of the first pin and the number of the second pin is within a preset unit range, the pin value conforming to the standard sequence is adopted.

[0019] Optionally, constructing a local search domain based on the spatial location of the connector entity region image, and associating the attribute text within the local search domain with the connector entity includes: Using the center of the detection box in the image of the connector entity region as a reference, a local search domain is constructed by expanding outward by a preset multiple; Calculate the distance between each text region within the local search domain and the center of the detection box; Select the nearest preset number of texts as candidate attribute texts, and lock the specification parameters of the connector entity through keyword matching.

[0020] Optionally, the step of performing tolerance verification on the numerical parameters in the fusion connector parameter set and incorporating the attribute text into the verified parameter set to obtain a corrected connector parameter set includes: Extract the pin count identification value from the parameter set of the fusion connector and perform tolerance matching in the connector standard specification database; if the absolute value of the deviation between the pin count identification value and the standard pin count of the corresponding model in the connector standard specification database is less than the preset tolerance threshold, then the pin count identification value is automatically corrected to the standard pin count. The pin count after tolerance matching is fused with the direction parameters, pull parameters, and attribute text in the fused connector parameter set to obtain the corrected connector parameter set.

[0021] Optionally, the step of generating a general specification model based on the modified connector parameter set, matching the general specification model with a standard material library, generating and outputting the BOM of the connector entity includes: The pin count, orientation parameters, pull tab parameters, and attribute text in the modified connector parameter set are merged according to a preset naming rule to generate a general specification model. Search the standard material library based on the general specifications and models to obtain the corresponding ERP procurement part number; Generate and output a BOM containing ERP purchase part numbers, specifications, usage, and tag numbers.

[0022] Secondly, this invention proposes an automatic connector BOM generation system for industrial engineering drawings, comprising: The acquisition and preprocessing module is used to acquire industrial engineering drawings and preprocess the industrial engineering drawings. The positioning module is used to locate the key drawing page containing connector specification parameters from the preprocessed industrial engineering drawings using a cascading retrieval strategy. The detection and cropping module is used to detect the connector entity and the wire position table in the key drawing page, and to crop out the connector entity area image and the wire position table area image from the key drawing page. The reasoning and parsing module is used to perform visual feature reasoning and text semantic parsing in parallel on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set, respectively. The consistency verification and conflict resolution module is used to perform consistency verification and conflict resolution on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set. The construction and association module is used to construct a local search domain based on the spatial location of the connector entity region image, and associate the attribute text within the local search domain with the connector entity; The tolerance verification module is used to perform tolerance verification on the numerical parameters in the fusion connector parameter set, and to integrate the attribute text into the verified parameter set to obtain the corrected connector parameter set. The generation and output module is used to generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity and output it.

[0023] The beneficial effects of this invention are as follows: By employing a cascaded retrieval strategy, the preprocessed industrial engineering drawings are sequentially subjected to coarse screening based on basic graphic elements and fine-grained semantic matching based on visual-linguistic multimodal features. This achieves rapid and accurate positioning of key drawing pages containing connector specification parameters, effectively solving the problems that fine-grained local information is easily submerged in global features due to the extremely small proportion of specification parameter tables or partial views in the entire set of drawings, and the difficulty in effectively distinguishing assembly drawings from definition drawings. This significantly improves the accuracy and processing efficiency of key page positioning; by analyzing the cropped connector entity area image and line position table area... The domain image executes visual feature inference and text semantic parsing in parallel, extracting multi-dimensional heterogeneous parameters such as connector orientation, straps, and pin count, achieving synergistic complementarity between visual and text modalities. This effectively overcomes the shortcomings of single visual models, which are susceptible to occlusion, scanning distortion, and difficulty in simultaneously covering multi-dimensional feature extraction, significantly improving the completeness and robustness of feature extraction. Through consistency verification and conflict resolution of the visual and text parameter sets, a preset modal priority strategy is implemented: when the text is clear, the text stream is prioritized; when the text is blurry, the visual stream is reverted to be prioritized; and the pin count deviation is kept within a preset unit range. Prioritizing values ​​conforming to standard sequences, this approach achieves intelligent fusion of dual-source data, effectively resolving the cross-modal conflict caused by the independence of visual and text recognition, and significantly improving the accuracy and reliability of parameter extraction. By constructing a local search domain based on the spatial location of the connector entity region image, calculating the distance between each text region within the search domain and the center of the detection box, and selecting a preset number of the closest text regions as candidate attribute texts, it achieves precise spatial binding between graphic entities and neighboring attribute texts, effectively solving the problem of spatial separation between connector entities and specification parameters in engineering drawings. This addresses the issue of incorrect parameter attribution, preventing parameter mismatches when multiple connectors exist on the same page. By performing tolerance matching on the pin count identification values ​​in the fusion connector parameter set against the connector standard specification database, the system automatically corrects the identification value to the standard value when the absolute value of the deviation between the identification value and the standard value is less than the preset tolerance threshold. This achieves automatic correction of AI identification deviations and accurate mapping of non-standard data to standardized values, effectively eliminating identification errors caused by factors such as occlusion and distortion in the AI ​​visual model. It establishes a fault-tolerant closed loop from non-standard identification to standard data, avoiding the problem of automated processes breaking down at this stage and requiring manual intervention for verification.By fusing the pin count obtained after tolerance matching with attribute text derived from orientation parameters, pull tab parameters, and spatial association, a universal specification model is generated. This model is then matched against a standard material library to obtain the corresponding ERP procurement part number, generating a Bill of Materials (BOM) containing the part number, specification description, usage quantity, and tag number. This achieves end-to-end automated BOM generation from industrial engineering drawing input to standard ERP part number output. Data flow at each stage requires no manual intervention, forming a complete automated closed loop. This significantly improves the efficiency and accuracy of connector BOM generation, reduces labor costs, and can be widely applied to connector selection and automated BOM generation scenarios in servers, communication equipment, automotive electronics, and industrial control.

[0024] The system of the present invention has other features and advantages that will be apparent from or will be set forth in detail in the accompanying drawings and following detailed description, which together serve to explain the particular principles of the invention. Attached Figure Description

[0025] The above and other objects, features and advantages of the present invention will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0026] Figure 1 A flowchart illustrating the steps of an automatic connector BOM generation method for industrial engineering drawings according to Embodiment 1 of the present invention is shown. Detailed Implementation

[0027] The invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0028] Example 1

[0029] like Figure 1 As shown, this embodiment provides a method for automatically generating connector BOMs for industrial engineering drawings, including: S1. Obtain industrial engineering drawings and preprocess them; Specifically, industrial engineering drawings are used as input data sources, including at least one of the following: electronic drawings in PDF format, rasterized images exported from CAD software, and digital copies of paper drawings acquired by a scanner.

[0030] Since multi-source input drawings differ significantly in page organization, image quality, and color format, directly feeding them into downstream AI models for processing will severely affect recognition accuracy. Therefore, it is necessary to perform systematic preprocessing operations on the acquired industrial engineering drawings.

[0031] First, the industrial engineering drawings are split into multiple pages, breaking them down into individual single-page images to ensure that subsequent processing can be performed independently on a single-page basis. Then, the original arrangement order of each single-page image is determined based on its own file metadata or page number markers, maintaining a logical order consistent with the original drawings. This allows for tracing the page number to which information belongs during subsequent BOM generation.

[0032] After the page splitting and sequential arrangement are completed, the images on each page are uniformly converted into a preset grayscale format. By compressing the three-channel data of the color image into single-channel grayscale data, the computational load of subsequent image processing is greatly reduced, while eliminating the interference of color differences between different scanning devices on feature extraction.

[0033] Next, the grayscale images of each page are denoised. A filtering algorithm is used to remove salt-and-pepper noise, Gaussian noise, and interference stripes generated during the scanning or transmission of industrial drawings, restoring the clear edges and line structure of the drawings. Based on this, geometric distortion correction is performed on each denoised page image. Perspective transformation or affine transformation is used to eliminate tilting, perspective distortion, or curling distortions generated during scanning or reproduction, restoring the correct geometric shape of the drawings. This solution specifically places geometric distortion correction after denoising. Its advantage lies in: firstly, denoising removes noise points that may interfere with the accuracy of geometric correction, allowing subsequent correction algorithms such as edge detection and Hough transform to extract the true boundaries and line features of the drawings, thereby calculating accurate rotation angles or mapping matrices and avoiding correction distortion caused by noise-induced false edges.

[0034] Finally, the distortion-corrected images of each page are scaled to a preset resolution to ensure that all pages are highly consistent in spatial scale, thereby ensuring that subsequent object detection models, multimodal retrieval models, and visual classification models can perform inference with a uniform input size.

[0035] Through the above preprocessing steps, the original multi-source heterogeneous industrial engineering drawings are converted into a standardized grayscale image set with uniform format, geometric correction, consistent scale, and noise elimination, providing a high-quality data foundation for subsequent cascaded page positioning and feature extraction.

[0036] S2. Use a cascading search strategy to locate the key drawing pages containing connector specifications from the preprocessed industrial engineering drawings. In this step, a cascading search strategy is used to locate key drawing pages containing connector specification parameters from the preprocessed industrial engineering drawings, including: Based on the object detection model, basic graphic elements are detected on all preprocessed drawing pages. Pages that detect preset basic graphic elements and have a confidence level higher than a preset threshold are marked as candidate pages. Based on the multi-vector vision-language retrieval model, the images of candidate pages are encoded into multiple local embedding vectors. By calculating the similarity between the preset query vector and the local embedding vectors of each candidate page, the page with the highest score is selected as the key drawing page.

[0037] Specifically, a cascading retrieval strategy is used to locate key drawing pages containing connector specifications from preprocessed industrial engineering drawings, which includes two levels of progressive filtering.

[0038] The first stage is the coarse screening stage, which uses an object detection model to detect basic graphic elements on all preprocessed drawing pages. This object detection model employs a lightweight, single-stage detection network architecture, pre-trained on identifiable basic graphic elements in industrial engineering drawings. These basic graphic elements include at least two categories: connector outlines and tables. Connector outlines, as the direct visual representation of the connector entity in the drawing, have relatively fixed geometric features, such as a rectangular body, pin array distribution, and latching protrusions. By annotating a large number of image areas containing connector outlines, the object detection model learns the ability to visually recognize connector entities. Tables, as the core carrier of specifications, contain structured information such as pin number sequences, model identifiers, and impedance values. Through annotation and training of table areas, the object detection model acquires the ability to locate table areas on the drawing page. During inference, each preprocessed drawing image is input into the object detection model, which outputs the category label and confidence score for each detection box. When the detection confidence score of any basic graphic element on a page is higher than a preset threshold, that page is marked as a candidate page. Through this mechanism, the original set of drawings, which may have contained dozens or even hundreds of pages, is quickly reduced to a candidate set of a few to a dozen pages, greatly reducing the amount of data that needs to be processed for subsequent fine-grained retrieval.

[0039] The second stage is the fine-tuning phase, which uses a multi-vector visual-language retrieval model to further refine the location of key drawing pages from the candidate pages. Unlike general retrieval methods that compress the entire page image into a single global vector, this multi-vector visual-language retrieval model employs a multi-vector representation architecture. Each page image in the candidate pages is encoded into multiple local embedding vectors. Each local embedding vector corresponds to a segmented image patch region, representing the visual semantic information of that local region, including but not limited to whether the region contains graphic outlines, table structures, text annotations, and their semantic categories. Simultaneously, the system pre-constructs natural language descriptive text related to the connector specification parameter page. This descriptive text contains a comprehensive description of the target page's visual content and semantic information, and is encoded as a query vector. In the matching phase, a fine-grained interactive matching mechanism is used to calculate the similarity between the query vector and each local embedding vector of the candidate page, and then aggregates the results to obtain the global similarity score for the candidate page. The core advantage of this matching mechanism is that it allows different semantic dimensions of the query vector to be matched with different local regions of the page image, thereby achieving a fine-grained understanding of the page content. For example, when the query text contains two semantic elements, "connector graphic" and "parameter table," the former can generate a high similarity response with the local embedding vector of the area where the connector entity is located on the page, and the latter can generate a high similarity response with the local embedding vector of the area where the table is located on the page. Through this combination of local matching and global aggregation, the system can accurately identify pages that contain both connector graphics and specification parameter tables, effectively distinguishing between assembly drawings that "only contain connector graphics but no parameter table" and definition drawings that "contain both graphics and parameter tables." Finally, the candidate page with the highest global similarity score is selected as the key drawing page for subsequent detection, cropping, and feature extraction steps. Through the above two-level cascaded retrieval strategy, accurate positioning of key drawing pages containing connector specification parameters is achieved while ensuring processing efficiency.

[0040] S3. Inspect the connector entity and wire position table in the key drawing page, and crop out the connector entity area image and wire position table area image from the key drawing page. In this step, the connector entity and wire location table are detected in the key drawing page, and the connector entity area image and wire location table area image are cropped from the key drawing page, including: Based on the target detection model, target detection is performed on key drawing pages, and the detection boxes of connector entities and line position tables are output. Based on the detection frame of the connector entity, cut out the image of the connector entity area from the key drawing page; Based on the detection box of the line position table, crop the image of the line position table area from the key drawing page.

[0041] Specifically, after locating the key drawing page containing connector specifications using a cascading search strategy, it is necessary to precisely separate the image regions required for subsequent visual reasoning and text parsing from this key page. Since the key drawing page often contains not only the target connector entity and its corresponding line position table, but also other connector graphics, dimension annotations, drawing frame information, technical description text, and other interfering elements, directly inputting the entire key page into the subsequent visual model or OCR engine would result in severe background interference and feature confusion. Therefore, it is necessary to further perform target-level fine-grained detection and cropping operations on the key page.

[0042] Key drawing pages are input into a pre-trained object detection model. This model differs from the lightweight detection model used to label candidate pages in the first-level coarse screening; it is a refined detection model specifically trained and optimized for two types of targets: connector entities and line position tables. During model training, a large number of engineering drawing pages containing connector entities and line position tables are collected. Boundary boxes for the connector entity region and line position table region are labeled on each training image. This labeled training set is used for supervised training of the object detection model. To adapt the model to special aspect ratios such as the slender connector entities and the horizontal length of the line position tables, a clustering algorithm is used to cluster the labeled boxes in the training set before training. This generates multiple sets of prior anchor boxes that adapt to the size distribution of connector entities and line position tables, thereby optimizing the regression accuracy of the detection model. In the inference phase, after inputting the key drawing pages into the trained object detection model, the model uses a convolutional neural network to extract features from the page images at multiple scales. It then performs category prediction and position regression on predefined anchor boxes on feature maps at different scales, finally outputting all detection results with a detection confidence higher than a preset threshold. Each detection result includes the target's category label, connector entity or line position table, and the corresponding bounding box coordinates. These bounding box coordinates are typically represented as the position information of the top left and bottom right corners of the rectangle in the image pixel coordinate system, or the center point coordinates and width and height information.

[0043] Through the target detection process described above, the system obtains the detection bounding boxes for the connector entity and the line position table. Subsequently, an image cropping operation is performed: based on the coordinates of the connector entity's detection bounding box, the rectangular region defined by the detection bounding box is cut out from the original key drawing page, generating the connector entity region image; based on the coordinates of the line position table's detection bounding box, the corresponding rectangular region is cut out from the same key drawing page, generating the line position table region image. The cropping operation essentially maps the detection bounding box coordinates onto the original image matrix, extracts all pixel data within that coordinate range, and forms independent sub-image patches.

[0044] Through the aforementioned detection and cropping steps, the original key drawing page, which contained a large amount of redundant background information, was decomposed into two highly focused sub-images: the connector entity area image focuses on presenting the connector's outline, pin array, shell shape, and surface silkscreen, directly serving the subsequent structural classification of the first visual model and the visual counting of the second visual model; the pin number table area image focuses on presenting the pin number sequence, explicit pin count description, and other tabular information, directly serving the subsequent OCR recognition and rule parsing. The two sub-images maintain spatial independence, providing their own dedicated input data sources for the subsequent "parallel execution of visual feature inference flow and text semantic parsing flow."

[0045] S4. Perform visual feature inference flow and text semantic parsing flow in parallel on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set, respectively. In this step, visual feature inference and text semantic parsing are performed in parallel on the connector entity region image and the line position table region image, respectively, to obtain the connector visual parameter set and connector text parameter set, including: The connector entity area image is input into the first vision model, which performs structural classification on the connector entity area image to identify the insertion direction and pull tab accessories of the connector entity; the wire position table area image is input into the second vision model, which performs visual counting on the wire position table area image to obtain the number of first pins. Simultaneously, optical character recognition is performed on the connector entity area image to extract the direction and accessory character labels on the connector entity; optical character recognition is performed on the wire position table area image to extract the first and last pin numbers and / or explicit pin number descriptions; semantic mapping is performed on the character labels according to preset rules to obtain the text direction and text pull tab of the connector entity; the number of second pins is obtained by calculating or directly extracting the explicit pin number descriptions based on the first and last pin numbers. The insertion direction, pull strap attachment, and number of first pins identified by the visual feature inference stream are used as the connector visual parameter set; The text direction, text pull, and number of second pins parsed from the text semantic parsing stream are used as the connector text parameter set.

[0046] Specifically, after obtaining the connector entity region image and the line position table region image through detection and cropping steps, a visual feature inference stream and a text semantic parsing stream are executed in parallel on the two region images to extract multi-dimensional heterogeneous parameters of the connector from different modal dimensions. The visual feature inference stream and the text semantic parsing stream adopt an independent and parallel processing architecture, with no data dependency between them, thereby making full use of computing resources and shortening the overall processing time.

[0047] In the visual feature inference flow, the image of the connector entity region is input into the first visual model. This first visual model employs an image classification architecture based on a deep convolutional neural network and is pre-trained on a dataset of connector entity images labeled with insertion direction categories and pull tab accessory categories. The insertion direction category includes at least two types: Straight (STR) and Right Angle (RA), corresponding to the mechanical assembly angle between the connector and the circuit board, respectively. The pull tab accessory category includes at least two types: with pull tab and without pull tab. The pull tab is a mechanical accessory on the connector used to assist in insertion and removal operations. The first visual model performs multi-layer convolution and pooling operations on the input image to extract the outline features, end shape features, and side structure features of the connector entity region image. The extracted high-dimensional features are then mapped to the classification space through a fully connected layer, ultimately outputting the insertion direction category and pull tab accessory category of the connector entity. This recognition process does not rely on text annotations on drawings but infers directly from the visual appearance of the connector, thus providing reliable visual parameters even when text annotations are missing or blurred.

[0048] Simultaneously, the pin count area image is input into the second visual model. This second visual model employs a multimodal large-scale model architecture, featuring an end-to-end structure of a visual encoder and a language decoder. It can receive multimodal inputs of images and text prompts and generate corresponding text outputs. During inference, the pin count area image and preset counting prompts are input into the second visual model. These prompts are natural language instructions guiding the model to perform the pin counting task, such as "Please count the total number of pins in the diagram." The visual encoder of the second visual model encodes the pin count area image into a visual feature sequence, and the language decoder, guided by the counting prompts, decodes the visual feature sequence into a text response describing the number of pins. After numerical extraction, the first pin count is obtained. This visual counting method leverages the powerful visual understanding and reasoning capabilities of the multimodal large-scale model, achieving direct estimation of the pin count without additional model training for the counting task. It is particularly suitable for scenarios where the pin count image is clear but lacks explicit numerical annotations.

[0049] In the text semantic parsing stream, optical character recognition (OCR) and rule parsing are performed on the connector entity region image and the wire position table region image, respectively. First, OCR is performed on the connector entity region image to extract text information from the connector entity's surface and edge areas. Since connector entities often have silkscreened direction abbreviations (such as "R / A", "RA", "ST", etc.) and accessory markings (such as "TAB", "PULLTAB", etc.), these character annotations provide textual evidence about direction and pull straps. Through OCR, these character regions in the image are converted into machine-readable text strings. Then, semantic mapping is performed on the recognized character annotations according to a preset rule mapping table. For example, "R / A" or "RA" is mapped to the text direction "RA", "ST" or "STR" is mapped to the text direction "STR", and "TAB" or "PULLTAB" is mapped to the text "pull strap has pull strap". This rule mapping is based on common abbreviation conventions in connector drawings, which can unify diverse annotation methods into standardized parameter values.

[0050] Simultaneously, optical character recognition (OCR) is performed on the thread position table area image to extract the first and last pin numbers and explicit pin count descriptions. The thread position table typically presents the pin numbering information in tabular form. The first and last pin numbers are the numerical values ​​of the starting and ending pins in the table, and the explicit pin count description is the total number of pins appearing directly in the form of "64Pin", "64-Pin", "64PIN", etc. After recognition, the extracted information is parsed according to preset rules: if the first and last pin numbers are identified, then the formula N=N is applied. end -N start +1 to calculate the second pin count; if an explicit pin count description is identified, the value is directly extracted as the second pin count; when both methods are effective, a consistency check can be performed or the explicit description can be prioritized. This OCR-based and rule-based calculation method can provide accurate pin count information and is not affected by visual occlusion or image distortion, making it particularly suitable for scenarios with clear line position tables and complete text annotations.

[0051] After completing the two parallel processing streams mentioned above, the insertion direction, zipper attachment, and number of first pins identified by the visual feature inference stream are output as the connector visual parameter set; the text direction, text zipper attachment, and number of second pins parsed by the text semantic parsing stream are output as the connector text parameter set. Thus, the system obtains two parameter sets from the visual modality and the text modality, respectively. These two sets correspond in their parameter composition: the insertion direction and text direction correspond to the same feature (direction); the zipper attachment and text zipper attachment correspond to the same feature (zipper attachment); and the number of first pins and number of second pins correspond to the same feature (pin count). This provides a data foundation for subsequent consistency checks and conflict resolution.

[0052] S5. Perform consistency verification and conflict resolution on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set. In this step, consistency checks and conflict resolution are performed on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set, which includes: Compare the insertion direction with the text direction, compare the pull strap attachment with the text pull strap, and compare the number of the first pin with the number of the second pin; If the insertion direction is consistent with the text direction, the pull strap attachment is consistent with the text pull strap, and the number of the first pin is consistent with the number of the second pin, then the connector visual parameter set or connector text parameter set is directly output as the fused connector parameter set. If the insertion direction is inconsistent with the text direction and / or the pull attachment is inconsistent with the text pull and / or the number of the first pin is inconsistent with the number of the second pin, then a conflict resolution will be performed according to the preset modal priority strategy, and the resolution result will be used as the fusion connector parameter set.

[0053] Specifically, after obtaining the connector visual parameter set and connector text parameter set by executing the visual feature inference stream and text semantic parsing stream in parallel, respectively, since the two parameter sets originate from different modal channels, the corresponding parameter values ​​extracted from each may be consistent or inconsistent. Therefore, it is necessary to perform systematic consistency verification and conflict resolution on the two parameter sets to obtain a final reliable fused connector parameter set. The specific process of consistency verification and conflict resolution is as follows.

[0054] First, the three sets of corresponding parameters in the connector visual parameter set and the connector text parameter set are compared one by one. The insertion direction in the visual parameter set is compared with the text direction in the text parameter set to determine if the two modalities agree on the same connector directional feature. The zipper attachment in the visual parameter set is compared with the text zipper in the text parameter set to determine if the two modalities agree on the same connector zipper feature. The number of the first pin in the visual parameter set is compared with the number of the second pin in the text parameter set to determine if the two modalities (visual modality and text modality) agree on the same pin count feature. These three sets of comparisons cover all the core parameters required for connector selection. Any inconsistency in any set of comparisons indicates an information conflict between the two modalities, requiring the initiation of an adjudication mechanism to resolve it.

[0055] If all three sets of comparisons are consistent—that is, the insertion direction matches the text direction, the zipper attachment matches the text zipper, and the number of the first pin matches the number of the second pin—it indicates that the visual feature inference flow and the text semantic parsing flow mutually verify the recognition results of the connector's various features and there is no conflict. In this case, either the connector visual parameter set or the connector text parameter set is directly output as the fused connector parameter set, because the two parameter sets have exactly the same parameter values ​​in this scenario, and using either as the fusion result will not affect the accuracy of subsequent processing steps.

[0056] If any of the three sets of comparisons above are inconsistent—that is, if the insertion direction is inconsistent with the text direction, the pull strap attachment is inconsistent with the text pull strap, or the number of the first pin is inconsistent with the number of the second pin—it indicates that there is a discrepancy between the two modalities in the recognition results of one or more features of the connector. The recognition result of a single modality cannot be directly accepted. In this case, conflict resolution is required according to a preset modality priority strategy. This modality priority strategy sets a hierarchical set of resolution rules based on the reliability differences of different modality information sources. The system resolves the conflicting parameters item by item according to this strategy, and combines the determined parameter values ​​after resolution into a fused connector parameter set output. Through the above consistency verification and conflict resolution process, the system realizes intelligent fusion of visual and text modalities. It can mutually verify and enhance confidence when the results of the two channels are consistent, and can also reasonably resolve and eliminate ambiguity according to the preset strategy when the results of the two channels conflict, thereby ensuring the accuracy and reliability of the fused connector parameter set.

[0057] The preset modal priority strategies include: When the text direction or text pull strip contains strong semantic keywords and the corresponding optical character recognition confidence is higher than the preset threshold, the corresponding parameter value that is inconsistent with the connector visual parameter set in the connector text parameter set is adopted. When the optical character recognition confidence of all text content in the connector text parameter set is not higher than the preset threshold, the corresponding parameter values ​​in the connector visual parameter set that are inconsistent with the connector text parameter set are adopted. When the deviation between the number of the first pin and the number of the second pin is within a preset unit range, the pin value conforming to the standard sequence is used.

[0058] Specifically, the preset modality priority strategy is used to determine the basis for deciding the final accepted parameter value when there is inconsistency between the output results of the visual feature inference stream and the text semantic parsing stream. Based on the reliability differences of different modal information in industrial engineering drawings, this strategy sets three levels of priority rules, and the system judges and executes the decision level by level in the following order.

[0059] The first priority rule is: when the text direction or text pull tab contains strong semantic keywords and the corresponding optical character recognition confidence level is higher than a preset threshold, the corresponding parameter values ​​that are inconsistent between the connector text parameter set and the connector visual parameter set are adopted. In industrial engineering drawings, direction information is often clearly marked near the connector entity in the form of abbreviations such as "R / A", "RA", "ST", and "STR", and pull tab information often appears in the form of keywords such as "TAB" and "PULLTAB". These strong semantic keywords directly originate from the designer's annotation intention and have high semantic clarity and reliability. When the optical character recognition model's recognition confidence level for the above characters is higher than the preset threshold, it indicates that the text recognition result has high credibility. At this time, the parameter values ​​provided by the text stream should be adopted first, replacing the visual insertion direction with the text direction, or replacing the visual pull tab attachment with the text pull tab. The design logic of this priority rule is that text annotations are explicit specification information intentionally placed by the designer on the drawings, and their reliability is higher than the reasoning judgment of the visual model based on the shape contour.

[0060] The second priority rule is: when the confidence level of optical character recognition for all text content in the connector text parameter set is not higher than a preset threshold, the corresponding parameter values ​​in the connector visual parameter set that are inconsistent with the connector text parameter set are accepted. In actual industrial drawings, the characters in some old scanned parts may have generally low OCR recognition confidence levels due to reasons such as drawing aging, insufficient scanning clarity, and excessively small fonts. When the confidence level of all text recognition results fails to reach the preset threshold, it indicates that the overall text information of the current drawing is unreliable, and the judgment result of the visual feature inference flow should be reverted. The design logic of this priority rule is that low-confidence text recognition results may contain erroneous characters, and their misleading nature is higher than that of visual inference; therefore, the result of the visual flow should be taken as the standard.

[0061] The third priority rule is: when the deviation between the number of the first and second pins is within a preset unit range, the pin value conforming to the standard sequence is adopted. The number of pins is a core parameter in connector selection, and its value typically follows industry-standard sequences, such as 64, 74, and 98. When the deviation between the first pin count obtained through visual counting and the second pin count calculated by OCR rules is within a preset unit range, it indicates that the recognition results of the two modalities, although slightly different, are nearly identical. This difference often stems from subtle factors such as image occlusion or character misreading. In this case, the recognition result is matched with the standard sequence, and the standard sequence value closest to the current recognition value is selected as the final pin count. The design logic of this priority rule is: when the results of the two modalities are close but not completely identical, the normative knowledge of the industry standard sequence is used to correct minor AI recognition deviations. This eliminates conflicts between modalities and anchors non-standard recognition values ​​to standardized values, providing industry-standard parameters for subsequent specification locking and material matching.

[0062] The three priority rules mentioned above constitute a complete adjudication chain from "high-confidence text priority" to "low-confidence text back to visual" and then to "standard sequence fallback correction", which can form reasonable adjudication results under different recognition conditions.

[0063] S6. Construct a local search domain based on the spatial location of the connector entity region image, and associate the attribute text within the local search domain with the connector entity; In this step, a local search domain is constructed based on the spatial location of the connector entity region image, and the attribute text within the local search domain is associated with the connector entity, including: Using the center of the detection box in the image of the connector entity region as a reference, a local search domain is constructed by expanding outward by a preset multiple; Calculate the distance between each text region within the local search domain and the center of the detection box; Select the nearest preset number of texts as candidate attribute texts, and lock the specification parameters of the connector entity through keyword matching.

[0064] Specifically, after obtaining the fusion connector parameter set through consistency verification and conflict resolution, the fusion connector parameter set includes parameters directly extracted from the connector entity itself, such as connector orientation, straps, and pin count. However, it does not yet include equally crucial specification attributes such as connector model and impedance. These specification attributes are usually presented as text within a certain range around the connector entity in engineering drawings, rather than being marked inside the connector entity or adjacent to its edge. Due to the layout characteristics of engineering drawings, there is often a spatial distance between the connector entity graphic and the corresponding specification parameter text. If the attribute text is searched directly across the entire drawing, it is easy to incorrectly associate the parameters of other connectors or unrelated components on the same page with the current connector entity. Therefore, it is necessary to construct a local search domain based on the spatial location of the connector entity area image, and accurately associate the attribute text within this search domain with the connector entity.

[0065] The specific process of spatial association is as follows: First, the spatial location information of the detection box in the connector entity area image is obtained. This detection box information comes from the aforementioned steps of detecting the connector entity and the line position table in the key drawing page and cropping the connector entity area image. Specifically, it includes the position coordinates of the connector entity detection box in the coordinate system of the key drawing page, usually represented as the coordinates of the upper left and lower right corners of a rectangle or the coordinates of the center point and its width and height. The spatial location of this detection box reflects the actual distribution area of ​​the connector entity in the key drawing page.

[0066] Then, using the center of the detection box in the connector entity area image as a reference, a local search domain is constructed by expanding outward by a preset multiple. The center of the detection box is the coordinate of the intersection of the two diagonals of the rectangular detection box. The preset multiple is a pre-defined expansion coefficient; for example, multiplying the width and height of the detection box by the preset multiple, and then defining a larger rectangular search area on the key drawing page centered on the center of the detection box. The value of this preset multiple is greater than 1, ensuring that the constructed local search domain can cover text information within a certain range around the connector entity, while not being too large to introduce parameter text unrelated to the connector or component. Through this operation, the originally unrestricted global text search is constrained to the vicinity of the connector entity's local area.

[0067] Based on this, the distance between each text region within the local search domain and the center of the detection box is calculated. The text regions within the local search domain are derived from the full-image optical character recognition results of key drawing pages. These results record all recognized text content on the page and its corresponding bounding box coordinates. The system traverses all text regions within the local search domain and calculates the spatial distance between each text region and the center of the connector entity's detection box, for example, using Euclidean distance or Manhattan distance.

[0068] Finally, a predetermined number of the closest texts are selected as candidate attribute texts, and the specification parameters of the connector entity are locked through keyword matching. The closer the text is, the stronger the spatial correlation between it and the connector entity, and the higher the probability that it belongs to the specification parameter of that connector. The system sorts all text regions located within the local search domain from near to far from the center of the detection box, and selects the predetermined number of the closest texts as candidate attribute texts. Then, keyword matching is performed on each candidate attribute text, such as checking whether the candidate text contains known model identification keywords, impedance value keywords, etc. Once a match is successful, the attribute text is associated with the current connector entity and locked as a specification parameter of that connector entity, supplementing it into the corrected connector parameter set.

[0069] Through the above spatial association steps, the system uses the principle of spatial proximity to accurately assign attribute text scattered around the connector entity to that connector entity, effectively avoiding the problem of parameter reading errors when multiple connectors exist on the same page, and providing an accurate data source for the subsequent generation of general specifications containing complete model and impedance information.

[0070] S7. Perform tolerance verification on the numerical parameters in the fusion connector parameter set, and integrate the attribute text into the verified parameter set to obtain the corrected connector parameter set. In this step, tolerance checks are performed on the numerical parameters in the fusion connector parameter set, and the attribute text is incorporated into the checked parameter set to obtain the corrected connector parameter set, including: Extract the pin count identification value from the fusion connector parameter set and perform tolerance matching in the connector standard specification database; if the absolute value of the deviation between the pin count identification value and the standard pin count of the corresponding model in the connector standard specification database is less than the preset tolerance threshold, the pin count identification value is automatically corrected to the standard pin count. The corrected connector parameter set is obtained by fusing the pin count after tolerance matching with the direction parameters, pull parameters, and attribute text in the fusion connector parameter set.

[0071] Specifically, after obtaining the fusion connector parameter set through consistency verification and conflict resolution, and acquiring the attribute text within the connector entity's neighborhood through spatial association, the fusion connector parameter set includes orientation parameters, pull tab parameters, and pin count identification values. The attribute text contains specification information such as model number and impedance. However, the pin count identification value may deviate from the actual value due to factors such as visual occlusion, image distortion, or OCR misreading. Directly using this identification value for subsequent specification generation may lead to incorrect selection. Therefore, it is necessary to perform tolerance verification on the numerical parameters in the fusion connector parameter set and integrate the attribute text into the verified parameter set to obtain a corrected connector parameter set.

[0072] The specific process of tolerance verification and parameter fusion is as follows: First, the pin count identification values ​​are extracted from the connector parameter set and then matched against the connector standard specification database for tolerance. The database pre-stores the standard pin counts for each connector model, along with the complete specification definitions for each standard pin count. The core logic of tolerance matching is to compare the extracted pin count identification values ​​one by one with the standard pin counts stored in the database, determining whether the absolute value of the deviation between the identification value and the standard value is within an acceptable tolerance range. If the absolute value of the deviation between the pin count identification value and the standard pin count for the corresponding model in the connector standard specification database is less than a preset tolerance threshold, the pin count identification value is automatically corrected to that standard pin count. For example, if a connector's pin count identification value is 163, the standard pin count in the database is 164, and the preset tolerance threshold is 10, since |163-164|=1<10, the system automatically corrects 163 to 164. The tolerance mechanism is based on the fact that AI models inevitably have minute errors of ±1 to ±9 in industrial drawing recognition. These errors usually originate from occlusions or scanning distortions in the drawings, but do not change the actual specifications of the connectors. Through tolerance matching, these minute deviations are automatically corrected to the closest standard value, which not only eliminates AI recognition errors but also achieves accurate mapping from non-standard recognition values ​​to standard database values.

[0073] Subsequently, the pin count after tolerance matching is fused with the orientation parameters, pull tab parameters, and attribute text obtained from spatial correlation in the fused connector parameter set to obtain the corrected connector parameter set. Specifically, the pin count after tolerance matching has been corrected for standard values; the orientation and pull tab parameters are from the fused connector parameter set and have been confirmed through consistency verification; and the attribute text contains specification information such as model and impedance locked through spatial correlation. The fused corrected connector parameter set covers all the core parameters required for connector selection, including model, pin count, orientation, pull tab, and impedance, providing a standardized data foundation with complete structure and accurate parameters for subsequent generation of general specifications.

[0074] Through the above tolerance verification and parameter fusion, the system automatically corrects the numerical deviations that may exist in the AI ​​recognition process to standard values. At the same time, it integrates the scattered information from the visual modality, text modality and spatial correlation into a unified parameter set, effectively opening up a fault-tolerant closed loop from non-standard recognition to standard data.

[0075] S8. Generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity and output it.

[0076] In this step, a general specification model is generated based on the modified connector parameter set. This model is then matched against a standard material library to generate and output the connector entity's Bill of Materials (BOM), which includes: The pin count, orientation parameters, pull tab parameters, and attribute text in the corrected connector parameter set are merged according to the preset naming rules to generate a general specification model. Search the standard material library based on general specifications and models to obtain the corresponding ERP purchase part number; Generate and output a BOM containing ERP purchase part numbers, specifications, usage, and tag numbers.

[0077] Specifically, the modified connector parameter set obtained after tolerance verification and attribute integration contains all the core parameters required for connector selection, such as model number, number of pins, orientation, pull tab, and impedance. However, these parameters still exist in discrete field form and cannot be directly used for retrieval and matching in the standard material library. Therefore, it is necessary to generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity, and output it.

[0078] The specific process for generating general specifications and outputting the Bill of Materials is as follows: First, the pin count, orientation, pull tab, and attribute text from the modified connector parameter set are merged according to a preset naming rule to generate a generic specification model. The preset naming rule is a specification model coding rule predefined by the company based on connector industry naming conventions and internal material management specifications. This rule specifies the order of each parameter in the specification model string, the character encoding method for each parameter, and the format of the separators between parameters. For example, when the modified connector parameter set contains model "MCIO", pin count "74", orientation "RA", pull tab "PULLTAB", and impedance "85ohm", the system merges these discrete parameters into a generic specification model string "MCIO-74-RA-PULLTAB-85ohm" according to the preset naming rule. Each parameter is arranged in the order of "series name-pin count-orientation-accessory-impedance" and separated by hyphens. Through this merging operation, the previously scattered parameters are integrated into a unique, structured specification model identifier. This identifier can be directly interpreted by engineers and automatically recognized and matched by the company's material management system.

[0079] Subsequently, the system retrieves the corresponding ERP procurement part number from the standard material library based on the generated general specification model. The standard material library is a pre-established set of standard material master data in the enterprise resource planning system. Each material record contains at least a general specification model field and its corresponding ERP procurement part number field, and may also include detailed definition fields such as plating type, packaging method, temperature rating, and environmental attributes. The system uses the generated general specification model as the query key to perform an exact match query in the standard material library, locates the material record corresponding to that general specification model, and extracts the ERP procurement part number from that record. In cases where the same general specification model corresponds to multiple ERP procurement part numbers—that is, when connectors of the same specification have multiple part numbers due to differences in plating, packaging, or other derivative attributes—the system can further filter based on preset optimization rules or additional information from the input drawings to determine the most matching procurement part number.

[0080] Finally, the system generates and outputs a Bill of Materials (BOM) containing ERP-purchased part numbers, specifications, quantities, and tag numbers. The BOM is a core data file for enterprise manufacturing and material procurement, and its data structure includes at least a part number column, a specification description column, a single-board quantity column, and a tag number column. The system fills the retrieved ERP-purchased part number into the part number column, the corresponding specification description into the specification description column, and the quantity and tag number values ​​determined based on the annotation information in the design drawings or user input into the corresponding columns, forming a complete BOM list data record.

[0081] Through the above steps, the system converts the full-dimensional parameters after multimodal extraction, consistency verification, spatial correlation and tolerance verification into standardized general specifications and models. Then, through material library matching, it completes the mapping from specifications and models to ERP procurement part numbers. Finally, it outputs a BOM that can be directly imported into the enterprise resource planning system, realizing end-to-end automated processing from industrial engineering drawing input to structured BOM output.

[0082] Example 2

[0083] This embodiment uses a server motherboard design drawing package as an example to illustrate the complete process of automatically generating a connector BOM list using the system described in this invention. The drawing package contains two different types of connectors, connector A (MCIO series) and connector B (another series), located in different areas of the same key page, with slight obstruction in the wire reference table area for connector B.

[0084] I. Model Building and Training Phase

[0085] Before the system is put into practical application, the training and configuration of each deep learning model must be completed in advance.

[0086] For data preparation and enhancement, at least 5,000 engineering drawing pages containing connector diagrams were collected, covering samples from different manufacturers, with different resolutions and scanning qualities. Mosaic enhancement was applied to the training data, randomly stitching four images into one to enhance the model's ability to detect small targets; perspective transformation was also used to simulate scanning distortion, with the transformation angle randomly within ±5° to improve the model's generalization ability.

[0087] For key page localization model training, the first-level coarse screening model uses YOLOv8-Nano, trained for 100 epochs on a dataset annotated with tables, connector outlines, and bounding boxes, with an input image size of 640×640 and a batch size of 16. The second-level fine screening model is based on the ColPali architecture, using PaliGemma-3B as a base, and fine-tuned using the InfoNCE loss function to enable it to learn the projection rules and layout semantics of engineering drawings.

[0088] In terms of training the connector entity and wire position table detection model, the base model adopts Faster R-CNN, and the K-means++ clustering algorithm is used to generate 9 sets of prior anchor boxes on the annotation boxes of the training set to adapt to the slender shape of the connector, so as to optimize the detection accuracy.

[0089] For the training of the orientation and attachment classification model, the base model uses ResNet18 and performs transfer learning using ImageNet pre-trained weights. It is fine-tuned on a connector entity image dataset labeled with insertion orientation category (Straight / Right Angle) and pull strap category (With PullTab / With out PullTab).

[0090] For OCR and multimodal large model configuration, optical character recognition directly adopts the open-source PaddleOCR (PP-OCRv5) framework without additional training. Visual counting directly adopts the open-source Qwen2.5-VL multimodal large model (version 7B) without additional training.

[0091] II. Reasoning Application Stage

[0092] Step S1: Engineering Drawing Acquisition and Preprocessing

[0093] Input a 100-page PDF engineering drawing file of a server motherboard. The system performs page splitting on the input PDF file, breaking down the 100 pages into 100 independent single-page images; determines the original arrangement order of each page based on the file metadata; converts each single-page image to a standard grayscale format to reduce the computational load of subsequent processing; performs filtering and denoising processing on each grayscale image to remove salt-and-pepper noise and Gaussian noise introduced during scanning; performs geometric distortion correction on each denoised single-page image, eliminating ±3° tilt distortion that may have occurred during scanning through perspective transformation; and scales each distortion-corrected single-page image to a preset resolution of 300 DPI. After the above preprocessing, the original multi-page PDF file is converted into a standardized grayscale image set of 100 pages with uniform format, geometric correction, and consistent scale.

[0094] Step S2: Locating Key Drawing Pages

[0095] Input the 100 pages of pre-processed drawing images into the cascade positioning module.

[0096] In the first stage of coarse screening, the images of 100 pages of drawings were input one by one into the trained YOLOv8-Nano object detection model to detect whether basic primitives such as connector outlines or tables exist on each page. After the detection was completed, the detection bounding boxes of 5 pages had a confidence score higher than the preset threshold of 0.5, and these pages were marked as candidate pages.

[0097] In the second stage of fine-tuning, the images of the aforementioned five candidate pages were input into the finely tuned ColPali multi-vector retrieval model. ColPali encodes each page image into multiple local embedding vectors, approximately 300 patches per page, with each patch corresponding to a 128-dimensional vector. The system inputs a preset natural language query text: "This page must contain both a connector outline graphic and a table identifying connector-related attributes (pin count, model, impedance, etc.)." ColPali calculates the similarity score between the query vector and the local embedding vectors of each candidate page using the MaxSim maximum similarity matching mechanism. After comprehensive scoring, page 42 has the highest similarity score and is locked as the key drawing page.

[0098] Step S3: Connector Inspection and Cutting

[0099] The key drawing page 42 is input into the trained Faster R-CNN fine-grained object detection model. The model outputs detection boxes for two connector entities and two line position tables. The entity detection box for connector A is located in the central area of ​​the page, with its line position table below the entity; the entity detection box for connector B is located in the right area of ​​the page, with its line position table below the entity, although there is slight occlusion near pin 163 due to a crease in the drawing. Based on the coordinates of these detection boxes, the system crops the entity region image, the line position table region image, the entity region image, and the line position table region image of connector B from the original image on page 42.

[0100] Step S4: Dual-path parallel feature extraction

[0101] Feature extraction of connector A: In the visual feature inference flow, the cropped image of the connector A entity region is input into the ResNet18 classification model, and the output insertion direction is "RA" (Right Angle), and the zipper attachment is "with zipper". At the same time, the cropped image of the connector A line position table region and the preset counting prompt are input into the Qwen2.5-VL multimodal large model, and the output first pin count is 74.

[0102] In the text semantic parsing stream, PaddleOCR recognition is performed on the connector A entity region image to extract the character label "R / A", which is then mapped to the text direction "RA"; the character label "TAB" is also extracted, and the text "with pull strap" is mapped to the text "with pull strap". OCR recognition is performed on the connector A line position table region image to extract the first and last pin numbers "1" and "74", with a recognition confidence of 0.96, higher than the preset threshold of 0.9. The number of the second pin is calculated using the formula N=74-1+1=74. Simultaneously, the text "MCIO Gen5 85ohm" is recognized near this line position table.

[0103] The visual parameter set for connector A is {insertion direction: RA, pull strap accessory: pull strap present, number of first pins: 74}, and the text parameter set is {text direction: RA, text pull strap: pull strap present, number of second pins: 74}.

[0104] Feature extraction of connector B: In the visual feature inference flow, the cropped image of the connector B entity region is input into the ResNet18 classification model, and the output insertion direction is "STR" (Straight), with the zipper attachment being "no zipper". Simultaneously, the cropped image of the connector B line position table region, along with preset counting prompts, is input into the Qwen2.5-VL multimodal large model. Due to a crease occlusion near pin 163, the model outputs the first pin count as 163 (one pin was missed).

[0105] In the text semantic parsing stream, OCR recognition is performed on the connector B entity region image, extracting the character annotation "ST", which is then mapped to the text direction "STR". No keywords related to the pull strap are detected, and the text "no pull strap" is obtained through rule mapping. OCR recognition is performed on the connector B pin number table region image, extracting the first and last pin numbers "1" and "164". However, the "4" in "164" has an OCR confidence score of only 0.82 due to insufficient scanning clarity, which is lower than the preset threshold of 0.9. The system calculates the number of the second pin, 164, according to the formula N=164-1+1=164. At the same time, the text "SlimSAS85ohm" is recognized near the pin number table.

[0106] The visual parameter set for connector B is {insertion direction: STR, pull strap accessory: no pull strap, number of first pins: 163}, and the text parameter set is {text direction: STR, text pull strap: no pull strap, number of second pins: 164}.

[0107] Step S5: Consistency Verification and Conflict Resolution

[0108] Verification and decision-making for connector A: The visual parameter set and text parameter set of connector A are compared one by one in three sets: the insertion direction (RA) is consistent with the text direction (RA); the pull strap accessory (with pull strap) is consistent with the text pull strap (with pull strap); and the number of the first pin (74) is consistent with the number of the second pin (74). If all three sets of comparisons are consistent, the visual parameter set or text parameter set of connector A is directly output as the fused connector parameter set. The fused connector parameter set of connector A is {direction: RA, pull strap: with pull strap, number of pins: 74}.

[0109] Verification and decision-making for connector B: The visual parameter set and text parameter set of connector B are compared one by one in three sets: the insertion direction (STR) is consistent with the text direction (STR); the zipper attachment (no zipper) is consistent with the text zipper (no zipper); the number of the first pin (163) is inconsistent with the number of the second pin (164). Due to the inconsistency in the number of pins, a conflict resolution is triggered. The OCR recognition confidence is 0.82, which is not higher than the preset threshold of 0.9, and the confidence of all text content is not higher than the threshold. According to the preset modal priority strategy (Scenario2: text fuzziness back to vision), the system decides to accept the first pin number 163 in the visual parameter set. The connector parameter set of connector B is fused as {direction: STR, zipper: no zipper, number of pins: 163}.

[0110] Step S6: Spatial Association

[0111] Spatial association of connector A: Using the center of the connector A entity detection frame as a reference, a local search domain is constructed by expanding outward by 1.5 times. The Euclidean distance between all OCR-recognized text regions within this search domain and the center of the detection frame is calculated. After sorting by distance from nearest to farthest, the five closest texts are selected as candidate attribute texts, and keyword matching is performed on each one. The nearest neighbor text "MCIO Gen585ohm" successfully matches the model keyword "MCIO" and the impedance keyword "85ohm," and the system associates this attribute text with connector A.

[0112] Spatial association of connector B: Using the center of the detection frame of connector B as a reference, a local search domain is constructed by expanding outward by 1.5 times. The Euclidean distance between all OCR-recognized text regions within this search domain and the center of the detection frame is calculated. After sorting by distance from nearest to farthest, the five closest texts are selected as candidate attribute texts, and keyword matching is performed on each one. The nearest neighbor text "SlimSAS 85ohm" successfully matches the model keyword "SlimSAS" and the impedance keyword "85ohm," and the system associates this attribute text with connector B. Since the local search domains of both connectors are limited to a finite area around their respective entities and do not overlap, their attribute texts will not be mismatched.

[0113] Step S7: Tolerance verification and parameter fusion

[0114] Tolerance verification and integration of connector A: Extract the pin count identification value 74 from the connector A fusion connector parameter set and perform tolerance matching in the connector standard specification database. The database contains a standard pin count of 74 with an absolute deviation of 0, which is less than the preset tolerance threshold of 10, requiring no correction and remaining at 74. Merge the tolerance-matched pin count 74 with the direction parameter RA, the pull tab parameter "with pull tab" from the fusion connector parameter set, and the spatially correlated attribute texts "MCIO" and "85ohm" to obtain the corrected connector parameter set for connector A as {Model:MCIO, Pin Count: 74, Direction: RA, Pull Tab: With Pull Tab, Impedance: 85ohm}.

[0115] Tolerance verification and integration of connector B: The pin count identification value 163 is extracted from the connector B fusion connector parameter set and subjected to tolerance matching in the connector standard specification database. The database contains a standard pin count of 164, with an absolute deviation of |163-164|=1, which is less than the preset tolerance threshold of 10. The system automatically corrects 163 to 164. The tolerance-matched pin count 164 is then fused with the orientation parameter STR, the pull tab parameter "no pull tab" from the fusion connector parameter set, and the spatially correlated attribute texts "SlimSAS" and "85ohm" to obtain the corrected connector parameter set for connector B as {Model:SlimSAS, Pin Count:164, Orientation:STR, Pull Tab: No Pull Tab, Impedance:85ohm}.

[0116] This step demonstrates that when AI recognition is biased due to drawing occlusion, the system uses a dual protection mechanism of "visual backtracking plus tolerance correction". First, it backtracks to the visual result due to insufficient OCR confidence. Then, it automatically corrects the deviation caused by occlusion in visual recognition (163→164) to the standard value through tolerance matching. This effectively eliminates AI recognition error and establishes a fault-tolerant closed loop from non-standard recognition value to standard database value.

[0117] Step S8: BOM Generation and Output

[0118] For connector A, the parameters in the modified connector parameter set are merged according to a preset naming rule (series name - number of pins - orientation - accessories - impedance) to generate the general specification string "MCIO-74-RA-PULLTAB-85ohm". Using this general specification as the search key, the standard material library is searched, and an exact match is performed to obtain the corresponding ERP purchase part number "101-55678-001". Similarly, for connector B, the parameters in the modified connector parameter set are merged according to a preset naming rule to generate the general specification string "SlimSAS-164-STR-85ohm". Using this general specification as the search key, the standard material library is searched, and an exact match is performed to obtain the corresponding ERP purchase part number "101-67890-001".

[0119] Finally, the system generates and outputs a complete Bill of Materials (BOM) containing both connectors. Each record includes the ERP purchase part number, specification description, quantity, and tag number. The corresponding record for connector A is: part number "101-55678-001", specification description "MCIO-74-RA-PULLTAB-85ohm gold-plated tape", quantity "2", tag number "J3,J7". The corresponding record for connector B is: part number "101-67890-001", specification description "SlimSAS-164-STR-85ohm gold-plated tape", quantity "4", tag number "J1,J2,J5,J8".

[0120] At this point, the system has completed the fully automated process from inputting 100-page server motherboard PDF drawings to outputting a structured BOM list. The entire reasoning process, from drawing input to BOM output, takes approximately 15 seconds, a significant improvement in efficiency compared to the traditional method of manually flipping through pages, reading parameters, and entering data into the ERP system (which typically takes 20 to 40 minutes).

[0121] Example 3

[0122] This embodiment provides an automatic connector BOM generation system for industrial engineering drawings, including: The acquisition and preprocessing module is used to acquire industrial engineering drawings and preprocess them. The positioning module is used to locate key drawing pages containing connector specification parameters from preprocessed industrial engineering drawings using a cascading retrieval strategy. The detection and cropping module is used to detect connector entities and wire position tables in key drawing pages, and to crop out the connector entity area image and wire position table area image from the key drawing pages. The reasoning and parsing module is used to perform visual feature reasoning and text semantic parsing in parallel on the connector entity region image and the line position table region image, to obtain the connector visual parameter set and connector text parameter set, respectively. The consistency verification and conflict resolution module is used to perform consistency verification and conflict resolution on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set. The construction and association module is used to construct a local search domain based on the spatial location of the connector entity region image, and associate the attribute text within the local search domain with the connector entity; The tolerance verification module is used to perform tolerance verification on the numerical parameters in the fusion connector parameter set and integrate the attribute text into the verified parameter set to obtain the corrected connector parameter set. The generation and output module is used to generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity and output it.

[0123] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for automatically generating connector BOMs based on industrial engineering drawings, characterized in that, include: Obtain industrial engineering drawings and preprocess them; A cascading retrieval strategy is used to locate key drawing pages containing connector specification parameters from the preprocessed industrial engineering drawings. Detect the connector entity and wire position table in the key drawing page, and crop out the connector entity area image and wire position table area image from the key drawing page; Visual feature inference stream and text semantic parsing stream are executed in parallel on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set, respectively. The connector visual parameter set and connector text parameter set are subjected to consistency verification and conflict resolution to obtain the fused connector parameter set; A local search domain is constructed based on the spatial location of the connector entity region image, and the attribute text within the local search domain is associated with the connector entity. The numerical parameters in the fusion connector parameter set are subjected to tolerance verification, and the attribute text is incorporated into the verified parameter set to obtain the corrected connector parameter set. Generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity and output it.

2. The automatic connector BOM generation method for industrial engineering drawings according to claim 1, characterized in that, The step of locating key drawing pages containing connector specification parameters from the preprocessed industrial engineering drawings using a cascading retrieval strategy includes: Based on the object detection model, basic graphic elements are detected on all preprocessed drawing pages. Pages that detect preset basic graphic elements and have a confidence level higher than a preset threshold are marked as candidate pages. Based on the multi-vector vision-language retrieval model, the images of the candidate pages are encoded into multiple local embedding vectors. By calculating the similarity between the preset query vector and the local embedding vectors of each candidate page, the page with the highest score is selected as the key drawing page.

3. The automatic connector BOM generation method for industrial engineering drawings according to claim 1, characterized in that, The step of detecting the connector entity and wire position table in the key drawing page, and cropping the connector entity area image and wire position table area image from the key drawing page includes: Based on the target detection model, target detection is performed on the key drawing pages, and the detection boxes of the connector entities and the line position table are output. Based on the detection frame of the connector entity, the image of the connector entity area is cropped from the key drawing page; Based on the detection frame of the line position table, the line position table area image is cropped from the key drawing page.

4. The automatic connector BOM generation method for industrial engineering drawings according to claim 1, characterized in that, The parallel execution of visual feature inference stream and text semantic parsing stream on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set respectively includes: The image of the connector entity region is input into a first visual model, and the first visual model is used to perform structural classification on the image of the connector entity region to identify the insertion direction and pull strap accessories of the connector entity; the image of the wire position table region is input into a second visual model, and the second visual model is used to perform visual counting on the image of the wire position table region to obtain the number of first pins; Simultaneously, optical character recognition is performed on the connector entity area image to extract the direction and accessory character labels on the connector entity; optical character recognition is performed on the line position table area image to extract the first and last pin numbers and / or explicit pin number descriptions; semantic mapping is performed on the character labels according to preset rules to obtain the text direction and text pull tab of the connector entity; the second pin number is obtained by calculating or directly extracting the explicit pin number description based on the first and last pin numbers; The insertion direction, the zipper attachment, and the number of the first pins identified by the visual feature inference stream are used as the connector visual parameter set. The text direction, text pull, and number of the second pins parsed from the text semantic parsing stream are used as the connector text parameter set.

5. The automatic connector BOM generation method for industrial engineering drawings according to claim 4, characterized in that, The process of performing consistency verification and conflict resolution on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set includes: The insertion direction is compared with the text direction, the pull strap attachment is compared with the text pull strap, and the number of the first pins is compared with the number of the second pins; If the insertion direction is consistent with the text direction, the pull strap attachment is consistent with the text pull strap, and the number of the first pins is consistent with the number of the second pins, then the connector visual parameter set or the connector text parameter set is directly output as the fused connector parameter set. If the insertion direction is inconsistent with the text direction and / or the pull strap attachment is inconsistent with the text pull strap and / or the number of the first pin is inconsistent with the number of the second pin, then a conflict resolution is performed according to a preset modal priority strategy, and the resolution result is used as the fusion connector parameter set.

6. The automatic connector BOM generation method for industrial engineering drawings according to claim 5, characterized in that, The preset modal priority strategy includes: When the text direction or text pull strip contains strong semantic keywords and the corresponding optical character recognition confidence level is higher than a preset threshold, the corresponding parameter value that is inconsistent with the connector visual parameter set in the connector text parameter set is adopted. When the optical character recognition confidence level of all text content in the connector text parameter set is not higher than a preset threshold, the corresponding parameter value in the connector visual parameter set that is inconsistent with the connector text parameter set is adopted. When the deviation between the number of the first pin and the number of the second pin is within a preset unit range, the pin value conforming to the standard sequence is adopted.

7. The automatic connector BOM generation method for industrial engineering drawings according to claim 3, characterized in that, The step of constructing a local search domain based on the spatial location of the connector entity region image, and associating the attribute text within the local search domain with the connector entity, includes: Using the center of the detection box in the image of the connector entity region as a reference, a local search domain is constructed by expanding outward by a preset multiple; Calculate the distance between each text region within the local search domain and the center of the detection box; Select the nearest preset number of texts as candidate attribute texts, and lock the specification parameters of the connector entity through keyword matching.

8. The automatic connector BOM generation method for industrial engineering drawings according to claim 1, characterized in that, The process of performing tolerance verification on the numerical parameters in the fusion connector parameter set and incorporating the attribute text into the verified parameter set to obtain a corrected connector parameter set includes: Extract the pin count identification value from the parameter set of the fusion connector and perform tolerance matching in the connector standard specification database; if the absolute value of the deviation between the pin count identification value and the standard pin count of the corresponding model in the connector standard specification database is less than the preset tolerance threshold, then the pin count identification value is automatically corrected to the standard pin count. The pin count after tolerance matching is fused with the direction parameters, pull parameters, and attribute text in the fused connector parameter set to obtain the corrected connector parameter set.

9. The automatic connector BOM generation method for industrial engineering drawings according to claim 1, characterized in that, The process of generating a general specification model based on the modified connector parameter set, matching the general specification model with a standard material library, generating and outputting the BOM of the connector entity includes: The pin count, orientation parameters, pull tab parameters, and attribute text in the modified connector parameter set are merged according to a preset naming rule to generate a general specification model. Search the standard material library based on the general specifications and models to obtain the corresponding ERP procurement part number; Generate and output a BOM containing ERP purchase part numbers, specifications, usage, and tag numbers.

10. An automatic connector BOM generation system for industrial engineering drawings, characterized in that, include: The acquisition and preprocessing module is used to acquire industrial engineering drawings and preprocess the industrial engineering drawings. The positioning module is used to locate the key drawing page containing connector specification parameters from the preprocessed industrial engineering drawings using a cascading retrieval strategy. The detection and cropping module is used to detect the connector entity and the wire position table in the key drawing page, and to crop out the connector entity area image and the wire position table area image from the key drawing page. The reasoning and parsing module is used to perform visual feature reasoning and text semantic parsing in parallel on the connector entity region image and the line position table region image to obtain the connector visual parameter set and connector text parameter set, respectively. The consistency verification and conflict resolution module is used to perform consistency verification and conflict resolution on the connector visual parameter set and connector text parameter set to obtain the fused connector parameter set. The construction and association module is used to construct a local search domain based on the spatial location of the connector entity region image, and associate the attribute text within the local search domain with the connector entity; The tolerance verification module is used to perform tolerance verification on the numerical parameters in the fusion connector parameter set, and to integrate the attribute text into the verified parameter set to obtain the corrected connector parameter set. The generation and output module is used to generate a general specification model based on the modified connector parameter set, match the standard material library according to the general specification model, generate the BOM of the connector entity and output it.

Images