A connector intelligent detection and structured intermediate representation construction method and system for industrial engineering drawing BOM generation

Abstract

The application discloses a connector intelligent detection and structured intermediate representation construction method and system for industrial engineering drawing BOM generation. The method comprises the following steps: acquiring multiple pages of industrial engineering drawings and performing preprocessing; extracting engineering semantic features from the preprocessed drawings to calculate a correlation score, and screening the drawings with a score higher than a first preset threshold as a key page set; performing multi-level target detection on the pages in the set to obtain connector overall targets, connector end target and cable target; establishing an association between the overall target and the end target based on a preset engineering constraint rule, generating a connector candidate instance set; performing engineering semantic modulation and consistency constraint verification on the set based on the cable target; calculating the comprehensive confidence of each candidate instance according to the verification result to screen effective candidate instances, constructing a structured intermediate representation and outputting. The application realizes efficient and automatic construction of a standardized structured intermediate representation for BOM generation.

Description

Technical Field

[0001] This invention belongs to the field of image processing and intelligent parsing of industrial drawings. More specifically, it relates to a method and system for intelligent detection and structured intermediate representation construction of connectors for BOM generation of industrial engineering drawings. Background Technology

[0002] In the industrial manufacturing sector, product design information is typically delivered in the form of multi-page engineering drawings. These drawings include connection diagrams, component layout diagrams, interface definition diagrams, and numerous text and symbol annotations, serving as the core basis for connector selection, wire harness assembly, and bill of materials (BOM) generation. As the complexity of electronic products continues to increase, the number of engineering drawings for a single product can reach dozens or even hundreds of pages. These drawings are structurally complex, information-dense, costly to interpret manually, and easily affected by the experience level of the personnel.

[0003] In existing technologies, automated processing solutions for industrial engineering drawings mainly focus on the following areas: (I) Connector identification method based on target detection Existing technologies have attempted to apply deep learning object detection models to connector recognition tasks. For example, existing solutions use object detection algorithms such as YOLO to detect and locate connectors in images; other solutions use machine vision technology to extract and analyze the geometric features of connectors. However, all of the above solutions identify the connector as a whole object, failing to delve into the connector's end structure level. They cannot obtain fine-grained engineering structural information such as the number of interfaces and pin arrangement, making it difficult to support subsequent accurate matching and BOM field mapping.

[0004] (II) OCR-based method for extracting information from engineering drawings

[0005] Existing technologies also attempt to extract textual information from engineering drawings using OCR technology and generate a Bill of Materials (BOM) based on the extraction results. For example, existing solutions locate and identify the title block and parts list areas in engineering drawings, and parse the table content to generate a BOM. However, such solutions mainly rely on tabular text information in the drawings and lack effective means to identify connector structural information presented graphically, making it difficult to associate and parse graphical targets with text annotations.

[0006] (III) Component Inspection Method Based on Circuit Schematic

[0007] In the field of circuit design, existing solutions determine the corresponding netlist and BOM by analyzing the circuit schematic to check the components and their connections. However, these solutions are geared towards highly standardized circuit schematics. For industrial engineering drawings that contain a large number of non-standardized graphic symbols and free layouts, there is a lack of effective page filtering and key information location mechanisms.

[0008] In summary, existing technologies still have the following shortcomings in the application of connector information extraction and BOM generation for industrial engineering drawings: First, there is a lack of a page-level intelligent filtering mechanism tailored to the BOM generation target. Existing technologies often process all drawing pages uniformly, failing to effectively distinguish between critical pages highly relevant to connector detection and BOM generation, leading to wasted computational resources and increased risk of false positives. For multi-page engineering drawings containing dozens or even hundreds of pages, the lack of efficient page filtering methods directly impacts the overall processing efficiency of the system.

[0009] Second, the engineering semantics of connector inspection results are insufficient. Existing inspection methods typically identify connectors as single targets, ignoring engineering structural information such as connector ends, number of interfaces, and arrangement. Connectors in engineering drawings not only include their overall graphic outline but also multi-layered structural information such as end distribution, pin numbers, and cable connection relationships. Simple overall target detection cannot meet the analytical requirements at the engineering semantic level.

[0010] Third, the coupling between detection results and BOM generation process is low. Existing technologies often directly enter the image matching or manual verification stage after target detection, lacking a unified and structured intermediate representation layer, resulting in poor scalability and traceability of the overall process. Detection results cannot be directly transmitted to the BOM generation module in a standardized data format, and data exchange between different modules relies on manual conversion or customized interfaces, making it difficult to achieve automated closed-loop process.

[0011] Therefore, there is an urgent need for a method for intelligent detection and structured intermediate representation construction of connectors for BOM generation of industrial engineering drawings. This method can intelligently filter multi-page industrial engineering drawings by using engineering semantic features to remove irrelevant pages, extract fine-grained engineering semantics including end-to-end and cable connection relationships through multi-level target detection and engineering constraint rules, and construct a standardized structured intermediate representation, thereby realizing a fully automated closed loop for BOM generation. Summary of the Invention

[0012] The purpose of this invention is to propose a method and system for intelligent connector detection and structured intermediate representation construction for BOM generation from industrial engineering drawings. This addresses the problem that existing connector detection methods lack engineering semantic understanding capabilities, making it difficult to accurately extract fine-grained structural information from multi-page industrial engineering drawings and efficiently support BOM generation. The invention achieves intelligent multi-page drawing filtering based on engineering semantic features, fine-grained semantic parsing of connectors integrating multi-level target detection and engineering constraint verification, and efficient automated construction of standardized structured intermediate representations for BOM generation.

[0013] To achieve the above objectives, in a first aspect, this invention proposes a method for intelligent detection and structured intermediate representation construction of connectors for industrial engineering drawing BOM generation, comprising: Acquire multi-page industrial engineering drawings and perform preprocessing; Extract engineering semantic features from each page of the preprocessed industrial engineering drawings; The relevance score of each page of industrial engineering drawings is calculated based on the engineering semantic features, and industrial engineering drawings with a relevance score higher than a first preset threshold are selected as a set of key pages. Perform multi-level target detection on the pages in the set of key pages to obtain overall connector targets, connector end targets, and cable targets; Based on preset engineering constraint rules, the association between the overall target of the connector and the target of the connector end is established, and a set of candidate connector instances is generated. Based on the cable target, perform engineering semantic modulation and consistency constraint verification on the set of connector candidate instances; The overall confidence level of each connector candidate instance is calculated based on the verification results, and connector candidate instances with an overall confidence level lower than the second preset threshold are removed to obtain a connector instance set. A structured intermediate representation is constructed and output based on the set of connector instances.

[0014] Optionally, the preprocessing includes: Standardize drawing formats, resolution, and manage page indexes and labels.

[0015] Optionally, the engineering semantic features include: The text and annotation semantic features are obtained by performing OCR recognition on each page of industrial engineering drawings and matching the OCR recognition results with keywords in a preset engineering semantic dictionary. The text and annotation semantic features include at least one of the following: the number of occurrences of keywords related to connectors, the number of PIN annotations, and the completeness score of the interface number sequence. Graphic and structural features are obtained by identifying connector graphics and wire position table graphics in each page of industrial engineering drawings based on a target detection model. The graphic and structural features include at least one of the number of connector graphics and the number of wire position table graphics. The layout and engineering role characteristics are obtained by performing OCR recognition on each page of industrial engineering drawings and extracting the drawing type information from the page title bar, header or footer of each page of industrial engineering drawings. The layout and engineering role characteristics are used to determine the probability that the industrial engineering drawings belong to wiring diagrams, schematic diagrams or assembly instruction diagrams.

[0016] Optionally, the relevance score is calculated using the following expression:

[0017] in, This represents the normalized feature value of page i on the semantic features of the k-th type of project. This represents the weight coefficient of the corresponding engineering semantic features. This represents the total number of categories of engineering semantic features.

[0018] Optionally, performing multi-level target detection on the pages in the key page set to obtain the overall connector target, the connector end target, and the cable target includes: The first target detection model is used to detect each page in the key page set, identify the overall graphic region of the connector in the page, and output the spatial location, category label and initial confidence of each overall target of the connector. The second target detection model is used to detect each page in the key page set, identify the connector end graphic area and cable graphic area in the page, and output the spatial position of each connector end target and each cable target.

[0019] Optionally, the step of establishing the association between the overall connector target and the connector end target based on preset engineering constraint rules, and generating a set of candidate connector instances, includes: Obtain the spatial position of the overall connector target and the spatial position of the connector end target; Based on the preset engineering constraint rules, determine the attribution relationship between each connector end target and each overall connector target; Associating connector end targets belonging to the same overall target with that overall connector target creates connector candidate instances; A set of connector candidate instances is generated based on each of the aforementioned connector candidate instances; The engineering constraint rules include: Spatial inclusion constraint: The bounding box of the connector end target should be located within the bounding box of the overall connector target. This is determined by calculating whether the intersection-union ratio of the two is greater than or equal to a preset minimum overlap threshold. Spatial proximity constraint: Multiple connector end targets are spatially distributed in a continuous or adjacent manner with the same overall connector target. This is determined by calculating whether the minimum spatial distance between the connector end target and the overall connector target is less than or equal to a preset proximity threshold. Quantity consistency constraint: The difference between the number of detected end targets and the number of interface annotations identified by OCR recognition results shall not exceed the preset engineering error range.

[0020] Optionally, performing engineering semantic modulation and consistency constraint verification on the connector candidate instance set based on the cable target includes: Size consistency verification: For each connector candidate instance in the connector candidate instance set, the scale information is identified from the OCR recognition results, the bounding box size of the detected connector candidate instance is converted into the engineering size, compared with the preset engineering size range, and a size consistency score is generated. Topology consistency analysis: The skeleton of the cable target is extracted, including the start and end points. The spatial adjacency between the connector end target and the cable end point is analyzed. When the cable end point is located within the boundary area of ​​the connector end target or the distance to the connector end target boundary is less than a preset proximity distance, the cable target and the connector end target are considered to have a valid connection, and the proportion of valid connection ends is calculated. ,in The target number of connector ends for effective connection. This represents the total number of connector end targets detected.

[0021] Optionally, the expression for calculating the overall confidence level is:

[0022] in, The initial confidence level for the overall target of the connector. Scoring for dimensional consistency To ensure effective connection of the end proportions, , , These are the corresponding weighting coefficients.

[0023] Optionally, constructing a structured intermediate representation based on the set of connector instances includes: The page source identifier and spatial location coordinates of each connector instance are encoded into spatial and page context encoding information; The structural features of each connector instance are encoded into engineering structural feature modeling information, and the structural features include at least the number of connector pins, pin arrangement, and external dimension parameters. The engineering attributes extracted from the OCR recognition results are encoded into engineering semantic attribute description information, and the engineering attributes include at least the connector model and / or material code.

[0024] Secondly, this invention proposes a connector intelligent detection and structured intermediate representation construction system for generating industrial engineering drawings (BOMs), comprising: The drawing input module is used to acquire multi-page industrial engineering drawings and perform preprocessing. The engineering semantic feature extraction module is used to extract engineering semantic features from each page of preprocessed industrial engineering drawings. The key page intelligent positioning module is used to calculate the relevance score of each page of industrial engineering drawings based on the engineering semantic features, and to filter the industrial engineering drawings with the relevance score higher than a first preset threshold as a set of key pages. A multi-level target detection module is used to perform multi-level target detection on the pages in the set of key pages to obtain the overall connector target, the connector end target, and the cable target; The connector candidate instance generation module is used to establish the association between the overall target of the connector and the target of the connector end based on preset engineering constraint rules, and generate a set of connector candidate instances; The engineering semantic modulation and consistency constraint verification module is used to perform engineering semantic modulation and consistency constraint verification on the candidate instance set of connectors based on the cable target. The comprehensive confidence calculation and filtering module is used to calculate the comprehensive confidence of each connector candidate instance based on the verification results, and remove the connector candidate instances whose comprehensive confidence is lower than the second preset threshold to obtain a connector instance set. The structured intermediate representation construction and output module is used to construct and output a structured intermediate representation based on the set of connector instances.

[0025] The beneficial effects of this invention are as follows: By acquiring multiple pages of industrial engineering drawings and performing preprocessing, engineering semantic features are extracted from each page of the drawings and a relevance score is calculated. Pages with scores higher than a first preset threshold are selected as a set of key pages, thereby effectively filtering out pages unrelated to connector detection, significantly reducing the computational burden of subsequent multi-level target detection, and improving the overall operating efficiency of the system. By performing multi-level target detection on the set of key pages, the overall connector target, end target, and cable target are acquired simultaneously. Based on engineering constraint rules, the association between the overall target and the end target is established, so that the connector detection results not only include overall location information, but also fine-grained engineering structural information such as end attribution, number of interfaces, and arrangement, overcoming the limitation of existing technologies that identify connectors as single targets, and providing accurate structure for subsequent BOM generation. The system establishes a solid data foundation. By performing engineering semantic modulation and consistency constraint verification on the candidate connector instance set based on cable targets, the detection results are verified and screened at the engineering semantic level from two dimensions: size consistency and topological relationship consistency. This effectively eliminates false detection results that do not conform to engineering specifications. In addition, candidate instances below the second preset threshold are eliminated by combining comprehensive confidence calculation, further improving the engineering credibility and reliability of the detection results. Finally, a structured intermediate representation is constructed and output based on the verified connector instance set. The detection results are transformed into a standardized data structure containing spatial location, engineering structural features, and engineering semantic attributes. This decouples the process between the connector detection stage and the BOM generation stage, provides a unified data interface for the downstream image matching and BOM generation modules, and enables the overall system to have good scalability and traceability.

[0026] The system of the present invention has other features and advantages that will be apparent from or will be set forth in detail in the accompanying drawings and following detailed description, which together serve to explain the particular principles of the invention. Attached Figure Description

[0027] The above and other objects, features and advantages of the present invention will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0028] Figure 1 A flowchart illustrating the steps of a connector intelligent detection and structured intermediate representation construction method for generating industrial engineering drawings BOMs according to Embodiment 1 of the present invention is shown. Detailed Implementation

[0029] The invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0030] Example 1

[0031] like Figure 1 As shown, this embodiment provides a method for intelligent detection and structured intermediate representation construction of connectors for industrial engineering drawing BOM generation, including: S1. Obtain multiple pages of industrial engineering drawings and perform preprocessing; Specifically, industrial engineering drawings originate from digital files delivered as product designs. These files can be in formats including, but are not limited to, PDF, TIFF, JPEG, PNG, or bitmap images exported from CAD. Each page of the drawing collectively constitutes a structured description of the product's connector layout, wiring relationships, and interface definitions. A unified preprocessing operation is performed after the drawings are acquired.

[0032] In this step, preprocessing includes: Standardize drawing formats, resolution, and manage page indexes and labels.

[0033] Specifically, the preprocessing first performs drawing format standardization, converting original drawing files from different sources and in different formats into a unified image format to eliminate the impact of format differences on subsequent processing. Next, resolution normalization is performed, scaling the images of each page of the unified-format drawing to a preset standard resolution, ensuring all pages participate in subsequent calculations at the same spatial scale and avoiding feature representation deviations caused by resolution inconsistencies. Finally, page indexing and identification management are performed, assigning a unique page identifier to each page of the drawing and establishing a logical order index between pages. The page number of each page and its relative position within the multi-page document are recorded, allowing any subsequent processing step to trace back to the specific page of the original drawing through this identification information, and ensuring accurate identification of the contextual relationships between pages during cross-page analysis. After the above preprocessing, each page of the drawing, with a unified format, unified resolution, and complete index identification status, enters the subsequent engineering semantic feature extraction stage, providing basic data support for cross-page analysis and processing.

[0034] S2. Extract engineering semantic features from each page of the preprocessed industrial engineering drawings; In this step, the engineering semantic features include: The semantic features of the text and annotations are obtained by performing OCR recognition on each page of industrial engineering drawings and matching the OCR recognition results with keywords in a preset engineering semantic dictionary. The semantic features of the text and annotations include at least one of the following: the number of occurrences of keywords related to connectors, the number of PIN annotations, and the completeness score of the interface number sequence. Graphic and structural features are obtained by identifying connector graphics and wire position table graphics in each page of industrial engineering drawings based on a target detection model. Graphic and structural features include at least one of the number of connector graphics and the number of wire position table graphics. The layout and engineering role features are obtained by performing OCR recognition on each page of industrial engineering drawings and extracting the drawing type information from the page title bar, header or footer of each page. The layout and engineering role features are used to determine the probability that the industrial engineering drawings belong to wiring diagrams, schematic diagrams or assembly instruction diagrams.

[0035] Specifically, the engineering semantic features are composed of three categories: text and annotation semantic features, graphic and structural features, and layout and engineering role features. These features are used to characterize the relevance of each page of drawings to the connector inspection task from different dimensions.

[0036] The extraction method for text and annotation semantic features is as follows: OCR recognition is performed on each page of the industrial engineering drawing to obtain all recognizable text content on the page. This text content includes title bar text, technical annotations, interface numbers, component labels, and annotations. Then, the OCR recognition results are matched with a pre-set engineering semantic dictionary for keyword matching. This dictionary contains at least keywords related to connectors and their synonyms, including "connector," "plug," "CONNECTOR," "PIN," and interface numbers from "J1" to "J9," and from "P1" to "P9." The frequency of mentions of connector-related terms on the page is obtained by counting the occurrences of the matched keywords. Interface number sequence patterns (such as "PIN1, PIN2, ..." or "J1, J2, ...") are identified using regular expressions to detect the continuity and completeness of interface numbers, forming an interface number sequence completeness score. The number of PIN annotations is obtained by detecting the occurrences of interface annotation symbols (such as "PIN" followed by numbers). The frequency of keyword occurrences, the number of PIN annotations, and the completeness score of the interface number sequence together constitute the semantic feature vector of text and annotations, which is used to characterize the degree of relevance between the page and the connector information at the text semantic level.

[0037] The extraction method for graphic and structural features is as follows: each page of industrial engineering drawings is initially detected by a target detection model (which can be a deep learning-based model, such as the lightweight YOLO model) to identify the connector graphic area and the line position table graphic area in the page; the number of detected connector graphics and the number of detected line position table graphics are counted; the number of connector graphics and the number of line position table graphics together constitute the graphic and structural feature vector, which is used to characterize the degree of correlation between the page and the connector information at the graphic content level.

[0038] The extraction method for layout and engineering role features is as follows: OCR recognition is performed on each page of industrial engineering drawings to locate and extract drawing type information from the page title bar, header, or footer area. Drawing type information includes keywords such as "wiring diagram," "wiring diagram," "schematic diagram," "schematic," "assembly instruction diagram," and "assembly." By detecting the occurrence of these keywords in the title bar, header, or footer area, the probability that the page belongs to a wiring diagram, a schematic diagram, or an assembly instruction diagram is calculated. These probabilities together constitute the layout and engineering role feature vector, used to characterize the degree of correlation between the page and connector information at the document structure level.

[0039] The three types of features described above represent each page of drawings from three dimensions: text content, graphic distribution, and document role, forming a complete page-level engineering semantic feature space, which provides multi-dimensional feature input for subsequent intelligent positioning of key pages.

[0040] S3. Calculate the relevance score of each page of industrial engineering drawings based on the semantic features of the engineering project, and select the industrial engineering drawings with a relevance score higher than the first preset threshold as the key page set. Specifically, the relevance score measures the degree of relevance between each page of the drawing and the connector inspection and BOM generation tasks. For each page of industrial engineering drawing, the relevance score is obtained by multiplying the normalized three types of feature values ​​by their corresponding weight coefficients and summing them. The higher the score, the stronger the relevance of the page of drawing to the connector inspection task at the engineering semantic level.

[0041] After obtaining the relevance scores of all pages, the score of each page is compared with a preset first threshold. Industrial engineering drawings with scores higher than the first preset threshold are identified as key pages containing connector-related engineering semantic information, and these key pages are aggregated into a key page set. Simultaneously, all pages can be sorted in descending order based on their relevance scores, and the sorting results can be output to allow for dynamic adjustment of the number of key pages selected during subsequent processing as needed.

[0042] By employing a weighted scoring and threshold filtering mechanism, key pages highly relevant to connector inspection and BOM generation can be quickly located from multiple pages of industrial engineering drawings. This effectively reduces the number of pages that subsequent multi-level target inspection modules need to process, thereby lowering the overall computational burden of the system.

[0043] In this step, the formula for calculating the relevance score is:

[0044] in, This represents the normalized feature value of page i on the semantic features of the k-th type of project. This represents the weight coefficient of the corresponding engineering semantic features. This represents the total number of categories of engineering semantic features.

[0045] Specifically, i represents the index number of the drawing page, k represents the category index of the engineering semantic features, and K is the total number of categories of engineering semantic features. In this embodiment, K=3, which correspond to three categories: text and annotation semantic features, graphic and structural features, and layout and engineering role features, respectively. This represents the normalized feature value of the i-th page on the k-th type of engineering semantic features, used to characterize the quantitative performance of the drawing on the corresponding feature dimension; This represents the weight coefficient corresponding to the semantic features of the k-th class of engineering projects, used to adjust the contribution of different features to the overall score.

[0046] Normalization can be achieved by using min-max normalization, which maps the text and annotation semantic features, graphic and structural features, and layout and engineering role features extracted in the previous step to the [0,1] interval, in order to eliminate the influence of differences in the units and numerical ranges of different features on the scoring results.

[0047] Weighting coefficient The value can be preset according to engineering experience rules, or it can be obtained by training through machine learning methods based on historical labeled data. For example, linear regression or logistic regression models can be used to train historical drawing data labeled with "critical page" or "non-critical page" to learn the optimal weight allocation.

[0048] S4. Perform multi-level target detection on the pages in the key page set to obtain the overall connector target, connector end target, and cable target; In this step, multi-level target detection is performed on the pages in the key page set to obtain overall connector targets, connector end targets, and cable targets, including: The first target detection model is used to detect each page in the key page set, identify the overall graphic region of the connector in the page, and output the spatial location, category label and initial confidence of each connector overall target. The second target detection model is used to detect each page in the key page set, identify the connector end graphic area and cable graphic area in the page, and output the spatial position of each connector end target and each cable target.

[0049] Specifically, the multi-level target detection adopts a two-level detection model in series to progressively identify and extract the overall structure of the connector, end details, and cable connection relationships in the key pages.

[0050] The first target detection model is used to detect each page in the key page set, identifying the overall graphic region of the connector within each page. The overall graphic region of the connector refers to the graphical area representing the outline of the connector's main body, including the connector's shell shape, mounting flange, locking structure, and other overall engineering structures. The first target detection model outputs the spatial location, category label, and initial confidence score of each overall connector target. The spatial location is represented by the coordinates of the target bounding box, used to determine the precise position of the overall connector in the page image; the category label identifies the type of connector, such as a circular aviation plug or a rectangular industrial connector; the initial confidence score characterizes the reliability of the detection result, providing basic data for subsequent comprehensive confidence score calculations.

[0051] The second target detection model detects each page in the key page set, identifying connector end graphic regions and cable graphic regions within each page. The connector end graphic region refers to the interface end graphic of the connector used for electrical connection, including pins, sockets, contacts, and other end structures. The cable graphic region is the graphical representation of the wires or wire harnesses connected to the connector end. The second target detection model outputs the spatial positions of each connector end target and each cable target. The spatial positions of the end targets are used to determine the distribution of each end within the page, providing a locational basis for determining the attribution of the end to the overall target; the spatial positions of the cable targets are used for skeleton extraction and endpoint localization in subsequent topology consistency analysis. End target detection and cable target detection employ a parallel detection architecture with a shared feature extraction network, where the same second target detection model simultaneously identifies both end graphic regions and cable graphic regions, reducing computational overhead while ensuring detection accuracy.

[0052] Through the cascaded processing of the above two-level detection models, three types of detection results were obtained from the key pages: overall connector target, end target, and cable target. This provides a complete target data foundation for subsequent correlation verification of engineering constraint rules and topology consistency analysis.

[0053] S5. Based on preset engineering constraint rules, establish the association between the overall target of the connector and the target of the connector end, and generate a set of candidate connector instances; In this step, the association between the overall connector target and the connector end target is established based on preset engineering constraint rules, generating a set of candidate connector instances, including: Obtain the spatial position of the overall connector target and the spatial position of the connector end target; Based on the preset engineering constraint rules, determine the attribution relationship between each connector end target and each overall connector target; Associating connector end targets belonging to the same overall target with that overall connector target creates connector candidate instances; A set of connector candidate instances is generated based on each connector candidate instance; Among them, the engineering constraint rules include: Spatial inclusion constraint: The bounding box of the connector end target should be located within the bounding box of the overall connector target. This is determined by calculating whether the intersection-union ratio of the two is greater than or equal to a preset minimum overlap threshold. Spatial proximity constraint: Multiple connector end targets are spatially distributed in a continuous or adjacent manner with the same overall connector target. This is determined by calculating whether the minimum spatial distance between the connector end target and the overall connector target is less than or equal to a preset proximity threshold. Quantity consistency constraint: The difference between the number of detected end targets and the number of interface annotations identified by OCR recognition results shall not exceed the preset engineering error range.

[0054] Specifically, after obtaining the spatial location of the overall connector target and the spatial location of the connector end target, it is necessary to determine the attribution relationship between each connector end target and each overall connector target according to the preset engineering constraint rules, and associate the end targets belonging to the same overall connector target with the overall target to form a structurally complete connector candidate instance.

[0055] Engineering constraints include three categories: spatial inclusion constraints, spatial proximity constraints, and quantity consistency constraints. These three types of constraints verify the matching relationship between the endpoint and the whole from different dimensions: Spatial inclusion constraints are used to determine whether a connector tip belongs to a larger connector object. In engineering drawings, the connector tip is typically located within the internal region of the overall connector outline. This constraint is determined by calculating the intersection-union ratio (IU) between the bounding box of the connector tip and the bounding box of the larger connector object, and whether this IU is greater than or equal to a preset minimum overlap threshold. To determine this, if the intersection-union ratio (IUU) of the bounding box of the endpoint target and the bounding box of the overall target satisfies... If so, then the target at the end is determined to belong to the whole target in terms of spatial inclusion; where, The bounding box representing the end target. The bounding box representing the overall target of the connector. This is the preset minimum overlap threshold.

[0056] Spatial proximity constraints are used to assist in determining the affiliation relationship between end targets and the overall connector target, especially when the end bounding box and the overall bounding box do not completely overlap. This is achieved by calculating their spatial proximity to supplement the determination of the affiliation relationship. In engineering drawings, multiple end targets belonging to the same connector are spatially distributed continuously or adjacently to the overall connector target. This constraint calculates whether the minimum spatial distance between the bounding box of the connector end target and the bounding box of the overall connector target is less than or equal to a preset proximity threshold. To determine this. If the minimum spatial distance between the end target and the overall connector target satisfies... Then, it is determined that the end target belongs to the overall target of the connector in terms of spatial proximity; where, The bounding box representing the end target. d(·) represents the bounding box of the overall connector target, and d(·) represents the minimum spatial distance between the end target bounding box and the overall connector target bounding box (i.e., the shortest Euclidean distance between the two bounding boxes). This is a preset proximity threshold.

[0057] The quantity consistency constraint is used to verify the rationality of connector attribution relationships from a quantity perspective. It compares the number of detected connector targets with the number of interface labels identified through OCR text recognition. In engineering drawings, the number of connector ends is usually consistent with the number of their corresponding interface labels. This constraint checks whether the absolute value of the difference between the detected number of connector targets and the number of interface labels identified through OCR text recognition does not exceed a preset engineering error range. To determine. If the difference between the two satisfies If the current endpoint is consistent in terms of quantity, then the quantity consistency check is passed; where, The number of detected end targets. This refers to the number of interface annotations identified through OCR text recognition (i.e., the total number of annotations corresponding to the interface number sequence extracted from the drawing text through OCR recognition). This is a preset engineering error range, designed to tolerate minor quantity deviations caused by incomplete drawing annotations or missed inspections. If the difference between the two exceeds... If there are omissions, false detections, or incomplete labeling, the candidate instance of the connector needs to be marked or removed to ensure the accuracy of subsequent processing.

[0058] The above three types of constraints together constitute the decision-making basis for determining the attribution of the endpoint and the whole. In the specific implementation process, the three types of constraints can be used in combination: first, a preliminary screening is carried out through spatial inclusion constraints; then, a supplementary judgment is made on boundary conditions (the case where the endpoint bounding box and the whole bounding box partially overlap or are close to each other but not completely contained) through spatial proximity constraints; finally, the overall rationality of all association results is verified through quantity consistency constraints, forming a complete attribution judgment process from local matching to overall verification.

[0059] After determining the attribution relationships between each connector end target and each overall target, connector end targets belonging to the same overall target are associated with that overall connector target to form connector candidate instances. A connector candidate instance contains an overall connector target and all its associated end targets. The bounding box of the overall target serves as the main bounding box of the instance, and the spatial location, category, and other information of the end targets are attached to the overall target as structural attributes of the instance. Based on these connector candidate instances, a set of connector candidate instances is generated, which serves as input data for subsequent engineering semantic modulation and consistency constraint verification.

[0060] S6. Perform engineering semantic modulation and consistency constraint verification on the candidate instance set of connectors based on the cable target; In this step, performing engineering semantic modulation and consistency constraint verification on the connector candidate instance set based on the cable target includes: Size consistency verification: For each connector candidate instance in the connector candidate instance set, the scale information is identified from the OCR recognition results, the bounding box size of the detected connector candidate instance is converted into the engineering size, compared with the preset engineering size range, and a size consistency score is generated. Topology consistency analysis: The skeleton of the cable target is extracted, including the start and end points. The spatial adjacency between the connector end target and the cable end point is analyzed. When the cable end point is located within the boundary area of ​​the connector end target or the distance to the connector end target boundary is less than a preset proximity distance, the cable target and the connector end target are considered to have a valid connection, and the proportion of valid connection ends is calculated. ,in The target number of connector ends for effective connection. This represents the total number of connector end targets detected.

[0061] Specifically, based on the cable target, engineering semantic modulation and consistency constraint verification are performed on the connector candidate instance set. The candidate instances are verified and screened at the engineering semantic level from two dimensions: size consistency and topological relationship consistency. The object of this step is the connector candidate instance set. Each candidate instance contains the overall connector target and its associated end target. The actual connection relationship between the end and the cable is further verified through topological analysis of the cable target.

[0062] Dimensional consistency verification is used to confirm whether the graphic dimensions of connector candidate instances conform to the dimensional specifications in real engineering scenarios. For each connector candidate instance in the candidate instance set, the scale information of the page is first identified from the OCR recognition results. The scale information is usually marked in the title block, frame area, or near the connector graphic in the form of "1:1", "1:2", "2:1", etc. Then, based on the identified scale information, the bounding box size (i.e., the pixel size in the image coordinate system) of the detected connector candidate instance is converted into engineering dimensions. The converted engineering dimensions are compared with a preset engineering dimension range, which is pre-set according to the actual physical size range of common connectors in industrial engineering drawings. A dimensional consistency score is generated based on the comparison result. A higher score indicates that the engineering dimensions of the candidate instance are closer to the preset range. When the engineering dimensions of the connector candidate instance are within the preset engineering dimension range, the dimensional consistency score is 1; when the engineering dimensions deviate from the preset engineering dimension range, the score decreases accordingly based on the degree of deviation, with the score decreasing as the deviation increases. Dimensional consistency verification is used to filter out detection results with obvious dimensional anomalies caused by inconsistent drawing scaling, image distortion, or false detections.

[0063] Topology consistency analysis is used to verify the actual connection relationship between connector end targets and cable targets, ensuring that the end targets and cables form a valid electrical connection in spatial topology. A skeleton extraction is performed on the cable targets, refining the cable's strip-shaped region into single-pixel-width lines to accurately extract the cable's geometric topology. After skeleton extraction, the start and end points of the cable are extracted from the two endpoints of the skeleton lines. The spatial adjacency relationship between each connector end target and each cable end target is analyzed. Specifically, the spatial distance between the cable end target and the end target's bounding box is calculated. When the cable end target is located inside the end target's bounding box, or when the spatial distance between the cable end target and the end target's boundary is less than a preset proximity distance, the cable target is considered to have a valid connection with the end target. After performing the above judgment on all end targets, the number of validly connected end targets is counted. And calculate the proportion of effective connection ends. ,in This represents the total number of detected end targets. The value is between 0 and 1. A higher C_t indicates a higher actual connection rate between the cable and the terminal in the candidate instance. The proportion of effectively connected terminals, C_t, is an important indicator of the structural integrity of the candidate instance and is used in subsequent calculations of the overall confidence level: if all terminal targets in the candidate instance form an effective connection with the cable, then... =1 indicates that the instance is completely identical in topology; if some end targets have no cable connection, then A value less than 1 indicates that the instance may have missed detections or incomplete drawing information, and adjustments need to be made at the confidence level.

[0064] By verifying dimensional consistency and analyzing topological relationships, each candidate instance in the connector candidate instance set is validated from two dimensions: engineering dimensions and topological connections. Dimensional consistency scoring. and effective connection end ratio The engineering credibility of candidate instances is quantitatively evaluated from two perspectives: graphic size and structural connectivity, providing engineering semantic level scoring input for subsequent comprehensive confidence calculation.

[0065] S7. Calculate the overall confidence level of each candidate connector instance based on the verification results, and remove candidate connector instances with an overall confidence level lower than the second preset threshold to obtain a set of connector instances. Specifically, this step involves executing the set of candidate connector instances after dimensional consistency verification and topological consistency analysis. Each candidate connector instance has received two engineering semantic scores: a dimensional consistency score and a valid connection end ratio. The comprehensive confidence score is used to comprehensively evaluate each candidate connector instance from two dimensions: detection confidence and engineering semantic confidence, in order to select engineering-reliable candidate instances to proceed to the subsequent structured representation construction stage.

[0066] After calculating the overall confidence score of each candidate connector instance, it is compared with a preset second threshold. The overall confidence score ranges from 0 to 1, with a higher value indicating better overall performance in terms of detection reliability and engineering structure consistency. When the overall confidence score is greater than or equal to the second preset threshold, the candidate connector instance is determined to be engineering reliable and retained; when the overall confidence score is less than the second preset threshold, the candidate connector instance is determined to be engineering unreliable and eliminated.

[0067] The second preset threshold is used to control the stringency of screening reliable connectors in engineering projects. Setting the threshold too high may result in the false rejection of some valid connectors, reducing recall; setting it too low may result in the retention of some invalid detection results, affecting the accuracy of subsequent BOM generation. In practical applications, an appropriate second preset threshold can be pre-set based on the specific characteristics of the industrial engineering drawings and the different accuracy and recall requirements of the downstream BOM generation task. Alternatively, the optimal threshold can be determined through statistical analysis of historical annotation data.

[0068] Through comprehensive confidence calculation and threshold elimination, candidate connector instances with a comprehensive confidence level not lower than the second preset threshold are retained. These retained candidate instances are identified as connector instances and are aggregated to form a connector instance set. Each instance in the connector instance set has passed the initial confidence verification at the detection level, the size consistency verification at the engineering level, and the topological connection relationship verification, possessing high engineering credibility. It can serve as a reliable data source for subsequent structured intermediate representation construction, ensuring that the final output structured data can effectively support the BOM generation task.

[0069] In this step, the formula for calculating the overall confidence level is:

[0070] in, The initial confidence level for the overall target of the connector. Scoring for dimensional consistency To ensure effective connection of the end proportions, , , These are the corresponding weighting coefficients.

[0071] Specifically, The initial confidence score of the overall connector target output by the first target detection model comes from the recognition result of the first target detection model on the overall graphic region of the connector in the multi-level target detection stage, and is used to characterize the credibility of the detection result at the visual level. The dimensional consistency score, derived from dimensional consistency verification, is used to characterize the degree of conformity between the engineering dimensions of the connector candidate instance and the preset engineering dimension range. The effective connection end ratio is derived from topology consistency analysis and is used to characterize the actual connection rate between cables and ends in connector candidate instances; , , These are the corresponding weighting coefficients, used to adjust the contribution ratio of the three types of confidence in the overall confidence. The specific values ​​of the weighting coefficients can be configured according to the actual needs of the engineering scenario.

[0072] S8. Construct and output a structured intermediate representation based on the set of connector instances.

[0073] Specifically, the structured intermediate representation serves as a data bridge between the connector detection stage and the BOM generation stage. It is used to convert the set of connector instances obtained after multi-level target detection, engineering constraint association, engineering semantic modulation and consistency constraint verification, and comprehensive confidence screening into a unified and standardized structured data format, providing directly parsable data input for the downstream image matching module and BOM generation module.

[0074] The construction process of the structured intermediate representation is as follows: For each connector instance in the connector instance set, firstly, spatial and page context encoding information is generated based on the page source identifier and spatial location coordinates of the instance. The page source identifier comes from the page index and identifier management system established in the preprocessing stage, and the spatial location coordinates come from the overall target bounding box position information of the connector output in the multi-level target detection stage. Then, based on the total number of end targets associated with the instance, the spatial distribution relationship of the end targets, and the engineering dimensions converted in the size consistency verification, engineering structural feature modeling information including the number of pins, pin arrangement, and external dimension parameters is generated. Finally, engineering attributes such as connector model and material code are extracted from the OCR recognition results of the page to which the instance belongs, and engineering semantic attribute description information is generated.

[0075] The three types of encoded information described above are combined and encapsulated according to a preset data structure to form a structured intermediate representation of the connector instance. The above construction operation is performed on each connector instance in the connector instance set, and the structured intermediate representations of all connector instances are aggregated into a unified dataset. This dataset is then output to the downstream image matching module or BOM generation module in a standardized data format (such as JSON or XML) through the data output interface module. The downstream module can directly obtain complete information about each connector instance, such as page source, spatial location, number of pins, pin arrangement, dimensions, connector model, and material code, by parsing this structured intermediate representation. This eliminates the need to re-parse the original drawings or perform manual data conversion, thereby achieving process decoupling and standardized data flow between the connector testing stage and the BOM generation stage.

[0076] In this step, constructing a structured intermediate representation based on the set of connector instances includes: The page source identifier and spatial location coordinates of each connector instance are encoded into spatial and page context encoding information; The structural features of each connector instance are encoded into engineering structural feature modeling information. The structural features include at least the number of connector pins, pin arrangement, and external dimensions. The engineering attributes extracted from the OCR recognition results will be encoded into engineering semantic attribute description information. The engineering attributes will include at least the connector model and / or material code.

[0077] Specifically, the spatial and page context encoding information is used to record the positioning information of each connector instance in the original drawing space, including two parts: page source identifier and spatial coordinates. The page source identifier records the drawing page number to which the connector instance belongs, as well as its sequential position in the original multi-page document. The page source identifier originates from the page index and identifier management system established in the preprocessing stage. Each page is assigned a unique page identifier during the preprocessing stage, and the page number order of the page in the multi-page document is recorded. This identifier information flows along with the page data during the key page screening and multi-level target detection stages. When the connector instance is constructed, its page source identifier is naturally inherited from the identifier information of the page to which it belongs, without the need for additional association operations. The spatial coordinates record the bounding box position information of the connector instance in the image of its page, represented by four parameters: the horizontal coordinate of the upper left corner, the vertical coordinate of the upper left corner, the width, and the height of the bounding box. This coordinate information directly comes from the spatial position of the overall connector target output by the first target detection model in the multi-level target detection stage. By combining page source identifiers and spatial location coordinates, any connector instance in the structured intermediate representation can be accurately located back to its corresponding position in the original drawing in subsequent processing steps, thus achieving traceability between the detection results and the original data.

[0078] The engineering structural feature modeling information is used to abstract and encode the structural features of each connector instance, including at least the number of connector pins, pin arrangement, and external dimensions. The number of pins is obtained by counting the total number of end targets belonging to the same connector instance; the number of end targets belonging to the same connector instance is the number of pins for that connector. The pin arrangement is determined by the spatial distribution relationship of the end targets belonging to the same connector instance. Specifically, it can be identified by the spatial distribution geometric features of the center points of the end target bounding boxes (such as arranged in a row, column, or matrix distribution), or by the arrangement pattern of the interface number sequence in the OCR recognition results. The external dimensions are provided by the engineering dimensions of the connector instance's bounding box, which are derived from the bounding box dimensions converted based on the scale information during dimensional consistency verification. These structural features characterize the connector's engineering structural characteristics from three aspects: the number of pins, spatial arrangement, and external dimensions, collectively constituting a complete structural description of the connector instance.

[0079] Engineering semantic attribute description information is used to encode the engineering attributes extracted from the OCR recognition results into structured description fields. Engineering attributes include at least one of the connector model and material code. The connector model is typically marked in alphanumeric format (such as "DB9", "RJ45", "USB Type-C", etc.) near the connector graphic on the drawing, and is extracted from the model marking area on the page through OCR recognition. The material code is a standardized number assigned to the connector by the enterprise in the material management system, usually appearing in the bill of materials area or title bar as a pure number or alphanumeric combination, and is extracted from the corresponding coding area through OCR recognition. The connector model and material code, as key engineering identification information for the connector, can be directly mapped to corresponding fields in the BOM table, serving as a semantic bridge for connecting the inspection results with the BOM generation process.

[0080] By combining the three types of encoded information described above, a structured intermediate representation of the connector is formed for BOM generation. This intermediate representation uses a standardized data format (such as JSON or XML) to uniformly encode information such as the page source identifier, spatial coordinates, number of pins, pin arrangement, external dimensions, connector model, and material code for each connector instance, and outputs it to the data output interface module. The downstream image matching module and BOM generation module can obtain the complete engineering semantic information of each connector instance by parsing this structured intermediate representation, without needing to re-parse the original drawings or perform manual conversion. This decouples the connector inspection stage from the BOM generation stage, improving the overall system's scalability and traceability.

[0081] In this embodiment, it also includes: The structured intermediate representation is output to the downstream processing stage for connector information parsing and data flow in subsequent engineering applications.

[0082] Specifically, the structured intermediate representation is uniformly encoded in JSON or XML format before being output. During output, the data integrity of the structured intermediate representation is first validated, checking whether the required fields of each connector instance are complete. After the validation passes, the structured intermediate representation is encapsulated into the corresponding data format for output according to the different processing requirements of downstream application scenarios.

[0083] For applications requiring image matching, the system outputs a labeled file containing page source identifiers, spatial coordinates, and structural feature information for each connector instance. Downstream users can directly access the spatial locations of each connector instance for precise template comparison and matching, eliminating the need to re-parse the original drawings. For applications requiring BOM generation, the system outputs structured table data containing connector model, material code, pin quantity, pin arrangement, and dimensional parameters. Downstream users can directly map each field to the corresponding material attribute columns in the BOM table, automatically generating or updating connector-related entries without manual drawing reading or data conversion.

[0084] Through the aforementioned output mechanism, the test results efficiently support various downstream engineering applications in a standardized and traceable manner, eliminating the gap between the connector testing stage and the downstream application stage.

[0085] In this embodiment, it also includes: Based on subsequent image matching results or BOM generation feedback, the weight coefficients of the relevance score, the preset threshold, and the constraint parameters in the engineering constraint rules are updated and optimized.

[0086] Specifically, based on subsequent image matching results or BOM generation feedback, the weight coefficients of the relevance score, the preset thresholds, and the constraint parameters in the engineering constraint rules are updated and optimized to form a closed-loop adaptive optimization mechanism. This enables the system to dynamically adjust the upstream processing parameters according to the actual downstream application effect, thereby continuously improving the overall processing performance.

[0087] When the image matching failure rate exceeds the preset tolerance range, it indicates that there may be missed or false detections during the detection phase. In this case, the weight coefficients of the relevance score should be adjusted first. Specifically, if the matching failures are mainly due to missed detections, the weight coefficients of graphic and structural features should be increased or the first preset threshold should be decreased to improve the sensitivity of key page screening and allow more pages containing connector graphics to enter the detection range. If the matching failures are mainly due to false detections, the weight coefficients of text and annotation semantic features, as well as layout and engineering role features, should be increased or the first preset threshold should be increased to improve the accuracy of screening and reduce irrelevant pages entering the detection range. The adjustment range of the weight coefficients is determined proportionally to the deviation between the matching failure rate and the tolerance range; the greater the deviation, the larger the adjustment range.

[0088] When missing fields or data anomalies appear in the BOM generation result, it indicates that the structural analysis of the connector candidate instance is incomplete or inaccurate. In this case, the constraint parameters in the engineering constraint rules should be adjusted first. Specifically, if the missing fields are mainly manifested as a discrepancy between the number of terminals and the number of pins in the BOM, the engineering error range η in the quantity consistency constraint should be adjusted, appropriately widening or tightening the error tolerance. If the missing fields are mainly manifested as an inability to determine the terminal arrangement, the minimum overlap threshold δ in the spatial inclusion constraint or the proximity threshold ε in the spatial proximity constraint should be adjusted to optimize the accuracy of the association between the terminals and the overall target. The adjustment of constraint parameters is based on the error type and distribution characteristics, and the optimal adjustment direction and magnitude are determined through statistical analysis.

[0089] The adjustment of the first and second preset thresholds is usually determined by combining the results of multiple rounds of feedback. If multiple rounds of feedback show an upward trend in the matching failure rate or field missing rate, the corresponding thresholds are gradually adjusted. After each adjustment, the direction of change in the feedback indicators is observed until the feedback indicators tend to stabilize or reach the preset optimization target. The threshold adjustment adopts a step-by-step adjustment strategy, and the adjustment range of a single adjustment does not exceed the preset maximum step size to avoid system performance oscillations due to excessive adjustment range.

[0090] Furthermore, the aforementioned parameter updates and optimizations can be performed using either online or offline learning methods. In the online learning mode, the system adjusts parameters in real-time based on feedback after each task, suitable for application scenarios with relatively fixed task types. In the offline learning mode, the system accumulates a certain amount of feedback data before performing batch analysis and parameter optimization, suitable for application scenarios with diverse task types or requiring version iterations. In actual deployment, the appropriate learning method should be selected based on the update frequency of industrial engineering drawings and the requirements of downstream BOM generation tasks, or a combination of both methods can be used.

[0091] Through the aforementioned feedback optimization mechanism, parameters such as the weight coefficient of the correlation score, the first preset threshold, the second preset threshold, and the minimum overlap threshold δ, the proximity threshold ε, and the engineering error range η in the engineering constraint rules can be adaptively adjusted according to the actual downstream application effect. This enables the system to maintain stable detection performance when facing industrial engineering drawings of different types, styles, and qualities, effectively avoiding the problem of limited universality caused by fixed parameters, and significantly improving the robustness and engineering adaptability of the system in diverse engineering scenarios.

[0092] Example 2

[0093] This embodiment provides a connector intelligent detection and structured intermediate representation construction system for generating BOMs from industrial engineering drawings, including: The drawing input module is used to acquire multi-page industrial engineering drawings and perform preprocessing. Specifically, the drawing input module takes multiple pages of industrial engineering drawings as input, receives the industrial engineering drawings, and performs unified data preprocessing operations on the industrial engineering drawings. The preprocessing operations include at least drawing format unification, resolution standardization, and page index and identification management. The output is the preprocessed unified format drawings and corresponding page index and identification information, so as to provide basic data support for subsequent cross-page analysis and processing.

[0094] The engineering semantic feature extraction module is used to extract engineering semantic features from each page of preprocessed industrial engineering drawings. Specifically, the engineering semantic feature extraction module takes as input each preprocessed page of industrial engineering drawings and extracts engineering semantic features from each page. These features include three categories: text and annotation semantic features, graphic and structural features, and layout and engineering role features. Text and annotation semantic features are obtained by performing OCR recognition on each page of industrial engineering drawings and matching the OCR results with keywords from a pre-defined engineering semantic dictionary. Graphic and structural features are obtained by recognizing connector graphics and line position table graphics in each page of industrial engineering drawings based on an object detection model. Layout and engineering role features are obtained by performing OCR recognition on each page of industrial engineering drawings and extracting drawing type information from the page title bar, header, or footer. The output is the engineering semantic features corresponding to each page of industrial engineering drawings, providing multi-dimensional feature input for subsequent key page selection.

[0095] The key page intelligent positioning module is used to calculate the relevance score of each page of industrial engineering drawings based on the semantic features of the engineering, and to filter the industrial engineering drawings with a relevance score higher than the first preset threshold as the key page set. Specifically, the key page intelligent positioning module takes as input the engineering semantic features of each page of industrial engineering drawings, calculates the relevance score of the corresponding industrial engineering drawings based on the engineering semantic features, filters industrial engineering drawings with relevance scores higher than a first preset threshold as key page sets, and outputs the key page set and the corresponding relevance score ranking results, so as to narrow the processing range and reduce the computational burden for subsequent multi-level target detection.

[0096] The engineering semantic feature set includes at least the following categories: a) Text and annotation semantic features The frequency and distribution of keywords related to connectors are statistically analyzed, and regular expressions are used to identify interface number sequence patterns to form page-level text semantic features.

[0097] [Keyword occurrence count, PIN label count, interface number sequence completeness score]; b) Graphic and structural features A deep learning-based object detection model is used to identify elements such as connectors and line positions in engineering drawings to obtain candidate graphic targets on the page. The frequency and distribution of these targets are then statistically analyzed.

[0098] ; c) Layout and Engineering Role Characteristics By extracting drawing type information from the page title bar, header, or footer using OCR, it can be determined whether the page belongs to wiring diagram, schematic diagram, or assembly instruction diagram, and the page's engineering role characteristics can be constructed.

[0099] .

[0100] For each page, a weighted scoring function is used to score its relevance. The weights of each feature can be set based on empirical rules or obtained through training on historical labeled data. For the first... The page relevance score of a page of engineering drawings can be expressed as:

[0101] in, Indicates the first Page 1 of Normalized eigenvalues ​​on the semantic features of engineering projects This represents the weight coefficient of the corresponding engineering semantic features. This represents the total number of categories for engineering semantic features. (When page relevance score is used...) Greater than the preset threshold At that time, the corresponding page is identified as a candidate for key page.

[0102] Based on the page relevance score, the multi-page engineering drawings are sorted, and the pages with scores higher than the preset threshold or those ranked higher are selected as the key page set.

[0103] The multi-level target detection module is used to perform multi-level target detection on pages in the key page set to obtain overall connector targets, connector end targets, and cable targets. Specifically, the multi-level target detection module takes as input images of each page in the key page set, and performs multi-level target detection on the pages in the key page set to obtain and output the overall connector target, the connector end target, and the cable target; the multi-level detection includes at least the following two levels: a) Overall target inspection of connectors The YOLO target detection model identifies the graphic area representing the overall engineering structure of the connector on the page, and outputs the spatial location, category label, and initial confidence level of the overall connector target.

[0104] b) Connector local target detection

[0105] The YOLO object detection model is used to further detect the ends and cable structures on the same page.

[0106] The connector candidate instance generation module is used to establish the association between the overall target of the connector and the target of the connector end based on preset engineering constraint rules, and generate a set of connector candidate instances. Specifically, the input to the connector candidate instance generation module is the spatial position of the overall connector target and the spatial position of the connector end target, as well as the OCR text recognition result. This is used to establish the association between the overall connector target and the connector end target based on preset engineering constraint rules, and to generate and output a set of connector candidate instances.

[0107] The preset engineering constraint rules include: a) Spatial inclusion constraint: The connector end target should be located within the internal area of ​​the overall connector target;

[0108] in, The bounding box representing the end target. The bounding box representing the overall target of the connector. This is the preset minimum overlap threshold.

[0109] b) Spatial proximity constraint: Multiple end targets maintain a continuous or adjacent distribution relationship with the overall target of the same connector in space, which is used to assist in the determination of the ownership of end targets.

[0110]

[0111] in, This represents the minimum spatial distance between the end target and the overall connector target. This is a preset proximity threshold.

[0112] c) Consistency constraint: The number of endpoints must be consistent with the number of interface annotations identified in the OCR text recognition results within the allowable error range;

[0113] in, The number of detected end targets. The number of interfaces identified on the page is labeled. This refers to the allowable range of engineering errors.

[0114] The above three types of constraints are used to establish the overall-end relationship of the connector and filter valid pages.

[0115] The engineering semantic modulation and consistency constraint verification module is used to perform engineering semantic modulation and consistency constraint verification on the set of candidate connector instances based on the cable target. Specifically, the engineering semantic modulation and consistency constraint verification module takes as input a set of candidate connector instances and the spatial location of the cable target, and outputs a set of candidate connector instances with size consistency scores and effective connection end ratios after size consistency verification and topology consistency analysis, providing engineering semantic level scoring input for subsequent comprehensive confidence calculation; it is used to enhance and constrain the detection results output by the multi-level target detection module of connectors at the engineering semantic level to improve the credibility of the detection results, which includes at least: Dimensional consistency verification: Identify scale information (e.g., 1:1, 1:2, etc.) from the page OCR results, convert the bounding box dimensions of multi-level detection results into engineering dimensions, and compare them with the preset engineering dimension range. If the dimensions exceed the reasonable range, the dimensional consistency score of the corresponding candidate instance is reduced. .

[0116] Topological consistency analysis: The skeleton of the detected cable target is extracted, and the start and end points are extracted.

[0117] Analyze the spatial adjacency relationship between the terminal target and the cable endpoint. When the cable endpoint is located within the terminal boundary area or maintains a preset proximity distance from the terminal, it is considered that the cable and the terminal form a connection relationship. Establish the connection correspondence between the terminal and the cable and calculate the proportion of effective connected terminals.

[0118] =

[0119] in, Number of validly connected endpoint targets. The comprehensive confidence calculation and filtering module is used to calculate the comprehensive confidence of each connector candidate instance based on the verification results, and to remove connector candidate instances with a comprehensive confidence score lower than a second preset threshold, thus obtaining a connector instance set; Specifically, the input is a set of candidate connector instances with size consistency scores and effective connection end ratios after size consistency verification and topology consistency analysis, and the initial confidence of each connector overall target output from the multi-level target detection stage. The output is a set of connector instances with a comprehensive confidence of not less than the second preset threshold, so as to provide reliable connector instance data for subsequent structured intermediate representation construction. Overall confidence level calculation: The confidence score of the detection results is calculated by combining the initial confidence score output by the integrated target detection model with engineering semantic scores such as size consistency and topological consistency.

[0120]

[0121] in: The initial confidence score output by the object detection model. Dimensional consistency score Topology consistency score.

[0122] When the calculated overall confidence level When the result is less than the preset threshold, the corresponding detection result is rejected.

[0123] The structured intermediate representation construction and output module is used to construct and output a structured intermediate representation based on a set of connector instances.

[0124] Specifically, the input to the structured intermediate representation construction and output module is the set of connector instances output by the comprehensive confidence calculation and filtering module, and the output is the structured intermediate representation of each connector instance; it is used to uniformly model the candidate connector instances after engineering semantic modulation before image matching, and construct a structured intermediate representation for BOM generation, which includes at least: The Spatial and Page Context Encoding submodule is used to record the page source information and spatial location characteristics of the connector instance; The engineering structure feature modeling submodule is used to abstract and encode the structural features of the connector; The engineering semantic attribute description submodule is used to generate structured fields that describe the engineering attributes of the connector; The data output interface submodule is used to output the structured intermediate representation in a unified and standardized data format to at least one subsequent processing module. The subsequent processing module includes, but is not limited to, an image matching module and a BOM generation module, thereby achieving process decoupling between the connector detection stage and the BOM generation stage.

[0125] The structured intermediate representation is output to the image matching module or BOM generation module through the data output interface submodule. Based on the subsequent image matching results or BOM generation feedback, the engineering semantic modulation parameters and structural consistency constraint rules are updated and optimized.

[0126] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for intelligent detection and structured intermediate representation construction of connectors for industrial engineering drawing BOM generation, characterized in that, include: Acquire multi-page industrial engineering drawings and perform preprocessing; Extract engineering semantic features from each page of the preprocessed industrial engineering drawings; The relevance score of each page of industrial engineering drawings is calculated based on the engineering semantic features, and industrial engineering drawings with a relevance score higher than a first preset threshold are selected as a set of key pages. Perform multi-level target detection on the pages in the set of key pages to obtain overall connector targets, connector end targets, and cable targets; Based on preset engineering constraint rules, the association between the overall target of the connector and the target of the connector end is established, and a set of candidate connector instances is generated. Based on the cable target, perform engineering semantic modulation and consistency constraint verification on the set of connector candidate instances; The overall confidence level of each connector candidate instance is calculated based on the verification results, and connector candidate instances with an overall confidence level lower than the second preset threshold are removed to obtain a connector instance set. A structured intermediate representation is constructed and output based on the set of connector instances.

2. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 1, characterized in that, The preprocessing includes: Standardize drawing formats, resolution, and manage page indexes and labels.

3. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 1, characterized in that, The engineering semantic features include: The text and annotation semantic features are obtained by performing OCR recognition on each page of industrial engineering drawings and matching the OCR recognition results with keywords in a preset engineering semantic dictionary. The text and annotation semantic features include at least one of the following: the number of occurrences of keywords related to connectors, the number of PIN annotations, and the completeness score of the interface number sequence. Graphic and structural features are obtained by identifying connector graphics and wire position table graphics in each page of industrial engineering drawings based on a target detection model. The graphic and structural features include at least one of the number of connector graphics and the number of wire position table graphics. The layout and engineering role characteristics are obtained by performing OCR recognition on each page of industrial engineering drawings and extracting the drawing type information from the page title bar, header or footer of each page of industrial engineering drawings. The layout and engineering role characteristics are used to determine the probability that the industrial engineering drawings belong to wiring diagrams, schematic diagrams or assembly instruction diagrams.

4. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 1, characterized in that, The formula for calculating the relevance score is: in, This represents the normalized feature value of page i on the semantic features of the k-th type of project. This represents the weight coefficient of the corresponding engineering semantic features. This represents the total number of categories of engineering semantic features.

5. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 3, characterized in that, The multi-level target detection performed on the pages in the key page set to obtain overall connector targets, connector end targets, and cable targets includes: The first target detection model is used to detect each page in the key page set, identify the overall graphic region of the connector in the page, and output the spatial location, category label and initial confidence of each overall target of the connector. The second target detection model is used to detect each page in the key page set, identify the connector end graphic area and cable graphic area in the page, and output the spatial position of each connector end target and each cable target.

6. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 1, characterized in that, The process of establishing the association between the overall connector target and the connector end target based on preset engineering constraint rules, and generating a set of candidate connector instances, includes: Obtain the spatial position of the overall connector target and the spatial position of the connector end target; Based on the preset engineering constraint rules, determine the attribution relationship between each connector end target and each overall connector target; Connector end targets belonging to the same overall target are associated with the overall connector target to form connector candidate instances; A set of connector candidate instances is generated based on each of the aforementioned connector candidate instances; The engineering constraint rules include: Spatial inclusion constraint: The bounding box of the connector end target should be located within the bounding box of the overall connector target. This is determined by calculating whether the intersection-union ratio of the two is greater than or equal to a preset minimum overlap threshold. Spatial proximity constraint: Multiple connector end targets are spatially distributed in a continuous or adjacent manner with the same overall connector target. This is determined by calculating whether the minimum spatial distance between the connector end target and the overall connector target is less than or equal to a preset proximity threshold. Quantity consistency constraint: The difference between the number of detected end targets and the number of interface annotations identified by OCR recognition results shall not exceed the preset engineering error range.

7. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 5, characterized in that, The step of performing engineering semantic modulation and consistency constraint verification on the connector candidate instance set based on the cable target includes: Size consistency verification: For each connector candidate instance in the connector candidate instance set, the scale information is identified from the OCR recognition results, the bounding box size of the detected connector candidate instance is converted into the engineering size, compared with the preset engineering size range, and a size consistency score is generated. Topology consistency analysis: The cable target is skeletonized, and the start and end points of the cable are extracted. The spatial adjacency relationship between the connector end target and the cable end point is analyzed. When the cable end point is located within the boundary area of ​​the connector end target or the distance to the connector end target boundary is less than a preset proximity distance, the cable target and the connector end target are considered to have a valid connection relationship, and the proportion of valid connection ends is calculated. ,in The target number of connector ends for effective connection. This represents the total number of connector end targets detected.

8. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 7, characterized in that, The formula for calculating the overall confidence level is as follows: in, The initial confidence level for the overall target of the connector. Scoring for dimensional consistency To ensure effective connection of the end proportions, , , These are the corresponding weighting coefficients.

9. The intelligent detection and structured intermediate representation construction method for connectors in industrial engineering drawing BOM generation according to claim 3, characterized in that, The construction of the structured intermediate representation based on the set of connector instances includes: The page source identifier and spatial location coordinates of each connector instance are encoded into spatial and page context encoding information; The structural features of each connector instance are encoded into engineering structural feature modeling information, and the structural features include at least the number of connector pins, pin arrangement, and external dimension parameters. The engineering attributes extracted from the OCR recognition results are encoded into engineering semantic attribute description information, and the engineering attributes include at least the connector model and / or material code.

10. A connector intelligent detection and structured intermediate representation construction system for generating BOMs from industrial engineering drawings, characterized in that, include: The drawing input module is used to acquire multi-page industrial engineering drawings and perform preprocessing. The engineering semantic feature extraction module is used to extract engineering semantic features from each page of preprocessed industrial engineering drawings. The key page intelligent positioning module is used to calculate the relevance score of each page of industrial engineering drawings based on the engineering semantic features, and to filter the industrial engineering drawings with the relevance score higher than a first preset threshold as a set of key pages. A multi-level target detection module is used to perform multi-level target detection on the pages in the set of key pages to obtain the overall connector target, the connector end target, and the cable target; The connector candidate instance generation module is used to establish the association between the overall target of the connector and the target of the connector end based on preset engineering constraint rules, and generate a set of connector candidate instances; The engineering semantic modulation and consistency constraint verification module is used to perform engineering semantic modulation and consistency constraint verification on the candidate instance set of connectors based on the cable target. The comprehensive confidence calculation and filtering module is used to calculate the comprehensive confidence of each connector candidate instance based on the verification results, and remove the connector candidate instances whose comprehensive confidence is lower than the second preset threshold to obtain a connector instance set. The structured intermediate representation construction and output module is used to construct and output a structured intermediate representation based on the set of connector instances.

Images