A method for automatically identifying and coloring risk areas of complex engineering drawings based on multi-model fusion

Abstract

This invention discloses an automatic identification and coloring method for risk areas in complex engineering drawings based on multi-model fusion, relating to the field of industrial safety technology. The method includes the following steps: drawing text parsing and non-destructive erasure; wireframe feature analysis and parameter decision-making; wireframe topology repair; and region semantic segmentation and attribute discrimination. This invention provides an automated processing scheme with high precision, robustness, and logical consistency when processing complex engineering drawings. It achieves pixel-level precise erasure, preserves the original wireframe topology to the greatest extent, and enables adaptive and dynamic adjustment of algorithm parameters, thereby improving the system's generalization ability.

Description

Technical Field

[0001] This invention relates to the field of industrial safety technology, and more specifically, to a method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion. Background Technology

[0002] In the field of computer vision and image processing, automated risk area identification and annotation of CAD engineering drawings or scanned drawings in fields such as architecture and factories is a key technology for intelligent safety management and digital factory construction. Existing technical solutions are typically based on traditional image processing algorithms, but they face several core challenges when processing complex and diverse real-world engineering drawings: Damage to wireframe structure: Traditional methods for erasing text on drawings typically use rectangular bounding boxes detected by OCR (Optical Character Recognition) to directly cover or erase the text area. However, text annotations in engineering drawings often intersect or overlap with wireframes (wall lines, axis lines) representing the outlines of walls, beams, and equipment. This method of erasing the entire rectangular bounding box can inadvertently damage the pixels of the wireframes overlapping with the text, causing breakage of the underlying geometric skeleton of the drawing, destroying the overall topological structure of the image, and creating serious hidden dangers for subsequent region segmentation and filling steps.

[0003] Poor robustness and adaptability of algorithms: Actual engineering drawings exhibit significant differences in visual features due to variations in design standards, drawing specifications, output scale, and scanning quality. This is primarily reflected in the varying thickness of wall lines and diverse line styles (e.g., solid lines, double lines / hollow lines representing thick walls). Most existing algorithms employ fixed morphological processing parameters (such as kernel size for dilation and erosion, gap detection thresholds, etc.), making it difficult to adaptively adapt to different drawing specifications. Consequently, while the same set of parameters may yield acceptable results for some drawings, it can lead to issues like incorrect wireframe connections, incorrect disconnections, or feature extraction failures for others.

[0004] Difficulty in Wireframe Topology Closure: Engineering drawings contain numerous dashed lines (representing hidden lines, planning lines, etc.) and doorway structures, which appear as discontinuous wireframes in the image. When using traditional region-filling algorithms (such as FloodFill), the algorithm "overflows" at these discontinuities, causing color to fill outside the intended area, resulting in "color leakage." Existing methods lack robust mechanisms to automatically and accurately detect and repair these specific discontinuous structures to form closed fill boundaries.

[0005] Difficulty in distinguishing semantic regions: Different regions in drawings possess different physical and functional semantics, such as independent functional rooms (exclusive areas), large spaces shared by multiple devices (non-exclusive areas), and corridors connecting these areas. Traditional methods can only perform simple segmentation based on connected components and cannot distinguish these semantic categories. This leads to the inability to adopt differentiated strategies based on regional semantics during automated coloring (e.g., corridors should not be colored over large areas), resulting in inaccurate coloring ranges and logically confused risk visualization results.

[0006] In summary, existing technologies have significant shortcomings in processing complex engineering drawings, particularly in terms of geometric preservation, parameter adaptation, topological integrity restoration, and high-level semantic understanding. There is an urgent need for a high-precision, robust, and automated processing solution that can comprehensively address these issues. Summary of the Invention

[0007] This invention overcomes the shortcomings of existing technologies and provides a highly accurate, robust, and logically sound method for automatically identifying and coloring risk areas in complex engineering drawings, based on multi-model fusion.

[0008] The technical solution of the present invention is as follows: An automatic identification and coloring method for risk areas in complex engineering drawings based on multi-model fusion includes the following steps: S1 Drawing Text Parsing and Non-destructive Erasing Steps: For the input engineering drawing, use an OCR model to perform text detection and recognition to obtain the text content and its bounding box coordinates; use the SAM2 model to perform pixel-level segmentation within the text content and its bounding box coordinate area, and extract only the mask of the area occupied by the character itself; based on the mask, erase the corresponding pixel area in the drawing to preserve the wireframe structure outside the text area. S2 wireframe feature analysis and parameter decision steps: Binarize the drawing after S1 processing, extract the wall line skeleton and calculate the global average line width; use the global average line width as a dynamic parameter to adjust the parameters of subsequent processing algorithms; S3 wireframe topology repair steps: Based on the text position information obtained in S1 and the dynamic parameters obtained in S2, the gaps between dashed lines and bottlenecks in the drawing are detected, and multiple geometric constraints and text collision avoidance strategies are adopted to perform pixel-level completion of the gaps between dashed lines and the structure of the doorway that meet the conditions, forming a topologically closed wireframe diagram. S4 Region Semantic Segmentation and Attribute Judgment Steps: Based on the closed wireframe obtained in S3, each region is assigned exclusive, non-exclusive, and passageway labels; for non-exclusive regions and passageways, synchronous expansion is performed using OCR boxes as seeds, and boundaries are detected through real-time connected component analysis, automatically stopping at the color boundaries of different entries to form colored regions.

[0009] Furthermore, for the text content and its bounding box coordinates in step S1, semantic verification and spatial reconstruction of the bounding box coordinates can be performed based on a preset standard term knowledge base to obtain an optimized text bounding box; using the optimized text bounding box as a prompt, the visual segmentation SAM2 model is called to perform pixel-level instance segmentation to generate an accurate text mask.

[0010] Furthermore, the normal direction of the wall lines is calculated based on the structural tensor, and spatial probing is performed along the normal direction to statistically analyze the proportion of skeleton pixels with parallel symmetry. Based on this, it is determined whether the drawing is a single-layer solid line or double-layer hollow line style, and the average spacing of the double-layer wall lines is extracted. The average spacing is used as a dynamic parameter to adjust the parameters of subsequent processing algorithms.

[0011] Furthermore, the detection of the dashed line gap in step S3 includes: scanning along the rows and columns of the image to identify white gaps, and extracting the gap length, the length and thickness of the wall segments on both sides; only when there are at least two candidate gaps on the same straight line that meet the following conditions: the gap length is less than a threshold, the length of the wall segments on both sides is greater than the minimum support length, the ratio of the thickness of the wall lines on both sides is less than a preset value, and there is an intersection in the vertical position, is the location determined to be a dashed line structure and is then completed.

[0012] Furthermore, the text collision avoidance strategy described in step S3 is as follows: the text area recognized by OCR is compressed according to a preset ratio to generate a protection mask; before repairing the detected doorway or dotted line gap, the coverage area of ​​the repair block is simulated; if the coverage area intersects with the protection mask, the repair operation is abandoned.

[0013] Furthermore, the process of assigning exclusive, non-exclusive, and passageway labels to each region is as follows: using the coordinates of each risk-annotated text as seed points, a flooding filling algorithm is executed to generate the corresponding initial region mask; the overlap between each region mask is calculated, and the disjoint-set data structure algorithm is used to aggregate masks with high overlap into the same logical group, and exclusive or non-exclusive labels are assigned to each region accordingly; through morphological dilation and collision detection, the number of adjacent regions of each logical group is counted, and if the number of adjacent regions exceeds a preset threshold, a passageway label is assigned to that region.

[0014] Furthermore, the specific criteria for determining the aisle labels are as follows: For each logical group region, its region mask is morphologically expanded until it intersects with the pixel masks of other regions to determine their adjacency; the number of all distinct adjacent regions of the logical group region is counted, and if the number is greater than a certain threshold, the region is determined to be a passageway.

[0015] Furthermore, the process includes an intelligent coloring and rendering step: based on the region mask obtained in S4 and its corresponding exclusive label and aisle label, combined with the mask's own fill completeness, a corresponding coloring strategy is selected for each region; the coloring strategy includes: directly filling exclusive, complete, and non-aisle masks; for non-exclusive or aisle masks, a pixel-level competitive expansion-based coloring algorithm is activated to define the precise coloring boundaries of each region; for incompletely filled masks, their corresponding OCR text bounding boxes are used as the coloring areas; finally, a minimum value fusion algorithm is used to mix and render the target color with the original drawing pixels to generate the final risk four-color image.

[0016] Furthermore, the pixel-level competitive expansion-based coloring algorithm is specifically as follows: each region to be expanded is assigned a unique ID in the global label map, and each region moves one pixel in turn in each direction according to a preset order, with its own seed point as the center; before each step, it is detected whether there are IDs of other regions in the adjacent pixels of the target direction, and if so, the expansion in that direction is stopped immediately, thereby forming an isolation zone between adjacent regions.

[0017] Furthermore, the minimum value fusion algorithm for hybrid rendering specifically involves taking the smaller value between the R, G, and B channel values ​​of the original pixel and the R, G, and B channel values ​​of the target color for each pixel, and using this smaller value as the final channel value of the output image at that pixel.

[0018] The advantages of this invention compared to the prior art are: This invention achieves pixel-level precise erasure while preserving the original wireframe topology to the greatest extent possible: by combining the coarse-grained text localization of OCR with the pixel-level instance segmentation capability of the SAM2 model, it precisely erases only the pixel areas of the characters themselves, while fully preserving engineering primitives such as wall lines and axes that intersect or overlap with the text. This fundamentally solves the problems of "wireframe damage" and "topological destruction" caused by traditional rectangular frame overlay erasure, laying a high-quality geometric foundation for all subsequent wireframe-based processing steps.

[0019] This invention achieves adaptive and dynamic adjustment of algorithm parameters, enhancing the system's generalization ability: by analyzing the skeleton and structural tensor of the wireframe in the drawing, it automatically identifies the wall line type (single-layer / double-layer) and accurately calculates key physical features such as the global average line width and the spacing between double-layer walls. These features are fed back as dynamic parameters to subsequent processing modules, enabling the thresholds and kernel sizes of algorithms such as dashed line detection, doorway closure, and morphological operations to adapt to drawings with different drawing standards, scales, and scanning qualities. This significantly improves the robustness and adaptability of the entire solution, overcoming the pain point of fixed-parameter algorithms being unable to adapt to various drawing specifications.

[0020] This invention, by establishing rules that include multiple geometric constraints such as gap length, support length of wall segments on both sides, thickness consistency, and the number of coaxial candidates, can accurately distinguish between true "dashed line structures" and "isolated wall openings," completing only the dashed lines that meet the collective characteristics, thus significantly reducing the false repair rate. Combined with a doorway detection mechanism that uses a "wide-narrow-wide" template and neighborhood overlap verification, it can accurately identify and close doorways.

[0021] By employing a "text collision avoidance strategy," the collision between the repair block and the compressed OCR text protection mask is simulated before repair. If a collision occurs, the repair is abandoned, ensuring that the topology repair operation does not destroy the original text annotation information in the drawing. This achieves linkage between geometric repair and semantic preservation, improving the logical correctness of the processing.

[0022] By calculating the overlap between region masks (using disjoint-set aggregation) and statistical topological adjacency (detecting the number of neighbors through morphological dilation), the system can automatically assign "exclusivity," "non-exclusivity," and "corridor" labels to each region. This enables the system to understand the spatial logic of the drawing, distinguish between independent rooms, shared spaces, and circulation corridors, and provide a semantic basis for differentiated coloring strategies.

[0023] This invention accurately identifies "corridors" and assigns them corresponding labels, preventing large-scale incorrect coloring of corridors due to incomplete wireframes or algorithmic logic defects. This ensures that the color blocks in the final generated "risk four-color map" more accurately focus on the actual production or functional rooms, improving the accuracy and interpretability of risk visualization. Furthermore, it implements multi-strategy adaptive coloring, balancing efficiency and boundary accuracy: the system intelligently selects three strategies—"direct filling," "competitive expansion," or "OCR box fallback"—based on the "exclusivity," "quality," and "corridor" labels of the region mask. This guarantees fast rendering of high-quality exclusive areas while solving the boundary delineation problem when multiple color blocks compete for space in non-exclusive areas and corridors.

[0024] The "competitive growth algorithm" of this invention assigns an independent ID to each area to be colored and expands synchronously, stopping immediately when the expansion boundaries meet and forming an isolation zone. This mechanism enforces clear, non-overlapping boundaries between different color areas at the algorithm level, completely solving the problem of color overlap in densely annotated areas. A "minimum value fusion algorithm" is used for final pixel blending. Utilizing the "black lines on a white background" characteristic of engineering drawings, this algorithm ensures that the pixel values ​​(close to 0) of black text, axes, and other elements on the original drawing are always lower than the pixel values ​​of the colored elements being painted. This allows important wireframes and annotation information to naturally float above the color blocks without additional transparency adjustments or post-processing, improving the professionalism and readability of the output. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of an original engineering drawing of the present invention; Figure 3 This is a schematic diagram of the text removal results of the present invention; Figure 4 This is a schematic diagram of the wall line expansion and background noise removal processing results of the present invention; Figure 5 This is a schematic diagram of the result of the dotted line and the doorway closure in this invention; Figure 6 This is a four-color diagram of the risk area of ​​the completed invention; Figure 7 This is a schematic diagram of another engineering drawing of the present invention; Figure 8 This is a diagram illustrating the text erasure result; Figure 9 This is a schematic diagram of a single-layer wall line of the present invention; Figure 10 This is a schematic diagram of the double-layer wall line of the present invention; Figure 11 This is a schematic diagram of the overlapping mask aggregation of the present invention. Detailed Implementation

[0026] Embodiments of the present invention are described in detail below, wherein the same or similar reference numerals denote the same or similar elements or elements with similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as limiting the present invention.

[0027] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0028] The numbering of steps mentioned in the various embodiments is merely for descriptive convenience and does not imply a sequential relationship. Different steps in various specific embodiments can be combined in different orders to achieve the inventive objective of this invention.

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 As shown, a method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion is characterized by the following steps: S1 Drawing Text Parsing and Non-destructive Erasing Steps: For the input engineering drawing, an OCR model is used for text detection and recognition to obtain the text content and its bounding box coordinates. After local merging and segmentation, the OCR recognition results can be selectively introduced into an LLM (Large Language Model) for semantic error correction. Then, the SAM2 model is used to perform pixel-level segmentation within the text content and its bounding box coordinates, extracting only the mask of the area occupied by the character itself. Based on the mask, the corresponding pixel area in the drawing is erased to preserve the wireframe structure outside the text area. This maximizes the preservation of the geometric wireframe skeleton around the text, avoiding accidental damage to the surrounding wireframes caused by rectangular frame erasing.

[0031] The text content and its bounding box coordinates can be semantically verified and spatially reconstructed based on a preset standard term knowledge base to obtain an optimized text bounding box. Using the optimized text bounding box as a prompt, the visual segmentation SAM2 model is called to perform pixel-level instance segmentation and generate an accurate text mask.

[0032] To address the complexities of irregular text layout, dense line breaks, and multiple parallel terms in engineering drawings, a term reconstruction scheme based on semantic-geometric bidirectional verification was designed on top of the underlying visual recognition. The specific steps are as follows: 1) Low-level information extraction based on a high-performance inference engine Engine Invocation: The system uses the RapidOCR framework to call the PP-OCRv4 deep learning inference engine to perform global text detection and recognition on the input original engineering drawings. It accurately extracts the string content of all text elements in the drawings, as well as their corresponding two-dimensional spatial coordinate sets (BoundingBoxes), forming an initial structured text and image dataset.

[0033] 2) "Line break merging" and "adhesive segmentation" mechanisms based on standard knowledge bases Due to the unique nature of industrial drawing annotations, the original OCR results often exhibit misalignment between physical boundaries and logical semantics. Therefore, a pre-defined "standard term knowledge base" is introduced to compare, verify, and spatially reconstruct the initial data.

[0034] For multi-frame terms, a line-breaking merging method is used: Drawings often contain text that is broken into multiple lines due to space constraints (e.g., "air compressor" and "machine room" displayed on separate lines), causing OCR to misidentify them as multiple isolated terms. The system calculates the spatial distance between candidate frames (Euclidean distance and vertical overlap) and compares the combined text with the knowledge base. If a standard term is matched, a merging operation is performed, and the minimum bounding rectangle encompassing all fragments is recalculated as the new coordinates.

[0035] For cases where a single bounding box contains multiple words, a parallel segmentation method is used: When multiple independent functional area labels are closely arranged on the same horizontal line (such as "rest area processing area"), OCR is very likely to misidentify it as a single long term. The system performs knowledge base segmentation and matching on the extracted string. If it finds that the string contains multiple independent standard terms, it performs a segmentation operation. The system dynamically divides the original long bounding box into multiple independent text boxes based on the physical pixel ratio of the string length, and assigns each box the correct coordinate information.

[0036] This breaks through the limitations of traditional image processing that relies solely on visual features. By introducing a priori "standard knowledge base," it completely solves the inherent pain points of "fragmentation" and "excessive adhesion" in OCR when dealing with complex industrial drawings from a semantic dimension. This ensures the subsequent geometric accuracy processing downstream and provides "seed coordinates" for subsequent SAM2 model erasure, dashed line / doorway closure, and floodfill algorithms.

[0037] Addressing the common problem of text annotations in engineering drawings frequently overlapping with building wall lines, equipment outlines, and dimension lines, this solution abandons the traditional, crude "rectangular outline erasure" approach. Instead, it introduces the SAM2 (SegmentAnythingModel2) visual segmentation model, achieving pixel-level processing that "erases only the text, without damaging the lines." The implementation steps are as follows: 1) BoxPrompting based on spatial coordinates The system receives structured data output from the front-end "OCR and knowledge base dual verification module" and extracts high-precision text bounding box coordinates after merging and segmentation correction. These two-dimensional coordinate sets are used as spatial prompt matrices and are batch-input into the SAM2 image segmentation model deployed in the inference backend, guiding the model to focus its attention mechanism on each local image region (ROI) containing text.

[0038] 2) Pixel-level text feature extraction and mask generation The SAM2 model leverages its powerful zero-shot generalization ability and deep image feature extraction network to perform fine-grained instance segmentation within each input bounding box. It automatically extracts the precise pixel-level contours of text strokes, generating a high-fidelity text mask. This process accurately distinguishes between the "black pixels of the character body" and "interfering pixels such as wall lines and pipes interspersed within the text box," ensuring that the mask's boundaries conform to the text topology.

[0039] 3) Lossless Text Erasure The precise text mask output by SAM2 is mapped back to the original engineering drawing (or binarized drawing). Targeted pixel replacement operations are performed on the areas covered by the mask (such as setting it to a white background pixel value of 255, or performing local image inpainting), while areas outside the mask (i.e., engineering primitives such as lines and arcs that are contained within the original OCR rectangle but are not part of the text strokes) are preserved to the greatest extent possible.

[0040] The traditional BoundingBox erasing method inevitably causes "physical breaks" in wall lines or pipes overlapping with the text when removing text, disrupting the geometric continuity of the drawing. However, the SAM2 model achieves more precise pixel removal, preserving the original global topological skeleton of the drawing to the maximum extent. This reduces the dimensionality and improves efficiency of subsequent algorithms. Because the original wireframe is preserved to the greatest extent, the pressure and misjudgment rate of subsequent morphological restoration algorithms such as "dashed line filling" and "doorway closure" are greatly reduced, fundamentally improving the robustness of connected component segmentation in the risky four-color image. Specific effects are shown below. Figure 7 and Figure 8 As shown.

[0041] S2 wireframe feature analysis and parameter decision steps: The drawing processed by S1 is binarized, the wall line skeleton is extracted, and the global average line width is calculated. Based on this, the wall line type (single-layer / double-layer line) can be automatically identified and its feature thickness can be calculated. The global average line width is used as a dynamic parameter to adjust the parameters of subsequent processing algorithms.

[0042] It also includes calculating the normal direction of wall lines based on structural tensors and performing spatial probing along the normal direction to statistically analyze the proportion of skeleton pixels with parallel symmetry. Based on this, it determines whether the drawing is in a single-layer solid line or double-layer hollow line style and extracts the average spacing of the double-layer wall lines. The average spacing is also used as a dynamic parameter to adjust the parameters of subsequent processing algorithms.

[0043] The specific steps for drawing feature analysis based on structural tensor and space exploration are as follows: 1) Image preprocessing and standardization First, the original RGB drawing is converted to grayscale and then adaptively binarized using Otsu's binarization method. To ensure consistency in subsequent algorithms, the system implements "intelligent inversion" by calculating the global proportion of bright pixels, ensuring that the processed objects are uniformly formatted as black background with white lines.

[0044] 2) Linewidth estimation based on "skeleton-range field" fusion The morphological skeleton extraction algorithm is used to obtain the central axis S of the line, and the binarized image is then processed. Euclidean distance transformation to generate a distance field map .

[0045] The calculation formula is the line at the point Local thickness ,in Extract the width values ​​of all points on the skeleton and apply a preset threshold. After filtering out abnormal noise, the average value is taken to obtain the global feature linewidth. .

[0046] 3) Calculation of the orientation field based on the structure tensor Instead of simple gradient calculations, a structure tensor is introduced to obtain the topological orientation of lines in the drawing, thereby utilizing eigenvalue decomposition to calculate the image's feature vectors. Feature values ​​are then extracted. The eigenvector corresponding to the dominant gradient direction This vector is the normal direction perpendicular to the direction of the wall line.

[0047] 4) Vectorized Spatial Parallel Probing Mechanism To distinguish between "single-layer solid lines" and "double-layer hollow lines," this scheme employs a detection logic based on normal stepping: i) Detection point generation: For the skeleton Every point on Along its normal Ray detection is performed in both positive and negative directions.

[0048] ii) Dynamic search range: The search range is defined as follows: Set minimum step size This avoids interference from the width of the current line itself, maximizing the step size. Used to capture potential parallel opposite lines.

[0049] iii) Collision detection: If the coordinates are within the search range If a skeleton pixel is detected again at this location, then the decision point is determined. It has parallel symmetry.

[0050] Single-layer and double-layer wall lines are as follows Figure 9 and Figure 10 As shown, Figure 9 This diagram illustrates a single-layer wall line. Figure 10 This is a diagram illustrating a double-layered wall line.

[0051] 5) Determination of wall line type based on coverage Sum the number of skeleton pixels that possess "parallel symmetry". Calculate its position in the total skeleton pixels. Percentage in: ,

[0052] when When the preferred threshold is 0.8, the drawing is determined to be in a double-line / hollow-line style (the same drawing is determined to be one wall line style, with no mixed styles); otherwise, it is determined to be in a single-layer solid-line style. If it is determined to be a double-line style, the system will automatically extract the detected average spacing. This spacing will serve as a dynamic reference parameter for subsequent "wireframe expansion" and "doorway detector" operations, enabling the algorithm to self-calibrate for drawings of different scales. If the style is determined to be single-line, then set... .

[0053] S3 Wireframe Topology Repair Steps: Based on the text position information obtained in S1 and the dynamic parameters obtained in S2, the gaps between dashed lines and bottlenecks in the drawing are detected. Multiple geometric constraints and text collision avoidance strategies are employed to perform pixel-level completion of eligible gaps and bottleneck structures, forming a topologically closed wireframe diagram. Specifically, multi-dimensional heuristic rules including gap length, support length, thickness consistency, and boundary protection are established, and completion is performed using vertical intersection calculations. A "width-narrow-width" geometric template detector is introduced, linked in real-time with the OCR semantic box. If a detected bottleneck coincides with a text mark, it is determined to be a non-bottom structure and is not closed.

[0054] Specifically, the detection of dashed line gaps includes: scanning along the rows and columns of the image to identify white gaps, and extracting the gap length, the length and thickness of the wall segments on both sides; only when there are at least two candidate gaps on the same straight line that meet the following conditions: the gap length is less than a threshold, the length of the wall segments on both sides is greater than the minimum support length, the ratio of the thickness of the wall lines on both sides is less than a preset value, and there is an intersection in the vertical position, is the area determined to be a dashed line structure and is then filled in.

[0055] The text collision avoidance strategy specifically involves compressing the text region recognized by OCR according to a preset ratio to generate a protection mask; before repairing the detected doorway or dotted line gap, the coverage area of ​​the repair block is simulated, and if the area intersects with the protection mask, the repair operation is abandoned.

[0056] Specifically, the gaps between the dashed lines and the bottlenecks in the doorway are inspected, mainly to distinguish between "dashed line markings" and "isolated openings".

[0057] The specific steps for creating gaps between dashed lines are as follows: i) Gap Search and Feature Extraction: Scan the binarized image along rows (or columns) to identify gaps formed by white pixels. Extract the gap length. Length of wall sections on both sides of the gap ( ) and the vertical thickness of the side wall lines ( ).

[0058] ii) Multi-heuristic constraint filtering: Length constraint: gap length It must be less than the preset maximum threshold, and the wall sections on both sides must have sufficient support length, that is... At the same time, it is greater than the preset minimum support length threshold.

[0059] Thickness consistency verification: the thickness ratio of the two side wall lines The ratio should be smaller than a preset ratio (preferably 2.0), and the intersection of their vertical positions should be smaller than a preset ratio. It must be greater than zero to ensure that they belong to the same straight line.

[0060] iii) Collective verification logic: On the same horizontal (or vertical) straight line, the line is only determined to be a "dashed line" when the number of candidate gaps N that meet the above conditions is greater than or equal to 2. This effectively prevents the algorithm from mistakenly blocking isolated gaps.

[0061] For dashed structures that have passed all validation mechanisms, this project will complete them.

[0062] 2) The bottleneck of the doorway is accurately located by analyzing the changing trend of the "width field". The specific operation is as follows: i) Width Map Generation: Calculate the horizontal width map of the entire image. and vertical width diagram The value of each pixel represents the geometric width of its connected region in the corresponding direction.

[0063] ii) One-dimensional Gaussian smoothing noise reduction: The width profile is processed by one-dimensional Gaussian filtering to eliminate width fluctuations caused by scanning noise or wall line burrs.

[0064] iii) Bottleneck identification: A sliding window search is used to find the local minimum bottleneck. The span k is determined by the average line thickness obtained. and average spacing Decide:

[0065] The "Round" option indicates that the result is rounded to the nearest integer.

[0066] If the current bottleneck width W(y) is significantly smaller than its neighborhood width W(y±k), and satisfies If the value is 1, it is marked as an effective bottleneck, and the preferred Ratio is 1.2.

[0067] iv) Neighborhood overlap check: Calculate the width of the projected intersection of the wide regions above and below (or to the left and right) of the bottleneck. Only when the intersection width is greater than the width of the current bottleneck position is it confirmed as a doorway area that needs to be closed. This is mainly used to eliminate false doorways formed by the alignment of corners of different rooms.

[0068] 3) Collision-escape security filling mechanism based on text-protected mask To prevent the repair algorithm from disrupting the connectivity of text-occupied areas in the drawing, an OCR-linked protection mechanism is introduced, as follows: i) Protection Mask Generation: The text region recognized by OCR is compressed by a certain ratio (preferably 0.8) and then converted into a highlight protection mask. .

[0069] ii) Repair Simulation: Simulate repairing the partially identified bottleneck in the doorway bottleneck, thus calculating the coverage area of ​​the filling block in the actual space. .

[0070] iii) Decision-making logic: If If the area to be repaired overlaps with the text area, the system will forcibly abandon the repair task and maintain the connectivity of the area; otherwise, the repair will proceed. This ensures that the repair process is "semantically safe" and will not cut off key text annotations due to closed wireframes.

[0071] S4 Region Semantic Segmentation and Attribute Judgment Steps: Based on the closed wireframe obtained in S3, each region is assigned exclusive, non-exclusive, and passageway labels; for non-exclusive regions and passageways, synchronous expansion is performed using OCR boxes as seeds, and boundaries are detected through real-time connected component analysis, automatically stopping at the color boundaries of different entries to form colored regions.

[0072] Using the geometric center coordinates of the bounding box of each risk label text as the seed point, a flooding fill algorithm is executed to generate the corresponding initial region mask; the overlap between each region mask is calculated, and the disjoint-set data structure algorithm is used to aggregate masks with high overlap into the same logical group, and each region is assigned an exclusive or non-exclusive label accordingly; through morphological dilation and collision detection, the number of adjacent regions of each logical group is counted, and if the number of adjacent regions exceeds a preset threshold, the region is assigned a passage label.

[0073] The specific determination of aisle labels is as follows: For each logical group region, morphological dilation of its region mask is performed until it intersects with the pixel masks of other regions to determine their adjacency; the number of all distinct adjacent regions in that logical group region is counted, and if this number exceeds a certain threshold, the region is determined to be an aisle. Specific steps: 1) Initial region mask generation based on flood filling Before performing topological and semantic analysis, the initial region mask generation based on floodfill requires obtaining the physical spatial boundary corresponding to each risk term. Based on the repaired closed binarized wireframe, the center of the coordinate frame of each term (risk point) is used as a seed point, and the floodfill algorithm is applied to perform pixel-level connected component diffusion. The filling process automatically stops when it touches the black wall boundary, thus extracting the corresponding initial region mask set for each of the n risk terms in the drawing. These masks visually define the independent physical rooms or isolated spaces where each risk point is located, laying the foundation for subsequent calculations of overlap and topological relationships.

[0074] 2) Overlapping mask aggregation based on disjoint-set data structure Since there may be multiple risk points in the same area of ​​the factory, the "exclusivity" of the risk point is first determined by calculating whether the mask overlaps with other masks: Overlap verification: Calculate the overlap between any two masks. and The intersection-over-union ratio (IoU) or one-way coverage. If (Preferably 0.5) then it is determined that the risk points corresponding to the two are in the same physical space. ).

[0075] like Figure 7 The image shown is the original image that needs to be colored. Figure 11 The area mask for the "embossing machine" in the middle room is on the left side of the first row. Figure 11 The mask for the "manual workbench" in the middle room is on the right side of the first row. Since the two overlap, they are both determined to be "non-exclusive". Figure 11 The second row in the middle is the mask for the "Hazardous Chemicals Warehouse" in the room on the left. Since there are no other masks that overlap with it, it is determined to be "exclusive".

[0076] Logical merging: The disjoint-set data structure algorithm is used to group mask indices with high overlap into the same logical group, avoiding duplicate counting due to dense labeling and ensuring the accuracy of subsequent neighbor statistics.

[0077] 3) Topological adjacency calculation based on "expansion detection" Establish a connectivity graph between regions through spatial search: Preliminary screening of the bounding rectangle: Initial screening is performed by calculating the Euclidean distance of the bounding rectangles of all region masks pairwise. For a given mask, its bounding rectangle is calculated as follows: find the minimum and maximum values ​​of all pixels in the mask in the x and y directions, and the rectangle formed by the envelope of these four values ​​is its bounding rectangle.

[0078] Dynamic expansion collision: for For regions smaller than the safety threshold, perform a morphological dilation operation on one of the masks (using an elliptical structuring element, with a radius of...). Dynamically defined, its value depends on the line thickness and... The value of (preferably equal to the k value mentioned above) and the dilation radius are the distance thresholds used to determine whether two regions are adjacent. The dilation method is a relatively simple algorithm for calculating the distance between the outer contours of two objects: when one shape is dilated by x pixels, the two shapes become connected, and the distance between the two shapes is x.

[0079] Adjacency determination: If the expanded mask has a pixel-level intersection with another mask, then these two regions are marked as "near neighbors". This prepares for the subsequent third step of counting the number of neighbors.

[0080] 4) Semantic discrimination of "corridors" based on neighbor counting threshold After acquiring the topological connectivity graph, the system infers the semantic attributes of nodes by analyzing their degrees. Degree statistics: Count the number of distinct external neighbors associated with each region group (NeighborCount).

[0081] Corridor determination logic: Corridors play a transit role in the building structure, and their physical boundaries are usually connected to a large number of rooms.

[0082] Threshold trigger: If (Preferably 8), then the area is marked as Otherwise, mark as .

[0083] Therefore, calculating the overlap can help determine whether a certain area is exclusively occupied by a single risk point or shared by multiple risk points, providing traversal information for subsequent coloring. Identifying corridors prevents large areas of corridors from being colored due to incomplete wireframes, ensuring that risky color blocks are concentrated in specific production or functional rooms.

[0084] As a preferred option, the process also includes the S5 intelligent color rendering step: based on the region mask obtained in S4 and its corresponding exclusive label and aisle label, combined with the fill completeness of the mask itself, a corresponding coloring strategy is selected for each region; the coloring strategy includes: directly filling exclusive, complete, and non-aisle masks; for non-exclusive or aisle masks, a coloring algorithm based on pixel-level competitive expansion is initiated to delineate the precise coloring boundaries of each region; for masks with incomplete filling, their corresponding OCR text bounding boxes are used as the coloring areas; finally, a minimum value fusion algorithm is used to mix and render the target color with the original drawing pixels to generate the final risk four-color image.

[0085] Specifically, the pixel-level competitive expansion-based coloring algorithm is as follows: each region to be expanded is assigned a unique ID in the global label map, and each region moves one pixel in each direction in turn, centered on its own seed point; before each step, it is detected whether there are IDs of other regions in the adjacent pixels of the target direction, and if so, the expansion in that direction is stopped immediately, thereby forming an isolation zone between adjacent regions.

[0086] The minimum value fusion algorithm for hybrid rendering is specifically as follows: for each pixel that needs to be colored, the smaller value between the R, G, and B channel values ​​of the original pixel and the R, G, and B channel values ​​of the target color is taken as the final channel value of the output image at that pixel.

[0087] Specifically, such as Figures 2-6 The diagram illustrates the process of handling changes to an engineering drawing. Figures 2-6 The images show the original engineering drawing, the results of text removal, the results of wall line expansion and background noise removal, the results of dashed lines and doorway closure, and the final coloring effect. The complete coloring decision-making process is as follows: 1) Multi-criteria coloring strategy decision logic Based on the geometric quality, topological exclusivity, and channel semantic attributes of the region mask, the optimal solution is automatically selected from the following three execution paths: High-confidence exclusive mode: Triggering condition: The area is determined to be "exclusive" ( The mask quality meets the standards (complete internal filling) and is not located in a passageway area. ).

[0088] Execution: Directly use the generated mask for pixel-level coloring.

[0089] Dynamic expansion mode: Triggering condition: The mask is complete but is judged as non-exclusive (multiple terms competing). ) or belongs to the passageway area ( ).

[0090] Execution: Add the task to the "Competitive Growth Task Pool" for subsequent iteration space allocation.

[0091] ROI Minimization Fallback Mode: Triggering condition: When the mask is determined to be incomplete (non-high quality) and cannot be effectively expanded.

[0092] Execution: Use the bounding box of the OCR text box as the coloring area to prevent the loss of risk points.

[0093] 2) Competitive Growth Algorithm Based on Neighborhood Collision Detection For the expansion tasks in the competitive growth task pool in the above path, a fair competition strategy is adopted to define the physical boundaries: Label matrix initialization: Mark the initial IDs of all candidate text boxes in the global label graph L.

[0094] Competitive iterative growth: Each region expands outward by 1 pixel in turn according to the step sequence of "top-right-bottom-left".

[0095] Neighborhood hard isolation mechanism: Before each region step, the system scans the adjacent pixels in the target direction of the region.

[0096] Judgment criterion: If a heterogeneous ID (label from another region) exists in the neighborhood, growth in that direction is immediately stopped. This hard isolation mechanism enforces a 1-pixel-wide safety barrier between two color regions at the algorithm level, solving the problem of color overlap and contamination in densely labeled areas.

[0097] 3) Maximal Connectivity Refinement and Min-Blend Rendering Spatial intersection and cleaning: The intersection of the LabelMap generated by the competitive growth algorithm and the original segmentation mask is taken, and the maximal connected components are extracted. Scattered debris blocks caused by broken lines are removed to make the coloring edge logic consistent.

[0098] High-fidelity "color without line overlap" rendering: Employing a pixel-level minimum value fusion algorithm, for each pixel that needs to be colored, the R, G, and B values ​​of the corresponding pixel in the original image are compared with the R, G, and B values ​​of the color to be applied, and the minimum value is used for coloring.

[0099] Taking advantage of the "black lines on a white background" characteristic of industrial drawings, colored pixels are automatically discarded when they are minimized by black pixels (close to 0), so that the text, axes, and dimensions of the original drawing naturally "float" on the color blocks without the need for manual adjustment of transparency.

[0100] In summary, this invention systematically solves the core problems that have long existed in the automatic processing of engineering drawings, such as wireframe damage, poor adaptability, rampant color leakage, and semantic indistinction, through a series of interconnected technical features, including pixel-level non-destructive erasure, environmentally aware parameter decision-making, multi-constraint topology repair, semantically aware region segmentation, and intelligent competitive coloring. Ultimately, it achieves high-precision, high-robustness, and high-logic fully automatic risk area identification and coloring, which has significant practical value and promising industrial application prospects.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The several embodiments described in this application are quite specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and refinements should also be considered within the scope of protection of this invention. Therefore, the scope of protection of this patent application should be determined by the appended claims.

Claims

1. A method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion, characterized in that, Includes the following steps: S1 Drawing Text Parsing and Non-destructive Erasing Steps: For the input engineering drawing, use the OCR model to perform text detection and recognition to obtain the text content and its bounding box coordinates; use the SAM2 model to perform pixel-level segmentation within the text content and its bounding box coordinate area, and extract only the mask of the area occupied by the character itself. Based on the mask, the corresponding pixel area in the drawing is erased to preserve the wireframe structure outside the text area; S2 wireframe feature analysis and parameter decision steps: Binarize the drawing after S1 processing, extract the wall line skeleton and calculate the global average line width; use the global average line width as a dynamic parameter to adjust the parameters of subsequent processing algorithms; S3 wireframe topology repair steps: Based on the text position information obtained in S1 and the dynamic parameters obtained in S2, the gaps between dashed lines and bottlenecks in the drawing are detected, and multiple geometric constraints and text collision avoidance strategies are adopted to perform pixel-level completion of the gaps between dashed lines and the structure of the doorway that meet the conditions, forming a topologically closed wireframe diagram. S4 Region Semantic Segmentation and Attribute Discrimination: Based on the closed wireframe obtained in S3, each region is assigned exclusive, non-exclusive, and passageway labels; for non-exclusive regions and passageways, synchronous expansion is performed using OCR boxes as seeds, and boundaries are detected through real-time connected component analysis, automatically stopping at the color boundaries of different entries to form colored regions.

2. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 1, characterized in that, For the text content and its bounding box coordinates in step S1, semantic verification and spatial reconstruction of the bounding box coordinates can be performed based on a preset standard term knowledge base to obtain an optimized text bounding box; using the optimized text bounding box as a prompt, the visual segmentation SAM2 model is called to perform pixel-level instance segmentation to generate an accurate text mask.

3. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 1, characterized in that, The normal direction of the wall lines is calculated based on the structural tensor, and spatial exploration is performed along the normal direction to statistically analyze the proportion of skeleton pixels with parallel symmetry. Based on this, it is determined whether the drawing is a single-layer solid line or double-layer hollow line style, and the average spacing of the double-layer wall lines is extracted. The average spacing is used as a dynamic parameter to adjust the parameters of subsequent processing algorithms.

4. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 1, characterized in that, The detection of the dashed line gap in step S3 includes: scanning along the rows and columns of the image to identify white gaps, and extracting the gap length, the length and thickness of the wall segments on both sides; only when there are at least two candidate gaps on the same straight line that meet the following conditions: the gap length is less than the threshold, the length of the wall segments on both sides is greater than the minimum support length, the thickness ratio of the wall lines on both sides is less than the preset value, and there is an intersection in the vertical position, is the location determined to be a dashed line structure and is then completed.

5. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 1, characterized in that, The text collision avoidance strategy described in step S3 is as follows: the text area recognized by OCR is compressed according to a preset ratio to generate a protection mask; before repairing the detected doorway or dotted line gap, the coverage area of ​​the repair block is simulated; if the coverage area intersects with the protection mask, the repair operation is abandoned.

6. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 1, characterized in that, The process of assigning exclusive, non-exclusive, and passageway labels to each region is as follows: using the geometric center coordinates of the bounding box of each risk label text as the seed point, a flooding fill algorithm is executed to generate the corresponding initial region mask; Calculate the overlap between the masks of each region, use the disjoint-set data structure algorithm to aggregate the masks with high overlap into the same logical group, and assign exclusive or non-exclusive labels to each region accordingly; through morphological dilation and collision detection, count the number of adjacent regions of each logical group region, and if the number of adjacent regions exceeds a preset threshold, assign a passage label to that region.

7. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 6, characterized in that, The specific criteria for determining the aisle labels are as follows: For each logical group region, its region mask is morphologically expanded until it intersects with the pixel masks of other regions to determine their adjacency; the number of all distinct adjacent regions of the logical group region is counted, and if the number is greater than a certain threshold, the region is determined to be a passageway.

8. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 1, characterized in that, It also includes an intelligent coloring and rendering step: based on the region mask obtained in S4 and its corresponding exclusive label and aisle label, combined with the fill completeness of the mask itself, a corresponding coloring strategy is selected for each region; the coloring strategy includes: directly filling exclusive, complete, and non-aisle masks; for non-exclusive or aisle masks, a coloring algorithm based on pixel-level competitive expansion is initiated to delineate the precise coloring boundary of each region; for incompletely filled masks, the corresponding OCR text bounding box is used as the coloring area; finally, the minimum value fusion algorithm is used to mix and render the target color with the original drawing pixels to generate the final risk four-color image.

9. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 8, characterized in that, The pixel-level competitive expansion-based coloring algorithm is as follows: each region to be expanded is assigned a unique ID in the global label map, and each region moves one pixel in each direction in turn, centered on its own seed point, according to a preset order; before each step, it is detected whether there are IDs of other regions in the adjacent pixels of the target direction, and if so, the expansion in that direction is stopped immediately, thereby forming an isolation zone between adjacent regions.

10. The method for automatic identification and coloring of risk areas in complex engineering drawings based on multi-model fusion as described in claim 8, characterized in that, The minimum value fusion algorithm for hybrid rendering is specifically as follows: for each pixel that needs to be colored, the smaller value between the R, G, and B channel values ​​of the original pixel and the R, G, and B channel values ​​of the target color is taken as the final channel value of the output image at that pixel.

Images