Intelligent drawing review method based on geometric topology and counterfactual reasoning and application thereof

Abstract

This invention relates to the field of intelligent review of engineering drawings and computer-aided design technology, and discloses an intelligent drawing review method based on geometric topology and counterfactual reasoning and its application. The method includes: performing structured modeling on a sequence of vector primitives to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed; converting the normative element triples into differentiable constraint functions with respect to constraint types; performing constraint reasoning and counterfactual optimization on the geometric topology diagram using the differentiable constraint functions to obtain modification operation vectors and violation heatmaps; inputting the modification operation vectors and violation heatmaps into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and outputting a structured drawing review report. This invention breaks away from the traditional multimodal large language model's "end-to-end illusion-style drawing review" paradigm, achieving high-precision geometric compliance checks, executable CAD-level modification output, and interpretable drawing review results with a low illusion rate.

Description

Technical Field

[0001] This invention relates to the field of intelligent review of engineering drawings and computer-aided design technology, and in particular to an intelligent review method based on geometric topology and counterfactual reasoning and its application. Background Technology

[0002] In professional fields such as architectural engineering, mechanical manufacturing, and electrical design, drawings are the core carriers of design intent and technical specifications. Traditional drawing review relies on manual review by experts, which suffers from low efficiency, inconsistent standards, and susceptibility to oversights. In recent years, with the rapid development of multimodal large language models, some automated drawing recognition and review systems have begun to explore the use of image and text understanding capabilities for drawing review.

[0003] However, existing technologies have fundamental shortcomings. Multimodal large language models treat drawings as pixel sequences and extract features through implicit self-attention mechanisms. They lack the ability to explicitly model strict geometric constraints (such as parallelism, perpendicularity, and collinearity) and spatial topological relationships (such as connection, containment, and adjacency) in drawings, resulting in poor numerical accuracy and difficulty in meeting the millimeter-level drawing review requirements in engineering fields. Furthermore, existing technologies heavily rely on large-scale annotated graphic data for fine-tuning, while professional engineering drawings are diverse and have high annotation costs. Additionally, large models are prone to producing illusions, such as reporting non-existent violations in the drawings, severely reducing the credibility of the review results. Moreover, existing technologies struggle to convert abstract physical rules and engineering specifications (such as fire separation distances, evacuation distances, and rebar anchorage lengths) into calculable model losses and cannot output specific, actionable modification suggestions, only providing general qualitative judgments.

[0004] Therefore, how to achieve intelligent review of professional drawings without relying on the illusion of the omnipotence of multimodal large language models has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned issues, this invention aims to provide an intelligent drawing review method and its application based on geometric topology and counterfactual reasoning. By constructing a three-layer heterogeneous intelligent agent consisting of "differentiable logic construction - graph neural network reasoning - multimodal model counterfactual correction", it achieves efficient, accurate, interpretable, and executable automated review of engineering drawings.

[0006] In a first aspect, embodiments of the present invention provide an intelligent image review method based on geometric topology and counterfactual reasoning, including: The input drawing to be reviewed is vectorized to obtain a vector primitive sequence, and the vector primitive sequence is structured to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed. The input engineering specification text is converted into a specification element triple containing constraint subject, constraint type and threshold parameter, and the specification element triple is converted into a differentiable constraint function with respect to the constraint type. By combining the differentiable constraint function with constraint reasoning and counterfactual optimization of the geometric topology, a modification operation vector that can make the drawing to be reviewed meet the engineering specification requirements and a violation heat map reflecting the distribution of the degree of violation of the drawing to be reviewed are obtained. The modification operation vector and the violation heatmap are input into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and a structured review report is output, which includes violation details, regulatory basis, executable modification instructions and visualization results.

[0007] Preferably, the step of vectorizing the input drawing to be reviewed to obtain a vector primitive sequence, and performing structured modeling on the vector primitive sequence to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed, includes: The system receives drawings to be reviewed from users via the front-end interface. Input methods include local file upload, network path import, and scanning device acquisition. Based on the format of the drawing to be reviewed, the drawing to be reviewed is read or vectorized to obtain a vector primitive sequence; Using each primitive in the vector primitive sequence as a node and the spatial relationships between primitives in the vector primitive sequence as edges, a geometric topology graph is constructed that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed.

[0008] Preferably, the step of converting the input engineering specification text into a specification element triple containing constraint subject, constraint type, and threshold parameter, and converting the specification element triple into a differentiable constraint function with respect to the constraint type, includes: Receive engineering specification text input by the user through the front-end interface; Natural language processing technology is used to extract the constraint subject, constraint type, and threshold parameter from the engineering specification text to form a specification element triplet; Select the constraint function template corresponding to the normative element triplet based on the constraint type; The threshold parameter is filled into the constraint function template to obtain a differentiable constraint function for the constraint type.

[0009] Preferably, the step of combining the differentiable constraint function to perform constraint reasoning and counterfactual optimization on the geometric topology to obtain a modification operation vector that enables the drawing to be reviewed to meet engineering specification requirements and a violation heatmap reflecting the distribution of the degree of violation in the drawing to be reviewed includes: By combining the differentiable constraint function with the geometric topology diagram for constraint reasoning, the overall violation loss value describing the degree of deviation between the drawing to be reviewed and the engineering specification requirements is obtained; With the goal of minimizing the overall violation loss value, the gradient descent method is used to iteratively optimize the original geometric parameters of each node in the geometric topology graph to obtain the optimized geometric parameters of the corresponding node. The difference between the original geometric parameters and the optimized geometric parameters of each node is transformed into a modification operation vector that enables the drawing to be reviewed to meet the engineering specifications. The violation confidence level of each node and the total violation loss value are mapped to the drawing to be reviewed, generating a violation heatmap that reflects the distribution of the violation degree of the drawing to be reviewed.

[0010] Preferably, the step of combining the differentiable constraint function to perform constraint reasoning on the geometric topology to obtain the overall violation loss value describing the degree of deviation between the drawing to be reviewed and the engineering specification requirements includes: By combining the differentiable constraint function, a heterogeneous graph neural network is used to perform constraint reasoning on the geometric topology graph to obtain the overall violation loss value describing the degree of deviation between the drawing to be reviewed and the engineering specification requirements.

[0011] Preferably, the step of inputting the modification operation vector and the violation heatmap into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and outputting a structured review report containing violation details, regulatory basis, executable modification instructions, and visualization results, includes: Based on the aforementioned heatmap of violations, the original image blocks corresponding to the violation areas are cropped from the drawings to be reviewed; The original image block is modified according to the modification operation vector to obtain the modified image block; The original image patch, the modified image patch, and the engineering specification text are input into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and a structured review report is output, which includes violation details, specification basis, executable modification instructions, and visualization results.

[0012] Preferably, the step of inputting the original image patch, the modified image patch, and the engineering specification text into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and outputting a structured review report containing violation details, specification basis, executable modification instructions, and visualization results, includes: The modified image block is semantically validated using the multimodal large language model to determine whether it conforms to the engineering specification requirements corresponding to the engineering specification text. Based on the original image block, the modified image block, and the engineering specification text, a structured review report containing violation details, specification basis, executable modification instructions, and visualization results is generated using the multimodal large language model.

[0013] Secondly, embodiments of the present invention provide an intelligent image review system based on geometric topology and counterfactual reasoning, applied to the intelligent image review method based on geometric topology and counterfactual reasoning as described above. The system includes: The vector processing and structural modeling module is used to vectorize the input drawing to be reviewed to obtain a vector primitive sequence, and to perform structural modeling on the vector primitive sequence to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed. The text conversion and element transformation module is used to convert the input engineering specification text into a specification element triplet containing constraint subject, constraint type and threshold parameter, and to convert the specification element triplet into a differentiable constraint function with respect to the constraint type. The constraint reasoning and counterfactual optimization module is used to perform constraint reasoning and counterfactual optimization on the geometric topology graph in combination with the differentiable constraint function, so as to obtain the modification operation vector that can make the drawing to be reviewed meet the engineering specification requirements and the violation heat map reflecting the distribution of the degree of violation of the drawing to be reviewed. The semantic verification and compliance interpretation module is used to input the modification operation vector and the violation heatmap into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and output a structured review report containing violation details, regulatory basis, executable modification instructions and visualization results.

[0014] Thirdly, embodiments of the present invention provide a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the intelligent image review method based on geometric topology and counterfactual reasoning as described above.

[0015] Fourthly, embodiments of the present invention provide a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the intelligent image review method based on geometric topology and counterfactual reasoning as described above.

[0016] This invention discloses an intelligent drawing review method based on geometric topology and counterfactual reasoning, and its application. Compared with existing technologies, its advantages are as follows: By vectorizing the drawings to be reviewed and constructing an explicit geometric topology diagram, pixel-level fuzzy modeling is abandoned, accurately restoring the geometric constraints and spatial topological relationships of the drawings, and ensuring the accuracy of the review; by converting engineering specification text into differentiable constraint functions, the engineering specifications can be expressed in a computable manner, and constraint reasoning is completed by combining heterogeneous graph neural networks. Modification operation vectors are generated based on counterfactual optimization, and directly executable modification instructions are output; by using a multimodal large language model to only undertake semantic verification and compliance interpretation, the risk of large model illusion is avoided, while generating traceable and easy-to-understand review results, ultimately improving the efficiency, reliability, and engineering adaptability of the review. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an intelligent image review method based on geometric topology and counterfactual reasoning according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the constraint reasoning and counterfactual optimization process according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the semantic verification and compliance interpretation process in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an intelligent image review system based on geometric topology and counterfactual reasoning according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a terminal device according to an embodiment of the present invention; Figure label: 01. Vector Processing and Structural Modeling Module; 02. Text Conversion and Feature Transformation Module; 03. Constraint Reasoning and Counterfactual Optimization Module; 04. Semantic Validation and Compliance Interpretation Module; 5000. Terminal Device; 5001. Processor; 5002. Bus; 5003. Memory; 5004. Transceiver. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] In the description of this invention, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by those skilled in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0020] like Figure 1The diagram shown is a flowchart illustrating an intelligent image review method based on geometric topology and counterfactual reasoning according to an embodiment of the present invention. (Refer to...) Figure 1 This invention provides an intelligent image review method based on geometric topology and counterfactual reasoning, comprising the following steps: S1. The input drawing to be reviewed is vectorized to obtain a vector primitive sequence, and the vector primitive sequence is structurally modeled to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed. Specifically, step S1 includes: 1) Receive drawings to be reviewed from users via the front-end interface; The system's front-end interface provides a standardized drawing input interface, supporting three input methods: local file upload, network path import, and scanning device acquisition, fully adapting to mainstream engineering drawing submission scenarios.

[0021] The received drawings for review cover CAD vector formats such as DWG and DXF, as well as raster image formats such as PDF, JPEG, and PNG, covering commonly used drawing types in fields such as architecture, machinery, and electrical engineering. The input data is a complete drawing file, containing original data such as drawing dimensions, layer information, element distribution, and annotation content, providing a complete data source for subsequent vectorization processing.

[0022] 2) Based on the format of the drawing to be reviewed, the data of the drawing to be reviewed is read or vectorized to obtain a vector primitive sequence; Specifically, different processing methods are adopted for different drawing formats: For CAD vector format drawings such as DWG and DXF, the binary entity data structure inside the file is read directly to extract core graphic elements such as line segments, arcs, polylines, dimensions, blocks, and text annotations. Each graphic element is assigned a unique identifier, and parameters such as coordinates, dimensions, layers, and geometric attributes are recorded synchronously. The data comes directly from the original vector information of the CAD file, with no loss of precision.

[0023] For raster format drawings such as PDF, JPEG, and PNG, a deep learning vector reconstruction algorithm is invoked. First, image preprocessing is completed through noise reduction, contrast enhancement, and edge detection. Then, features such as lines, closed regions, and text annotations are extracted based on the contour recognition algorithm. The contour recognition confidence threshold is set to 0.8 and the pixel matching threshold is set to 1px, and the pixel information is converted into standard vector primitives.

[0024] The final output is a sequence of vector primitives containing unique IDs, types, coordinates, dimensions, layers, and attribute parameters, enabling a unified vectorized representation of multi-format drawings and providing a standardized data foundation for geometric topology modeling.

[0025] 3) Using each primitive in the vector primitive sequence as a node and the spatial relationships between primitives in the vector primitive sequence as edges, construct a geometric topology graph that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed.

[0026] A geometric topology graph is constructed using spatial indexing techniques from computational geometry. The process is explained in detail below: ① Node construction: Each independent primitive in the vector primitive sequence is treated as a unique node, and the node attributes include primitive type, coordinates, size, layer, unique ID, and geometric parameters; ② Construction: Geometric / spatial relationships between primitives are identified through spatial proximity detection, and standardized judgment thresholds are set, including: connecting edge: distance between the endpoints of two primitives ≤ 2px; parallel edge: angle between direction vectors ≤ 5°; perpendicular edge: angle 90°±3°; coaxial edge: center distance ≤ 1px, radius difference ≤ 5%; containing edge: polygon boundary completely encloses another primitive (coordinate difference ≤ 1px).

[0027] Edge attributes record the relationship type and confidence level, and finally output a geometric topology diagram in JSON format, which explicitly carries the geometric constraints (parallel, perpendicular, coaxial) and spatial topology (connection, containment) relationships of the drawing.

[0028] S2. Convert the input engineering specification text into a specification element triplet containing constraint subject, constraint type and threshold parameter, and convert the specification element triplet into a differentiable constraint function with respect to constraint type; Specifically, step S2 includes: 1) Receive engineering specification text input by the user through the front-end interface; Users can submit various engineering specification texts through the system's front-end interface. These texts cover current national standards, industry-wide specifications, enterprise internal control standards, and specific drawing review rules. The system supports multiple input formats, including plain text, structured documents, and excerpts from specifications. The input content includes the complete original text of the specification clauses, explanations of applicable scenarios, and specific constraints, providing a complete and authoritative compliance basis for subsequent specification analysis.

[0029] 2) Natural language processing technology is used to extract constraint subjects, constraint types, and threshold parameters from engineering specification texts to form specification element triples; It employs a lightweight natural language processing technology stack, sequentially performing text segmentation, part-of-speech tagging, terminology matching, named entity recognition, dependency parsing, and semantic role labeling.

[0030] Specifically, the process first involves segmenting and tagging sentences by word segmentation and part-of-speech tagging, filtering out invalid function words using an engineering terminology database, and identifying core entities. Then, named entity recognition accurately extracts constraint subjects (e.g., primitives, components, spatial objects), constraint types (e.g., distance, angle, area, containment, proportional constraints), and threshold parameters (including numerical values, units, and comparison operators). Finally, syntactic and semantic analysis clarifies logical relationships, eliminates ambiguous expressions, and outputs standardized triplet elements (constraint subject, constraint type, and threshold parameter) with a uniform format and unique semantics.

[0031] For example, for the specification that "the distance from an evacuation door to the nearest safe exit should not exceed fifteen meters," natural language processing technology can identify the constraint subjects as "evacuation door" and "safe exit," the constraint type as "distance constraint," the comparison operator as "not greater than," and the value as "fifteen meters." Based on this, dependency parsing and semantic role labeling are used to further clarify the grammatical and logical relationships between the various elements, ultimately converting each engineering specification text into structured specification element triples. These triples express "object, condition, and requirement" in a unified and machine-readable form.

[0032] 3) Select the constraint function template corresponding to the triplet of the specification element based on the constraint type; The system has a built-in pre-built constraint function template library. The template library stores standardized function templates according to constraint type, covering core constraint types for engineering drawing review such as distance, angle, area, inclusion relationship, scale, and position. The templates are based on a general differentiable loss function structure, which is adapted to the inference needs of heterogeneous graph neural networks.

[0033] The constraint type field in the triplet of the specification element is parsed, and the corresponding template is matched through the type-template mapping rule. At the same time, the consistency between the applicable scenario of the template and the applicable scope of the specification is verified, and the constraint function template with the best adaptability is selected. The templates are all predefined and differentiable mathematical function frameworks, which provide a standardized carrier for subsequent parameter filling.

[0034] 4) Fill the threshold parameter into the constraint function template to obtain a differentiable constraint function with respect to the constraint type.

[0035] The threshold parameters in the triplet of the standard elements are standardized by unifying the numerical units and converting them to floating-point data, and the parameter format deviation is corrected. The processed threshold parameters are then substituted into the matching constraint function template, and the parameter placeholders in the template are replaced to generate a differentiable constraint function of the corresponding constraint type.

[0036] Differentiable constraint functions are constructed in the form of loss functions, satisfy the continuous differentiability property, support gradient backpropagation, and can be directly input into graph neural networks for constraint inference and violation loss calculation. The final output function contains explicit constraint logic, parameter thresholds, and applicable primitive types, and can be directly used for drawing compliance verification.

[0037] For example This function is a differentiable loss function corresponding to the distance constraint, where d represents the actual spatial distance between two target primitives in the drawing. When the actual spatial distance is no more than 15 meters, the loss value is zero (compliant). When the actual spatial distance is greater than 15 meters, the loss value increases with the increase of the deviation (non-compliant). It can realize the quantitative assessment of distance compliance and gradient backpropagation.

[0038] S3. Combine differentiable constraint functions to perform constraint reasoning and counterfactual optimization on the geometric topology diagram to obtain modification operation vectors that can make the drawings under review meet the engineering specifications and violation heat map reflecting the distribution of the degree of violation of the drawings under review. like Figure 2 As shown, this is a flowchart illustrating step S3. (Refer to...) Figure 2 Step S3 includes: S301. Combine differentiable constraint functions to perform constraint reasoning on the geometric topology diagram to obtain the overall violation loss value describing the degree of deviation between the drawings to be reviewed and the engineering specification requirements; By combining differentiable constraint functions, a heterogeneous graph neural network is used to perform constraint reasoning on the geometric topology graph to obtain the overall violation loss value describing the degree of deviation between the drawings to be reviewed and the engineering specification requirements.

[0039] Specifically, the heterogeneous graph neural network configures independent feature transformation weight matrices for different node types (component primitives such as doors, windows, walls, pipelines, etc.) and edge types (geometric relationships such as connection, parallelism, and inclusion) in the geometric topology graph. Through multi-layer cross-type message passing and neighborhood feature aggregation, the latent features of each node are integrated with its own geometric parameters and surrounding topological association information.

[0040] Furthermore, the system iterates through all differentiable constraint functions in the differentiable constraint rule library, calculating the sub-item violation loss for each single node and edge. The weighted sum of all sub-item violation losses yields the overall violation loss value for the entire drawing. This loss value directly quantifies the deviation of the drawing from the engineering specifications. The differentiable constraint rule library stores all differentiable constraint functions generated from specification element triples through constraint function template matching and threshold parameter filling. It supports dynamic loading by drawing professional type, addition and deletion of constraint rules, and version management, providing a standardized and directly callable set of computable rules for constraint inference.

[0041] S302. With the goal of minimizing the overall violation loss, the gradient descent method is used to iteratively optimize the original geometric parameters of each node in the geometric topology graph to obtain the optimized geometric parameters of the corresponding node. An automatic differential algorithm is used to solve the gradient value of the overall violation loss value with respect to the original geometric parameters (coordinates, line segment length, arc radius, etc.) of each node. The gradient direction represents the optimization direction of parameter correction.

[0042] The mini-batch gradient descent algorithm is used for iterative optimization, and the learning rate is set to a range of values. The iteration termination threshold is when the total loss change between two consecutive iterations is less than 1%. The iterative process involves slightly correcting the primitive parameters along the negative gradient direction, repeating the loss calculation and parameter update process until the overall violation loss value converges to the threshold range, and finally outputting the optimized geometric parameters of each node after convergence.

[0043] It should be noted that the optimized geometric parameters obtained through iterative optimization correspond to a counterfactual drawing state; this state is a virtual compliant drawing form that meets all engineering specification constraints after reasonable modification of the original drawing element parameters, which is different from the actual factual state of the original drawing.

[0044] S303. Transform the difference between the original geometric parameters and the optimized geometric parameters of each node into a modification operation vector that can make the drawing to be reviewed meet the engineering specifications. The differences between the original and optimized geometric parameters in coordinate, dimension, and angle dimensions are calculated node by node. Based on the modification category corresponding to the difference (translation, scaling, sizing fine-tuning, position correction), vector components are decomposed. Each component value represents the modification magnitude and direction of movement in the corresponding dimension. These are uniformly encapsulated into standardized multidimensional modification operation vectors. For example, a translation operation includes the direction and distance of movement, a scaling operation includes the scaling center point and scale factor, and a replacement operation specifies the type of primitive to be replaced. Each modification operation vector corresponds to one primitive node, and the vector data can be directly parsed and converted into editing instructions recognizable by CAD software.

[0045] S304. Map the violation confidence level and total violation loss value of each node to the drawing to be reviewed, and generate a violation heat map reflecting the distribution of the violation degree of the drawing to be reviewed.

[0046] Based on the confidence prediction branch at the end of the heterogeneous graph neural network, and based on the final latent feature output of the node. The confidence level of a node violation within a given interval is such that the higher the confidence level is (closer to 1), the greater the violation of that element.

[0047] The confidence level of each node and the sub-item violation loss value are matched to the original pixel coordinates of the drawing to be reviewed. Multiple color gradients are divided according to the numerical range. The severity of the violation is mapped by the color depth. A visual violation heat map is generated by splicing the entire area, which intuitively marks the distribution of high-risk violation areas in the drawing to be reviewed.

[0048] S4. Input the modification operation vector and violation heatmap into the fine-tuned multimodal large language model for semantic verification and compliance interpretation, and output a structured review report containing violation details, regulatory basis, executable modification instructions and visualization results.

[0049] like Figure 3 As shown, this is a flowchart illustrating step S4. (Refer to...) Figure 3 Step S4 includes: S401. Based on the violation heatmap, cut out the original image block corresponding to the violation area from the drawing to be reviewed; Based on the pixel coordinates and bounding box information of each violation area marked by the violation heatmap, the local violation areas in the drawing to be reviewed are located, and the original image blocks are generated by cropping them to a preset size (preferably 512×512 pixels). This image block fully contains the violation elements and surrounding related components, and retains the visual information such as layers, line types, and dimension annotations of the original drawing, ensuring that the content of the image block can fully reflect the violation scene.

[0050] S402. Modify the original image block according to the modification operation vector to obtain the modified image block; The system analyzes and modifies parameters such as translation, scaling, size adjustment, and position correction in the operation vector, calls the drawing vector rendering engine, parametrically modifies the vector primitives corresponding to the original image block, and synchronously updates the primitive coordinates, size, and spatial topology. It then renders and generates a modified image block with the same size and viewpoint alignment as the original image block, accurately restoring the geometric shape and spatial layout of the modified primitives, ensuring that the image blocks before and after modification are directly comparable.

[0051] S403. Input the original image block, the modified image block, and the engineering specification text into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and output a structured review report containing violation details, specification basis, executable modification instructions, and visualization results.

[0052] Specifically, step S403 includes: 1) Use a multimodal large language model to perform semantic verification on the modified image blocks to determine whether the modified image blocks meet the engineering specification requirements corresponding to the engineering specification text; The multimodal large language model used is a lightweight model, which has been fine-tuned and trained with a small sample of engineering drawing images, standard texts and compliance modification cases, and has the ability to understand drawings visually, analyze engineering semantics and match standards.

[0053] A multimodal large language model extracts the visual features of the modified image blocks and parses the engineering semantics. It verifies the geometric rationality, spatial relationship compliance, and engineering implementation feasibility of the modified primitives, determines whether the modification results meet the constraints of the engineering specification text, and outputs the compliance verification results.

[0054] 2) Based on the original image blocks, the modified image blocks, and the engineering specification text, a structured review report containing violation details, specification basis, executable modification instructions, and visualization results is generated using a multimodal large language model.

[0055] This system integrates original image blocks, modified image blocks, violation heatmaps, node violation confidence levels, original engineering specification clauses, and compliance verification results, sorting them in descending order of violation severity to automatically generate a structured drawing review report. The report includes the unique ID of the violating element, precise coordinates, violation type, confidence level, specification name and clause, natural language compliance explanation, standardized CAD executable modification instructions, and before-and-after image comparison snapshots. Its format is compatible with mainstream engineering document tools, meeting the needs for drawing review result traceability, manual review, and engineering application.

[0056] This invention discloses an intelligent drawing review method based on geometric topology and counterfactual reasoning. By vectorizing the drawings to be reviewed and constructing an explicit geometric topology diagram, it abandons pixel-level fuzzy modeling and accurately restores the geometric constraints and spatial topological relationships of the drawings, ensuring review accuracy. By converting engineering specification text into differentiable constraint functions, it achieves a computable expression of engineering specifications. Constraint reasoning is completed using a heterogeneous graph neural network, and modification operation vectors are generated based on counterfactual optimization, outputting directly executable modification instructions. A multimodal large language model is used to handle only semantic verification and compliance interpretation, avoiding the risk of large model illusions. Simultaneously, it generates traceable and easily understandable review results, ultimately improving review efficiency, reliability, and engineering applicability.

[0057] like Figure 4 The diagram shown is a structural schematic of an intelligent drawing review system based on geometric topology and counterfactual reasoning according to an embodiment of the present invention. This intelligent drawing review system based on geometric topology and counterfactual reasoning is applied to the intelligent drawing review method based on geometric topology and counterfactual reasoning described above.

[0058] Reference Figure 4 An embodiment of the present invention provides an intelligent image review system based on geometric topology and counterfactual reasoning, comprising: The Vector Processing and Structural Modeling Module 01 is used to vectorize the input drawing to be reviewed to obtain a vector primitive sequence, and to perform structural modeling on the vector primitive sequence to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed. The text conversion and element conversion module 02 is used to convert the input engineering specification text into a specification element triplet containing constraint subject, constraint type and threshold parameter, and to convert the specification element triplet into a differentiable constraint function with respect to constraint type; The constraint reasoning and counterfactual optimization module 03 is used to perform constraint reasoning and counterfactual optimization on the geometric topology diagram by combining differentiable constraint functions, so as to obtain the modification operation vector that can make the drawing to be reviewed meet the engineering specification requirements and the violation heat map reflecting the distribution of the degree of violation of the drawing to be reviewed. The Semantic Validation and Compliance Explanation Module 04 is used to input the modification operation vector and violation heatmap into a finely tuned multimodal large language model for semantic validation and compliance explanation, and output a structured review report containing violation details, regulatory basis, executable modification instructions and visualization results.

[0059] It should be noted that the modules in the aforementioned intelligent drawing review system based on geometric topology and counterfactual reasoning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module. For specific limitations regarding the intelligent drawing review system based on geometric topology and counterfactual reasoning, please refer to the limitations of the intelligent drawing review method based on geometric topology and counterfactual reasoning mentioned above; both have the same function and role, and will not be repeated here.

[0060] This invention also provides a terminal device, which includes: Processor, memory, and bus; The bus is used to connect the processor and the memory; The memory is used to store operation instructions; The processor is configured to execute operations corresponding to the intelligent image review method based on geometric topology and counterfactual reasoning described above by invoking the operation instructions.

[0061] In one alternative embodiment, a terminal device is provided, such as Figure 5 As shown, Figure 5 The terminal device 5000 shown includes a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are connected, for example, via a bus 5002. Optionally, the terminal device 5000 may also include a transceiver 5004. It should be noted that in practical applications, the transceiver 5004 is not limited to one type, and the structure of this terminal device 5000 does not constitute a limitation on the embodiments of the present invention.

[0062] Processor 5001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 5001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0063] Bus 5002 may include a path for transmitting information between the aforementioned components. Bus 5002 may be a PCI bus or an EISA bus, etc. Bus 5002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0064] The memory 5003 may be a ROM or other type of static storage device capable of storing static information and instructions, RAM or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0065] The memory 5003 is used to store application code that executes the present invention, and its execution is controlled by the processor 5001. The processor 5001 is used to execute the application code stored in the memory 5003 to implement the content shown in any of the foregoing method embodiments.

[0066] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the intelligent image review method based on geometric topology and counterfactual reasoning described above.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] In summary, this invention provides an intelligent drawing review method and its application based on geometric topology and counterfactual reasoning. By vectorizing the drawings to be reviewed and constructing an explicit geometric topology diagram, it abandons pixel-level fuzzy modeling and accurately restores the geometric constraints and spatial topological relationships of the drawings, ensuring the accuracy of the review. By converting engineering specification text into differentiable constraint functions, it achieves a computable expression of engineering specifications. Combined with heterogeneous graph neural networks, it completes constraint reasoning and generates modification operation vectors based on counterfactual optimization, outputting directly executable modification instructions. By using a multimodal large language model to only undertake semantic verification and compliance interpretation, it avoids the risk of large model illusion and generates traceable and easy-to-understand review results, ultimately improving the efficiency, reliability, and engineering adaptability of the review.

[0072] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0073] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. An intelligent image review method based on geometric topology and counterfactual reasoning, characterized in that, include: The input drawing to be reviewed is vectorized to obtain a vector primitive sequence, and the vector primitive sequence is structured to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed. The input engineering specification text is converted into a specification element triple containing constraint subject, constraint type and threshold parameter, and the specification element triple is converted into a differentiable constraint function with respect to the constraint type. By combining the differentiable constraint function with constraint reasoning and counterfactual optimization of the geometric topology, a modification operation vector that can make the drawing to be reviewed meet the engineering specification requirements and a violation heat map reflecting the distribution of the degree of violation of the drawing to be reviewed are obtained. The modification operation vector and the violation heatmap are input into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and a structured review report is output, which includes violation details, regulatory basis, executable modification instructions and visualization results.

2. The intelligent image review method based on geometric topology and counterfactual reasoning according to claim 1, characterized in that, The process of vectorizing the input drawing to be reviewed to obtain a vector primitive sequence, and then performing structured modeling on the vector primitive sequence to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed, includes: The system receives drawings to be reviewed from users via the front-end interface. Input methods include local file upload, network path import, and scanning device acquisition. Based on the format of the drawing to be reviewed, the drawing to be reviewed is read or vectorized to obtain a vector primitive sequence; Using each primitive in the vector primitive sequence as a node and the spatial relationships between primitives in the vector primitive sequence as edges, a geometric topology graph is constructed that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed.

3. The intelligent image review method based on geometric topology and counterfactual reasoning according to claim 1, characterized in that, The process of converting the input engineering specification text into a specification element triplet containing constraint subject, constraint type, and threshold parameter, and then converting the specification element triplet into a differentiable constraint function with respect to the constraint type, includes: Receive engineering specification text input by the user through the front-end interface; Natural language processing technology is used to extract the constraint subject, constraint type, and threshold parameter from the engineering specification text to form a specification element triplet; Select the constraint function template corresponding to the normative element triplet based on the constraint type; The threshold parameter is filled into the constraint function template to obtain a differentiable constraint function for the constraint type.

4. The intelligent image review method based on geometric topology and counterfactual reasoning according to claim 1, characterized in that, The process of combining the differentiable constraint function with constraint reasoning and counterfactual optimization of the geometric topology graph yields a modification operation vector that enables the drawing to be reviewed to meet engineering specifications and a violation heatmap reflecting the distribution of the degree of violation in the drawing to be reviewed, including: By combining the differentiable constraint function with the geometric topology diagram for constraint reasoning, the overall violation loss value describing the degree of deviation between the drawing to be reviewed and the engineering specification requirements is obtained; With the goal of minimizing the overall violation loss value, the gradient descent method is used to iteratively optimize the original geometric parameters of each node in the geometric topology graph to obtain the optimized geometric parameters of the corresponding node. The difference between the original geometric parameters and the optimized geometric parameters of each node is transformed into a modification operation vector that enables the drawing to be reviewed to meet the engineering specifications. The violation confidence level of each node and the total violation loss value are mapped to the drawing to be reviewed, generating a violation heatmap that reflects the distribution of the violation degree of the drawing to be reviewed.

5. The intelligent image review method based on geometric topology and counterfactual reasoning according to claim 4, characterized in that, The constraint reasoning performed on the geometric topology diagram using the differentiable constraint function yields a total violation loss value describing the degree of deviation between the drawing to be reviewed and the engineering specification requirements, including: By combining the differentiable constraint function, a heterogeneous graph neural network is used to perform constraint reasoning on the geometric topology graph to obtain the overall violation loss value describing the degree of deviation between the drawing to be reviewed and the engineering specification requirements.

6. The intelligent image review method based on geometric topology and counterfactual reasoning according to claim 1, characterized in that, The modification operation vector and the violation heatmap are input into a finely tuned multimodal large language model for semantic verification and compliance interpretation. The output is a structured review report containing violation details, regulatory basis, executable modification instructions, and visualization results, including: Based on the aforementioned heatmap of violations, the original image blocks corresponding to the violation areas are cropped from the drawings to be reviewed; The original image block is modified according to the modification operation vector to obtain the modified image block; The original image patch, the modified image patch, and the engineering specification text are input into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and a structured review report is output, which includes violation details, specification basis, executable modification instructions, and visualization results.

7. The intelligent image review method based on geometric topology and counterfactual reasoning according to claim 6, characterized in that, The process involves inputting the original image patch, the modified image patch, and the engineering specification text into a finely tuned, multimodal large language model for semantic verification and compliance interpretation. The output is a structured review report containing violation details, specification basis, executable modification instructions, and visualization results, including: The modified image block is semantically validated using the multimodal large language model to determine whether it conforms to the engineering specification requirements corresponding to the engineering specification text. Based on the original image block, the modified image block, and the engineering specification text, a structured review report containing violation details, specification basis, executable modification instructions, and visualization results is generated using the multimodal large language model.

8. An intelligent image review system based on geometric topology and counterfactual reasoning, characterized in that, The system, applied to the intelligent image review method based on geometric topology and counterfactual reasoning as described in any one of claims 1 to 7, comprises: The vector processing and structural modeling module is used to vectorize the input drawing to be reviewed to obtain a vector primitive sequence, and to perform structural modeling on the vector primitive sequence to obtain a geometric topology diagram that explicitly describes the geometric constraints and spatial topology in the drawing to be reviewed. The text conversion and element transformation module is used to convert the input engineering specification text into a specification element triplet containing constraint subject, constraint type and threshold parameter, and to convert the specification element triplet into a differentiable constraint function with respect to the constraint type. The constraint reasoning and counterfactual optimization module is used to perform constraint reasoning and counterfactual optimization on the geometric topology graph in combination with the differentiable constraint function, so as to obtain the modification operation vector that can make the drawing to be reviewed meet the engineering specification requirements and the violation heat map reflecting the distribution of the degree of violation of the drawing to be reviewed. The semantic verification and compliance interpretation module is used to input the modification operation vector and the violation heatmap into a finely tuned multimodal large language model for semantic verification and compliance interpretation, and output a structured review report containing violation details, regulatory basis, executable modification instructions and visualization results.

9. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the intelligent map review method based on geometric topology and counterfactual reasoning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the intelligent drawing review method based on geometric topology and counterfactual reasoning as described in any one of claims 1 to 7.

Images