Smart / smart computer-aided design (CAD) block

Abstract

Embodiments of the present invention provide smart / wisdom blocks that know what they are, have context-aware capabilities, understand their surroundings, and know what they are similar and what they are connected and associated. More specifically, the functionality provided by the wisdom blocks includes similar block suggestion, object detection, block conversion, and wisdom block placement / replacement.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit under 35 U.S.C. Section 119(e) of the following co-pending and commonly assigned U.S. provisional patent applications, which are incorporated herein by reference:

[0003] Provisional application serial No. 63 / 567,764, filed on March 20, 2024, titled “Intelligent Geometry,” is filed under the name “Intelligent Geometry” and is assigned attorney docket number 30566.0620USP1. Inventors are Anand Rajagopal, Dan Whitcombe, Yingshen Yu, Danfeng Chen, Ping Zou, Yu Chen, Bo Li, Marina Petzel, Xin Xu, Yufeng Ding, Jian An Wei, Xiaofen Lan, and Britta Ritter-Armour. Background Art 1. Technical field.

[0004] The present invention relates generally to computer-aided design (CAD) applications, and in particular, to a method, apparatus, system, and article of manufacture for intelligent geometry (also known as smart blocks) consisting of CAD features that facilitate the creation, search, placement, replacement, and cleanup of CAD design components.

[0005] 2. Relevant technical description.

[0006] When utilizing CAD applications, users typically create "blocks" from scratch and / or utilize pre-made blocks. As used herein, a "block" is a collection of objects grouped into a single named object. In this regard, a block is a group of geometry and metadata that typically represents a real-world object or a symbolic annotation. Blocks are typically stored in CAD files (e.g., "dwg" files) and are used to create repetitive content in drawings. However, the workflow for creating, using, and managing blocks can be very tedious. Furthermore, such workflows are time-consuming, inefficient, and fail to provide users with assistance / help in making the right decisions and / or assist them in fully utilizing the CAD tools (i.e., they fail to increase the user's proficiency in the design process). Additionally, the workflows of the prior art fail to verify and validate blocks.

[0007] Prior art systems provide various workflows to address different issues related to the creation, use, and management of blocks. More specifically, there are issues related to detecting objects / blocks, finding objects / blocks, and placing objects / blocks. Each of these different issues will be described in further detail below.

[0008] Object Detection

[0009] A common problem with existing systems is identifying exploded geometry / content within a drawing that is not composed of blocks and therefore not considered a block. In other words, existing systems do not provide the ability to effectively / easily identify a set of components within a drawing that should be considered a block. Furthermore, some existing applications (e.g., the BRICSCAD BLOCKIFY application) appear to only recognize identical underlying blocks (without tolerance for minor differences)—for which they do not utilize ML as a technical solution.

[0010] Find / Search Objects

[0011] To place or replace an object or block in a drawing, users must first find or identify the desired or suitable replacement block. Prior art systems require users to create or design new blocks from scratch, search for existing blocks in previous files or drawings, and / or search large libraries of existing blocks. These libraries are often large and often poorly organized. Consequently, starting with an existing block in a drawing and finding a suitable replacement block in the existing block library can unfortunately be time-consuming.

[0012] Placing Objects / Blocks

[0013] Once a suitable block has been identified, the question arises of how to actually insert the block into the drawing. For example, how should the block be oriented / rotated and / or scaled? Additionally, the question arises of where exactly the block should be placed in the drawing (e.g., should the block be aligned to a certain grid position, should it be placed relative to other blocks / objects, should it be offset from existing drawing features, etc.). Prior art systems lack the ability to provide appropriate and effective assistance / suggestions for scaling, positioning, and placing blocks / objects in a drawing.

[0014] In addition to the above, once multiple blocks are placed in a drawing, it is desirable to be able to easily iterate changes made to all instances of the block. Furthermore, in larger projects, managers may want to ensure that all users approve blocks using the same standards. Existing technology systems do not provide this capability.

[0015] Given the above, it's understandable that prior art Building Information Modeling (BIM) applications know what objects are, and to do so, they can automate processes based on their knowledge of the objects (e.g., cutting an opening in a wall when a door is installed in it, or automatically generating labels and annotations). However, to understand what a model object is, a BIM program requires pre-typed or defined geometry before creation—objects in BIM are "hypertyped," meaning everything is done and predetermined ahead of time. In BIM, geometry representing a wall always represents that wall, whereas in CAD, a line can be part of a wall, a chair, or a door. This introduces rigidity when using a BIM program and requires planning and setup before you begin creating a BIM model.

[0016] In view of the above, there is a need for a CAD application that provides intelligent geometry that helps users create, manage and use blocks. Summary of the Invention

[0017] Embodiments of the present invention provide "smart geometry" (also referred to herein as intelligent geometry or smart blocks) with meaningful geometry / blocks. In other words, smart blocks actually understand the geometry the user intends. This allows CAD applications to know how to handle geometry—how to update it, how to clean it, validate it, link it between views and drawings. By leveraging machine learning to recognize and understand geometry, embodiments of the present invention can handle redundant, monotonous drawing tasks, allowing CAD application users to return to doing what they love and do with the greatest value—unleash their creativity and innovation.

[0018] To provide such capabilities, embodiments of the present invention can employ a unique approach to identifying and understanding geometry after it has been created. This allows CAD applications to maintain the fast startup and flexibility that users appreciate, while adding the BIM benefit of understanding and assigning meaning to the geometry in drawings. Thus, by using data, geometry, and machine learning to retroactively understand what someone has drawn, embodiments of the present invention can achieve: anticipating user needs, next steps, verification / validation capabilities, aiding decision making, and more.

[0019] In addition to the foregoing and to provide the benefits of the present invention, an embodiment provides smart blocks with at least four capabilities: similar block suggestion, object detection, block conversion, and block placement / replacement. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Referring now to the drawings, wherein like reference numerals represent corresponding parts, there are:

[0021] Figure 1Ashows a query based on shape similarity according to one or more embodiments of the present invention, and Figure 1B A query based on name similarity is shown;

[0022] Figure 2 shows a data extraction process according to one or more embodiments of the present invention;

[0023] Figure 3 Model-assisted marking according to one or more embodiments of the present invention is shown;

[0024] Figure 4 shows a workflow for finding similarly shaped blocks according to one or more embodiments of the present invention;

[0025] Figure 5 illustrates contrastive learning used by an encoder in accordance with one or more embodiments of the present invention;

[0026] Figure 6 shows a suggestion engine process according to one or more embodiments of the present invention;

[0027] Figure 7 shows further details of a block similarity architecture according to one or more embodiments of the present invention;

[0028] Figure 8 shows a logic flow for providing similar computer-aided design (CAD) blocks according to one or more embodiments of the present invention;

[0029] Figure 9 A vector image showing an overlay graph with nodes and edges according to one or more embodiments of the present invention;

[0030] Figure 10 shows the use of a sparse transformer model as an encoder according to one or more embodiments of the present invention;

[0031] Figure 11 shows the logic flow for enriching tags from a block according to one or more embodiments of the present invention;

[0032] Figure 12 shows an exemplary graphical user interface in which object detection is in progress and after object detection is complete according to one or more embodiments of the present invention;

[0033] Figure 13 An exemplary graphical user interface illustrating a view / inspection mode according to one or more embodiments of the present invention;

[0034] Figure 14shows a logic flow for detecting a computer-aided design (CAD) object according to one or more embodiments of the present invention;

[0035] Figure 15 shows a workflow for converting detected results / instances into blocks according to one or more embodiments of the present invention;

[0036] Figure 16 shows ungrouping of detection results according to one or more embodiments of the present invention;

[0037] Figure 17 shows the results of a shape-based automatic grouping algorithm according to one or more embodiments of the present invention;

[0038] Figure 18 In-situ conversion according to one or more embodiments of the present invention is shown;

[0039] Figure 19 Calculation of alignment coordinates using a master skeleton according to an embodiment of the present invention is shown;

[0040] Figure 20 An exemplary graphical user interface is shown in which a user has selected geometry using an options dialog box to convert selected objects to blocks according to one or more embodiments of the present invention;

[0041] Figure 21 An exemplary graphical user interface is shown in which a user is provided with the option of accepting a placement or moving / rotating / scaling an object placement according to one or more embodiments of the present invention;

[0042] Figure 22 An exemplary conversion dialog box is shown according to one or more embodiments of the present invention, the dialog box providing options for defining a new block by specifying a block name and an insertion point;

[0043] Figure 23 shows a logic flow for converting a geometric object into a block according to one or more embodiments of the present invention;

[0044] Figure 24 shows an exemplary graphical user interface for performing block replacement according to one or more embodiments of the present invention;

[0045] Figure 25 illustrates a logic flow for placing / replacing blocks in a computer-aided design (CAD) application according to one or more embodiments of the present invention;

[0046] Figure 26 is an exemplary hardware and software environment for implementing one or more embodiments of the present invention; and

[0047] Figure 27 A typical distributed / cloud-based computer system according to one or more embodiments of the present invention is schematically illustrated. DETAILED DESCRIPTION

[0048] In the following description, reference is made to the accompanying drawings which form a part hereof and in which are shown by way of illustration several embodiments of the present invention. It should be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.

[0049] Overview

[0050] Embodiments of the present invention provide "Smart Blocks," which consist of a series of AI / ML (artificial intelligence / machine learning) based features that help users create, find, place, replace, and clean up their CAD design components - blocks.

[0051] Previously, existing technology hadn't addressed this problem from a data / AI-driven perspective. Some existing technology applications sought to match exact geometry. However, embodiments of the present invention extend this even further, determining similarity based on the semantic meaning of blocks within their context. Furthermore, embodiments of the present invention build on cutting-edge industry research in computer vision for raster images and neural networks for geometry. Furthermore, embodiments of the present invention extend this previous work to CAD data.

[0052] More specifically, embodiments of the present invention provide intelligent geometry / geometric intelligence that makes CAD geometry smart, connected, and context-aware, thereby automating time-consuming and monotonous processes, guiding user workflows, enhancing decision-making assistance, and unlocking complete design capabilities.

[0053] Wisdom Block

[0054] As mentioned above, blocks are a standard CAD concept. They are groups of geometry and metadata, typically representing real-world objects or symbolic annotations.

[0055] Embodiments of the present invention provide intelligent / smart blocks that know what they are, are context-aware, understand their surroundings, and know what they are similar to and what they are connected and associated with.

[0056] Smart blocks provide the ability to guide users to help them make the right decisions, assist them in getting the most out of CAD tools, and alleviate the tedious workflow associated with creating, using, and managing blocks. To provide such capabilities, embodiments of the present invention utilize ML to recognize objects, their context, and more.

[0057] There's a question about the purpose of smart blocks. In this context, smart blocks save time, optimize and automate monotonous, time-consuming, and tedious processes / workflows and subsequent steps (e.g., background updates, automatically generated schedules, etc.), aid decision-making, and perform verification and validation (applications, objects, blocks). Furthermore, smart blocks focus solely on efficiency. They also increase user proficiency in CAD applications and CAD application integrity tools. Intelligent / smart geometry guides users and helps them make the best decisions, maximizing and optimizing their design potential.

[0058] As mentioned above, smart blocks provide various capabilities, including similar block suggestions, object detection, block conversion, and smart placement. Each of these capabilities will be described in further detail below.

[0059] Similar block suggestions

[0060] To facilitate the creation and replacement of blocks in drawings, embodiments of the present invention generate similar block suggestions to help users find the blocks they are looking for when replacing and creating blocks. In this regard, the meaning / intent of a block can be inferred by understanding which other blocks it is similar to. As part of the process of determining similar block suggestions, embodiments of the present invention may also provide the ability to group, categorize, and classify similar blocks. To provide such capabilities, embodiments of the present invention utilize ML to determine similar blocks based on shape, visual geometry, and naming (metadata). Details of such capabilities are described below.

[0061] Block Replacement uses shape and name similarity ML models. The shape similarity model receives a raster-based block image. The name similarity model receives metadata information associated with the block (block name, drawing name, file path).

[0062] The ML model outputs are embeddings from the shape and name similarity models, which are later combined using the suggestion engine. The return / output of the suggestion engine identifies the most similar blocks for a given query block.

[0063] The training data for the shape similarity model consists of labeled images, each with a class label (e.g., sink, single door, etc.). Most of the training data represents blocks from the architecture and engineering industries. The model learns to use these labels to group similar blocks closer together in feature space while separating dissimilar blocks to guide the learning process. The data used for the name similarity model represents block metadata information extracted from drawing (e.g., DWG) files.

[0064] Block similarity

[0065] Block similarity may include / be based on both name similarity and shape similarity. Figure 1A shows a query based on shape similarity according to one or more embodiments of the present invention, and Figure 1B 1 shows a query based on name similarity. More specifically, the query image 102 is Figure 1A and Figure 1B is the same in , and includes both the image 102A and the name 102B (ie, "OFFICE BEL 22"). Figure 1A In , the similarity is based on shape 102A, and thus different similar images 104A through 104E are retrieved. Figure 1B , the text 102B "OFFICE BEL 22" is compared with text (e.g., in metadata of other objects / blocks) to find similar names (e.g., OFFICE BEL 10 104F, OFFICE BEL 4 104G, OFFICE BEL 11 104H, OFFICE TBL 2 104I, and BEL 22 CHAIR 104J).

[0066] In an embodiment of the present invention, ML solutions differ based on the query type. For example, a shape similarity ML model may be used to determine Figure 1A Similar shapes. In addition, a block name similarity ML model can be used to determine name-based similarity models, such as Figure 1B Additionally, the suggestion engine can combine both shapes and names (i.e., utilize a shape similarity ML model and a block name similarity model) to determine the most similar shapes / blocks.

[0067] Data Collection

[0068] Collecting data from the drawing file is the first step to find similar blocks. The first step in data collection is to extract the data. Figure 2 The data extraction process according to one or more embodiments of the present invention is shown. As shown, an extractor 202 extracts key information from an original drawing (e.g., DWG) file 204, including user-defined blocks and all geometric shapes (e.g., the entire drawing). The extractor 202 then outputs the extracted data into various data formats 206 (e.g., PNG file 206A, SVG file 206B, JSON file 206C, etc.).

[0069] Once extracted, embodiments of the present invention perform model-assisted labeling. Figure 3Model-assisted labeling according to one or more embodiments of the present invention is shown. In a first step, during unsupervised pre-training 300, a convolutional neural network (CNN) backbone 304 is pre-trained on an entire dataset 302 (e.g., a dataset of existing files with known / unknown blocks / geometry). The CNN process can utilize a decoder / projector 306 to deconstruct / reconstruct a compressed image (i.e., from an encoded image) into its original form. In this regard, the decoder / projector 306 can be used to perform image-to-image regression tasks and / or learn how to map similar blocks (i.e., between images).

[0070] Once trained using unsupervised pre-training, embodiments of the present invention may perform a semi-supervised learning process 308. During semi-supervised learning, a CNN classifier 310 (built on top of the CNN backbone 304) is trained on a small labeled dataset 312 and a large unlabeled dataset 314.

[0071] The CNN classifier 310 generates / provides proxy labels 316 to 318 for the unlabeled samples.

[0072] Finally, domain expert 320 reviews proxy tags 316 - 318 and then includes / adds reviewed proxy tags 322 to labeled dataset 312 .

[0073] Similar Block Workflow

[0074] Figure 4 The following is a workflow for finding similarly shaped blocks according to one or more embodiments of the present invention: In the first step, a block query 402 is received. The block query 402 consists of an image 402A and metadata 402B that the user is trying to find.

[0075] Feature extraction 404 is then performed on the block query 402. More specifically, features in the block query 404 are extracted using an encoder 404A.

[0076] The next step is similarity embedding 406. During step 406, similar blocks are retrieved (from the block library) (ie, similar block retrieval 406A).

[0077] At step 408, the top k most correlated blocks are identified / selected. In other words, once similar blocks are retrieved at 406, similar blocks that meet / exceed a certain threshold (ie, the top k most correlated blocks) are selected.

[0078] Figure 51 shows contrastive learning used by encoder 404A according to one or more embodiments of the present invention. As shown, block 1 502A, block 2 502B, and block 3 502C are processed by encoder 404A to generate different representations (i.e., representation 1 504A, representation 2 504B, and representation 3 504C). The different representations 504A to 504C can be ranked / sorted by similarity / dissimilarity, as indicated at 506A and 506B.

[0079] Figure 6 4. Suggestion engine processing according to one or more embodiments of the present invention is shown. More specifically, a query block 402 consists of an image 402A and a block name / metadata 402B. Each query block 402A / 402B is processed by a different ML model (e.g., a shape similarity model 602A is used to process the image 402A, while a name similarity model 602B is used to process the block name / metadata 402B). The output of the different models 602A / 602B provides the top five (5) similar blocks based on shape 604A and name 604B. Then, embodiments of the present invention combine the two to obtain an output 606 of the most similar blocks based on both shape and name.

[0080] Figure 7 , further details of a block similarity architecture according to one or more embodiments of the present invention are shown. As shown, a query block 402 is received and decomposed into a shape embedding 702A and a name embedding 702B (i.e., both the shape / image 702A and the name / metadata 702B are embedded / encoded). A block library 704 (consisting of a plurality of blocks) is processed by an indexer 706 to generate an index 708 (e.g., consisting of the shape embedding and the name embedding for each block in the block library 704).

[0081] The embedded shape 702A, name 702B, and index 708 are input to a (query adaptive algorithm) searcher 710. The searcher 710 then identifies the top k blocks 606 (e.g., blocks 2, 5, 9, ... k) that are similar (in both shape and name) to the query block 402.

[0082] In light of the foregoing, embodiments of the present invention can use machine learning to search the block library 704 and identify the top k blocks 606 that are similar to the query block 402 in both shape and name.

[0083] Logical Flow

[0084] Figure 8 A logic flow for providing similar computer-aided design (CAD) blocks is shown according to one or more embodiments of the present invention.

[0085] At step 802, a shape similarity machine learning (ML) model is trained based on labeled images. The labeled images consist of / include blocks from the architecture and engineering industries. Each labeled image is a class label. The shape similarity model uses the class labels to group blocks based on shape to guide learning.

[0086] At step 804 , the name similarity ML model is / is composed of a transformer-based model (Sentence-BERT) that is fine-tuned based on block metadata information extracted from one or more drawing files, wherein the name similarity ML model groups blocks together based on their names.

[0087] In one or more embodiments, the shape similarity model is generated using a convolutional neural network (CNN), and the name similarity model is generated using a transformer model.

[0088] At step 806, a block query for a block is received. The block query includes: (i) a raster-based block image; and (ii) metadata information associated with the block. In one or more embodiments, the metadata includes a block name, a drawing name, and a file path.

[0089] At step 808, the shape similarity model processes the raster-based patch image and the label to output a shape embedding. Additionally, at step 808, the name similarity model processes metadata information associated with the patch to output a name embedding. In one or more embodiments, the shape similarity model processes the raster-based patch image using an encoder that extracts features from the labeled image.

[0090] At step 810, the suggestion engine combines the shape embedding and the name embedding to identify similar blocks. In one or more embodiments, the suggestion engine: ranks the identified similar blocks based on similarity; selects a limited number of most relevant identified similar blocks based on the ranking; and provides the limited number of most relevant identified similar blocks in response to the block query.

[0091] At step 812, identified similar blocks are provided in response to the block query.

[0092] Object Detection

[0093] An additional feature of embodiments of the present invention is object detection. This feature uses machine learning to scan drawings for objects that can be converted to blocks. Specifically, groups of geometric shapes are identified as objects in CAD drawings (e.g., a rectangle with an arc along a horizontal line is a door). The purpose of this object detection is to convert the identified objects into blocks. Once converted into blocks, users can realize the efficiency and best practice benefits of using blocks. Embodiments of the present invention utilize ML models to perform this object detection.

[0094] First, a CAD drawing image is obtained and then converted into a graph. This graph contains geometric markers as nodes, which are a set of features extracted from the geometric graph, and for each geometric marker, edges are constructed between the marker and its K nearest neighbor markers determined by distance. This graph is then input into the ML model. Therefore, the input data (to the ML model) is a graph of geometric markers. The output from the ML model consists of a node mask for each predicted object. Each mask consists of a list of indices of the nodes / geometric markers that form the object in the CAD drawing.

[0095] Data Collection

[0096] Similar to the workflow used for similar block suggestions, Figure 2 A workflow for data collection for object detection according to one or more embodiments of the present invention is shown. In one or more embodiments, different file formats can be used for different purposes. For example, PNG file 206A can be used for a pixel-based solution, SVG file 206B can be used for a vector-based solution, and JSON file 206C can be used for metadata.

[0097] Figure 9 A vector image of an overlay graph with nodes and edges according to one or more embodiments of the present invention is shown. As shown, the vector image 900 includes 2D primitive geometric figures 902A to 902D that have been converted into nodes 904 (representing geometric markers) and edges 906 that link to the K nearest neighboring data points / nodes 904. In this regard, the data transformation of the vector image 900 includes converting the 2D primitive geometric figures 902A to 902D (e.g., in an SVG file) into geometric markers / nodes 904.

[0098] Once a graph with nodes and edges has been received, embodiments of the present invention encode the data represented by the graph. More specifically, a sparse transformer model is used as an encoder that incorporates an attention mechanism between nodes in the graph. During attention computation, each node updates its embedding by aggregating the embeddings of its neighbors in the graph. Figure 10 The use of a sparse transformer model as an encoder according to one or more embodiments of the present invention is shown. As shown, adjacency relations 1002, geometric tag embeddings 1004, and relative position embeddings 1006 are encoded into block / object representations 1010 and geometric figure representations 1012 via a series of Nx attention blocks 1008 for use in a variety of tasks.

[0099] In addition to the above, the labels from the blocks as part of object detection can be more enriched. Figure 111 shows a logical flow of enriching tags from a block according to one or more embodiments of the present invention. As shown in the figure, a sample drawing (e.g., from AUTOCAD-DM TM -CAD design and document management system). In step 1104, blocks are extracted from the drawing. In step 1106, the extracted blocks are replaced with candidate blocks of the same category. In step 1108, the newly replaced blocks are merged to create a newly marked drawing.

[0100] user interface

[0101] When the object detection workflow starts, the drawing content is sent to the ML service for recognition. The detection results can then be displayed in a window / panel. Figure 12 An exemplary graphical user interface is shown where object detection is in progress at 1202 and upon completion at 1204 , 8 groups of objects and 34 instances have been found.

[0102] After detection, the user may then choose to examine the identified instance (eg, by clicking on the "Review Object" icon / button 1206). Figure 13 An exemplary graphical user interface for a review / inspection mode according to one or more embodiments of the present invention is shown. During review / inspection mode, a toolbar 1302 may be displayed that provides various options for navigating different collections and managing detected objects. A "collection" consists of a group of instances that collectively represent the same real-world object (e.g., a door). In each collection, a master instance 1304 (which may be displayed in a visually distinguishable manner, such as with a blue border) defines the object and direction when converting detected instances into new blocks. A user may encounter incorrect collections or instances during a detection review. In addition, if similar objects are not detected and included in a collection, the user can report such errors (i.e., back to the ML service via a feedback loop), thereby helping to improve the overall performance and accuracy of the ML service.

[0103] Logical Flow

[0104] Figure 14 A logic flow for detecting a computer-aided design (CAD) object is shown.

[0105] At step 1402, a machine learning (ML) model is trained to process graphics representing CAD objects. Training is based on ground truth labels and ground truth CAD objects. In one or more embodiments, the ML model uses a sparse transformer model as an encoder. This sparse transformer model encodes geometric labels and position embeddings into object / block representations and geometric representations.

[0106] At step 1404, a CAD drawing is obtained and is composed of / includes two-dimensional (2D) geometric primitives.

[0107] At step 1406, a CAD drawing is represented as an input graph. 2D geometric primitives are represented by geometric tags in the nodes of the input graph. Spatial KNN (k-nearest neighbor) adjacency relationships between the 2D geometric primitives are represented as edges between the nodes of the input graph.

[0108] The input graph is processed in / by the ML model at step 1408. After processing, the ML model outputs nodes representing predicted objects of the 2D geometric primitive geometry.

[0109] At step 1410, the predicted object is used to represent the 2D geometric primitive geometry.

[0110] At step 1412, the predicted objects are merged to create a new labeled drawing.

[0111] In addition to the above, the ML model may be updated via a feedback loop based on confirmation of the predicted blocks by user input.

[0112] Block Conversion

[0113] Block conversion is the process of easily converting a group of geometric entities (e.g., detected objects) into instances of a block definition. This block conversion is used to clean up drawings, avoid duplicate block definitions, facilitate easy and quick block creation, and more. More specifically, one or more embodiments of the present invention allow users to convert multiple similar groups of geometries / objects into instances of new or existing block definitions. The process includes suggestions for existing block definitions that are similar to the unblocked geometry, making it easy and convenient for users to reuse their existing blocks. This can improve efficiency, save users time, and reinforce good block hygiene and best practices.

[0114] Figure 15 A workflow for converting detected results / instances into blocks according to one or more embodiments of the present invention is shown. At step 1502, detection results are obtained (e.g., as described above with respect to object detection). At step 1504, geometry-based block definition grouping is performed (i.e., the detected geometries / objects are grouped into potential blocks). At step 1506, in-place conversion is performed on the groupings. A more detailed breakdown / description of this workflow is described below.

[0115] The detection result of the ML model is a single object described by a list of geometry indices (further accurate shape grouping / rotation estimation is required to determine the block definition). For example, the chairs in the result set 1602 may look different in the result set and are not grouped based on shape similarity. Therefore, embodiments of the present invention can classify the detected geometry / object results by shape. More specifically, embodiments of the present invention can perform shape-based automatic grouping (i.e., using a fault-tolerant shape-based automatic grouping algorithm) to post-process the ML results (from object detection). Figure 17 As shown in , results with different shapes in a result set 1602 are classified based on shape similarity 1702 .

[0116] The algorithm can be designed to effectively solve the challenge of optimal classification based on shape similarity, especially when geometric data is limited. The key ideas of the shape-based automatic grouping algorithm include: main feature extraction; rich geometric feature data; and optimal group number determination.

[0117] One approach to shape-based automatic grouping algorithms is to prepare geometric feature data. This preparation may include increasing the number of sampled vertices on the geometry to obtain richer information; using PCA (Principal Component Analysis) to calculate eigenvalues ​​using all vertices (similar shapes have similar eigenvalues); and including the length of the geometry and the ratio of the perimeter to the circumference of its smallest enclosing circle.

[0118] Another feature of the shape-based automatic grouping algorithm is that it classifies (geometric figures / objects) based on geometric feature data. This classification can: use k-means clustering to classify feature data from entities; determine the optimal number of clusters by evaluating results with increasing k values; use the Silhouette algorithm to evaluate the quality of the classification; use a compactness-based judgment condition to resolve premature iteration exits due to local optimal solutions; and use an enhanced Bikmeans algorithm to enhance stability, thereby eliminating the uncertainty of random sampling of initial cluster centers.

[0119] Figure 17 16. The result of an automatic shape-based grouping algorithm according to one or more embodiments of the present invention is shown. As shown, result set 1602 is processed by the algorithm to generate geometry-based groupings (i.e., objects are grouped based on their geometry / shape).

[0120] Due to factors such as insufficient training data and complex recognition scenarios, the quality of recognition results returned by machine learning models is sometimes suboptimal. This primarily involves two issues: first, some parts of an object that should be recognized are missed, while other parts that should not be recognized are mistakenly identified as part of the object; and second, the recall problem, where some objects that should be recognized are not recognized by the model. This can result in minimal variance within the same group after the ML results are grouped by shape, with missing objects. Therefore, the recognition results need to be corrected during post-processing. In one embodiment of the present invention, a voting strategy is employed to correct for discrepancies between objects within the same group. Specifically, if a recognized object is missing parts compared to other objects in the group, and these parts are present in the current drawing, the missing parts are added to the results. Conversely, if a recognized object contains parts that are not present in other objects in the group, these irrelevant parts are removed from the results. After correcting for objects within a group, all objects have identical shapes. A geometric search algorithm is then used to identify objects in the drawing that have the same shape but were not recognized by the ML model, and these objects are added to the group.

[0121] user interface

[0122] As described above, a feature of smart blocks includes the ability to easily convert multiple instances of a specified geometric figure into blocks. In this regard, when a user specifies a geometric figure to be converted, embodiments of the present invention can identify (e.g., via object detection as described above) and highlight all instances of the same geometric figure. Thereafter, the user can select to convert the source object or instance into a block.

[0123] Figure 20 An exemplary graphical user interface is shown in which a user has selected a geometric figure 2002, all other instances of the geometric figure are highlighted, and the user is provided with an option 2004 to convert the selected object to a block.

[0124] When converting to an existing block, several options are possible: the user can pick a block from the drawing, or from a list of recently used blocks, blocks in the current drawing, or blocks already in the library, or directly from the drawing canvas, selecting a recently used block, or a similar block already in the library suggested by the ML algorithm (as described above). After selecting an existing block definition, the alignment of the block can be adjusted relative to the source geometry. Figure 21 An exemplary graphical user interface is shown in which the user is provided with the option 2102 to accept the placement or to move / rotate / scale the object placement.

[0125] Alternatively, embodiments of the present invention may convert the source object or selected instances into new blocks. Figure 22An exemplary conversion dialog box is shown according to one or more embodiments of the present invention, which provides options for defining a new block by specifying a block name 2202 and an insertion point 2204. By default, the insertion point may be set to the center of the specified geometry 2206. The ability to quickly convert repeated geometry to blocks minimizes redundancy during the design process and provides a wider range of options for organizing drawings.

[0126] Logical Flow

[0127] Figure 23 A logic flow for converting geometric objects to blocks is shown according to one or more embodiments of the present invention.

[0128] In step 2302, geometric feature data is prepared based on the predicted object. This step may include: increasing the number of sampled vertices on the geometric figure; using principal component analysis (PCA) calculation to calculate eigenvalues ​​using all sampled vertices; and including the length and perimeter of the 2D geometric primitive geometric figure to the ratio of the circumference of the minimum enclosing circle.

[0129] In step 2304, the predicted objects are grouped based on their geometric shapes. The classification / grouping of the predicted objects can utilize k-means to classify the geometric feature data from the 2D geometric primitives into objects. In addition, the classification can determine the optimal number of classifications by evaluating the results of the classification within increasing k values. In addition, the classification can use the silhouette algorithm to evaluate the quality of the classification. In addition, the classification can use a judgment condition based on compactness to resolve premature iteration exit. Moreover, the classification process can use an enhanced Bikmeans algorithm to enhance stability, which eliminates the uncertainty of random sampling of the initial classification center.

[0130] Additionally, step 2304 may perform grouping by selecting a category / group from an existing category library, creating a new category / group, selecting a category / group from the current drawing (eg, the user accesses a previous category), and the like.

[0131] At step 2306, the classified / grouped predicted objects are converted into blocks.

[0132] Steps 2302 to 2306 can also utilize fuzzy logic to mitigate positional deviations caused by the absence or presence of geometric figures in the geometric feature data. This fuzzy logic processing can: sort the predicted objects by length; scale the predicted objects to the target; compare the geometric figure types and lengths between the targets and scale the comparison results in order; and select the most matching geometric figure based on the order to participate in the alignment coordinate calculation.

[0133] Smart Placement

[0134] Smart Object Placement provides automatic orientation of blocks using a rule-based algorithm. This automatic orientation saves the time that would otherwise be required to correctly rotate and orient blocks for each instance.

[0135] In addition to the above, embodiments of the present invention may utilize a tile placement engine, which is a geometry-based tile placement recommendation system. The engine learns the scene (of existing instances of placed tiles) to infer possible contexts, and matches the learned contexts against the current scene to obtain possible placement suggestions. The engine may consist of two subsystems:

[0136] Background Extractor: Obtains a patch background from an existing instance scene.

[0137] Background Matcher: Finds possible block backgrounds in the current scene and provides placement suggestions.

[0138] Background Extractor

[0139] Background is a set of objects closely associated with a given block. It consists of meaningful geometric data with fixed relative positions and orientations relative to the associated block. Roughly speaking, embodiments of the present invention can obtain possible block backgrounds through the following steps: sampling, analysis, and determining background patterns.

[0140] sampling

[0141] The geometric data of objects surrounding all existing instances of the block are sampled. The collected geometric data is coded and recorded for subsequent context analysis and extraction. Embodiments of the present invention can use a set of concentric circles constructed on top of the local coordinates of the block instance to sample the surrounding geometric data.

[0142] analyze

[0143] Since the backgrounds of different block instances are different, all sample data need to be further analyzed. The embodiment of the present invention merges the sample data with the same distribution in the sample points and removes the possible noise data (noise data is the sample points that only appear in a specific sample).

[0144] Determine background mode

[0145] The set of sample points with the same position in the sample data, along with the geometric objects containing these sample points, form a "background pattern." Multiple instances of the same block may exist in the current drawing, allowing for multiple "background patterns" to be extracted after sampling and analysis. By merging identical "background patterns," the resulting "background pattern" is unique.

[0146] Background Matcher

[0147] Placement involves searching for a background pattern around the cursor location that has a block instance to be placed. There are two methods for finding a possible matching background pattern. The block instance is then placed based on the relationship between the block instance and the background pattern.

[0148] Geometry-based matching

[0149] The geometry-based matching method uses the current mouse position as the center of the circle and the radius of the circle surrounding the background pattern as the detection range. It searches for a combination of entities around the mouse that can fully or partially match the geometry in the background pattern. When a possible background pattern is found, the current background geometry data is sampled using the same sampling method used to generate the background pattern. The sampled geometry data is then compared with the sampling points in the background pattern to determine whether the background pattern can be applied to the current position. Once established, the coordinate system of the current background is calculated through the sampling points, and the rotation angle and position offset are calculated based on the difference between the coordinate system and the coordinate system of the background pattern.

[0150] Location-based matching

[0151] When the geometry in the background pattern is too simple (for example, only a line or an arc), it may not be possible to determine whether the background pattern is applicable to the current background by looking for matching geometry. Instead, the embodiment of the present invention reversely calculates the center of the placed block based on the position of the sampling point on the geometric object closest to the current mouse position. This position is then used as the center of the circle, and the current position is sampled using the same sampling method as when generating the background pattern. Thereafter, whether the "background pattern" is applicable to the current background is determined by comparing whether the sampling points match.

[0152] user interface

[0153] As mentioned above, the smart block functionality provides placement suggestions based on where the user has previously placed the block in a drawing, and can also replace a specified block reference by selecting from a panel of suggested similar blocks.

[0154] Block placement

[0155] Automatic block placement is now available for blocks inserted using the Blocks palette, helping users place blocks more easily and saving time. The block placement engine learns how existing block instances are placed in a drawing to infer how the same block should be placed next. When a user inserts a block from the Blocks palette into a drawing, the engine provides placement suggestions close to similar geometry where the user previously placed the block.

[0156] For example, if a user has already placed a chair block near the corner of a wall, when inserting another instance of the same chair block, embodiments of the present invention automatically position the chair as the user moves the chair closer to the similar corner. As the user moves the block, the wall may be highlighted, and the position, rotation, and scale of the chair block may be adjusted to match the other block instances. The user can click to accept the suggestion, press CTRL to switch to a different suggestion, or move the cursor away to ignore the current suggestion. To temporarily turn off suggestions when placing a block, the user can hold down the Shift+W or Shift+[ keys while inserting or moving the block.

[0157] Block Replacement

[0158] When a user selects a block reference to replace, embodiments of the present invention can suggest similar blocks for the user to choose from. The user can also replace the selected block with another block that the user specifies from a drawing or a list of recently inserted or created blocks (e.g., by selecting from a panel of suggested similar blocks). The replaced block reference can retain the scale, rotation, and property values ​​of the original block. Figure 24 An exemplary graphical user interface for performing block replacement according to one or more embodiments of the present invention is shown. As shown, a user clicks on an icon 2402 next to a block reference of Chair7, and a dialog box 2404 is displayed, allowing the user to select a block at 2406, along with a list of suggested blocks 2408 and recently used blocks 2410. In other words, when a user has selected a block reference to replace, embodiments of the present invention suggest similar blocks for the user to select.

[0159] In addition to the above, the first time a user completes a block replacement, a consent dialog may be displayed requesting access to the block data. The content data used during block replacement can then be shared with the ML model in a feedback loop to improve block suggestions and other smart block functionality. Therefore, block replacement uses an ML algorithm to generate suggestions based on block names and visual similarities to existing blocks (see the description above for similar block suggestions). ML can be heavily data-dependent, so insufficient block data may result in inaccurate suggestions. Therefore, granting access to block data can help improve the accuracy of block suggestions. In addition, it may be important to note that block suggestions can be based on a set of block definitions stored in the drawing file. Additionally, adding more block libraries can provide more meaningful and relevant suggestions.

[0160] Logical Flow

[0161] Figure 25 The logic flow for placing / replacing blocks in a computer-aided design (CAD) application is shown according to one or more embodiments of the present invention.

[0162] In step 2502, blocks are identified to be placed in a computer-aided design (CAD) drawing. A block is a set of geometric objects.

[0163] In step 2504, the content extractor obtains a block context from an existing instance of the block in the CAD drawing. The block context includes: an object set associated with the block; geometric data of the object set, which has a fixed relative position and orientation with respect to the block; and a background pattern of the geometric data of the object set.

[0164] The background extractor can obtain the block background by sampling, analyzing, and determining background patterns. More specifically, it samples the object geometry data of all existing instances of the block. The sampling process encodes and records the sampled geometry data for subsequent background analysis and extraction. The analysis examines the sampled geometry data by merging sampled geometry data with identical distributions of sample points. The background pattern is determined as a set of sample points with identical positions and the geometry data of the objects that share this set of sample points.

[0165] The background matcher searches for background patterns around the cursor location where the block is to be placed at step 2506. To find background patterns, embodiments of the present invention may use geometry-based matching or position-based matching.

[0166] For geometry-based matching, the cursor position is used as the center of the circle, and the radius of the circle is used as the detection range. The content matcher then searches for entity combinations around the cursor position within the detection range that can match the geometric data in the background pattern. After a match is found, the current background geometry is sampled and compared with the sampled geometry set in the background pattern to determine whether the background pattern can be applied to the cursor position. Once it is determined that the background pattern can be applied to the cursor position, the coordinate system of the current background is calculated, and the rotation angle and position offset are calculated based on the difference between the coordinate system of the current background and the coordinate system of the background pattern.

[0167] For position-based matching, the center of the block is reverse-calculated to be placed starting from the position of the sampling point on the geometry data of the object in the collection closest to the cursor position. This reverse-calculated center is used as the center of the circle. The cursor position is then sampled using the sampling method. Finally, the background mode is determined to be applicable to the current background by comparing the sampled points at the cursor position for a match.

[0168] At step 2508, the block is placed in the CAD drawing based on the relationship between existing instances of the block and the background pattern.

[0169] Hardware environment

[0170] Figure 2626 is an exemplary hardware and software environment 2600 for implementing one or more embodiments of the present invention (referred to as a computer-implemented system and / or computer-implemented method). The hardware and software environment includes a computer 2602 and may include peripheral devices. The computer 2602 may be a user / client computer, a server computer, or a database computer. The computer 2602 includes a hardware processor 2604A and / or a dedicated hardware processor 2604B (hereinafter collectively referred to as processor 2604) and memory 2606, such as random access memory (RAM). The computer 2602 may be coupled to and / or integrally formed with other devices, including input / output (I / O) devices such as a keyboard 2614, a cursor control device 2616 (e.g., a mouse, a pointing device, a pen and tablet, a touch screen, a multi-touch device, etc.), and a printer 2628. In one or more embodiments, the computer 2602 may be coupled to or may include a portable or media viewing / listening device 2632 (e.g., an MP3 player, an iPod, a NOOK, a portable digital video player, a cellular device, a personal digital assistant, etc.). In yet another embodiment, the computer 2602 may include a multi-touch device, a mobile phone, a gaming system, an Internet-enabled television, a television set-top box, or other Internet-enabled device that executes on a variety of platforms and operating systems.

[0171] In one embodiment, computer 2602 operates by hardware processor 2604A executing instructions defined by a computer program 2610 (e.g., a computer-aided design [CAD] application) under the control of an operating system 2608. Computer program 2610 and / or operating system 2608 may be stored in memory 2606 and may interface with users and / or other devices to accept input and commands and provide output and results based on such input and commands and the instructions defined by computer program 2610 and operating system 2608.

[0172] The output / results may be presented on a display 2622 or provided to another device for presentation or further processing or action. In one embodiment, the display 2622 comprises a liquid crystal display (LCD) having a plurality of individually addressable liquid crystals. Alternatively, the display 2622 may comprise a light emitting diode (LED) display having a cluster of red, green, and blue diodes driven together to form full-color pixels. Each liquid crystal or pixel of the display 2622 changes to an opaque or translucent state in response to the processor 2604 applying the instructions of the computer program 2610 and / or the operating system 2608 to the data or information generated by the input and commands to form part of an image on the display. The image may be provided by a graphical user interface (GUI) module 2618. Although the GUI module 2618 is depicted as a separate module, the instructions for performing the GUI functions may reside or be distributed in the operating system 2608, the computer program 2610, or implemented using dedicated memory and a processor.

[0173] In one or more embodiments, the display 2622 is integrally formed with / incorporated into the computer 2602 and comprises a multi-touch device (e.g., a trackbox, a touch screen, a smartwatch, smart glasses, a smartphone, a laptop or non-laptop personal mobile computing device) having a touch-sensing surface with the ability to recognize the presence of two or more points of contact with the surface. Examples of multi-touch devices include mobile devices (e.g., iPhones, Android devices, Windows phones, Google Pixel devices, Nexus S, etc.), tablet computers (e.g., iPads, HP Touchpads, Surface devices, etc.), portable / handheld game / music / video player / console devices (e.g., iPhone Touch, MP3 players, NINTENDO SWITCH, PlayStation Portable, etc.), touch tables, and touch walls (e.g., where an image is projected through acrylic and / or glass and then illuminated with an LED backlight).

[0174] Some or all of the operations performed by computer 2602 according to computer program 2610 instructions may be implemented in a special-purpose processor 2604B. In this embodiment, some or all of the computer program 2610 instructions may be implemented via firmware instructions stored in a read-only memory (ROM), programmable read-only memory (PROM), or flash memory within the special-purpose processor 2604B, or in memory 2606. The special-purpose processor 2604B may also be hardwired by circuit design to perform some or all of the operations to implement the present invention. In addition, the special-purpose processor 2604B may be a hybrid processor that includes dedicated circuits for performing a subset of functions and other circuits for performing more general functions (such as responding to computer program 2610 instructions). In one embodiment, the special-purpose processor 2604B is an application-specific integrated circuit (ASIC).

[0175] The computer 2602 may also implement a compiler 2612 that allows an application or computer program 2610 written in a programming language (such as C, C++, assembly language, SQL, Python, Prolog, MATLAB, RUBY, RAILS, Haskell, or other languages) to be translated into code readable by the processor 2604. Alternatively, the compiler 2612 may be an interpreter that directly executes instructions / source code, converts source code into an intermediate representation that is executed, or executes stored precompiled code. Such source code may be written in a variety of programming languages ​​(such as Java, JavaScript, Perl, BASIC, etc.). Once completed, the application or computer program 2610 uses the relationships and logic generated by the compiler 2612 to access and manipulate data received from the I / O devices and stored in the memory 2606 of the computer 2602.

[0176] Computer 2602 also optionally includes external communication means, such as a modem, satellite link, Ethernet card, or other means for accepting input from other computers 2602 and providing output to the computer.

[0177] In one embodiment, the instructions implementing the operating system 2608, computer program 2610, and compiler 2612 are tangibly embodied in a non-transitory computer-readable medium (e.g., data storage device 2620), which may include one or more fixed or removable data storage devices such as a zip drive, a floppy disk drive 2624, a hard drive, a CD-ROM drive, a tape drive, etc. In addition, the operating system 2608 and computer program 2610 consist of computer program 2610 instructions that, when accessed, read, and executed by the computer 2602, cause the computer 2602 to perform the steps required to implement and / or use the present invention, or load the program of instructions into the memory 2606, thereby creating a specialized data structure, thereby causing the computer 2602 to operate as a specially programmed computer to perform the method steps described herein. The computer program 2610 and / or operating instructions may also be tangibly embodied in the memory 2606 and / or the data communication device 2630, thereby making a computer program product or article of manufacture according to the present invention. For this purpose, the terms "article of manufacture," "program storage device," and "computer program product," as used herein, are intended to encompass a computer program accessible from any computer-readable device or media.

[0178] Of course, those skilled in the art will recognize that any combination of the above components, or any number of different components, peripherals, and other devices, may be used with the computer 2602 .

[0179] Figure 27 A typical distributed / cloud-based computer system 2700 is schematically shown in which a client computer 2702 is connected to a server computer 2706 using a network 2704. A typical combination of resources may include: a network 2704, which may include the Internet, a LAN (local area network), a WAN (wide area network), an SNA (system network architecture) network, etc.; a client 2702, which may be a personal computer or workstation (e.g., Figure 26 and a server 2706, which is a personal computer, workstation, minicomputer, or mainframe computer (such as Figure 26 However, it may be noted that different networks, such as a cellular network (e.g., GSM [Global System for Mobile Communications], etc.), a satellite-based network, or any other type of network, may be used to connect the client 2702 and the server 2706, according to embodiments of the present invention.

[0180] A network 2704 (such as the Internet) connects the client 2702 to the server computer 2706. The network 2704 can utilize Ethernet, coaxial cable, wireless communication, radio frequency (RF), etc. to connect the client 2702 and the server 2706 and provide communication between them. In addition, in a cloud-based computing system, resources (e.g., storage devices, processors, applications, memory, infrastructure, etc.) in the client 2702 and the server computer 2706 can be shared by the client 2702, the server computer 2706, and users across one or more networks. Resources can be shared by multiple users and can be dynamically reallocated based on demand. In this regard, cloud computing can be referred to as a model for supporting access to a shared pool of configurable computing resources.

[0181] Client 2702 can execute a client application or web browser and communicate with server computer 2706 executing a web server 2710. Such web browsers are typically programs such as MICROSOFT INTERNET EXPLORER / EDGE, MOZILLA FIREFOX, OPERA, APPLE SAFARI, GOOGLE CHROME, etc. In addition, software executed on client 2702 can be downloaded from server computer 2706 to client computer 2702 and installed as a plug-in or ACTIVEX control for the web browser. Thus, client 2702 can utilize ACTIVEX components / Component Object Model (COM) or Distributed COM (DCOM) components to provide a user interface on the display of client 2702. Web server 2710 is typically a program such as MICROSOFT'S INTERNET INFORMATION SERVER.

[0182] The web server 2710 can host an Active Server Pages (ASP) or Internet Server Application Programming Interface (ISAPI) application 2712 that can execute scripts. The scripts call objects (referred to as business objects) that implement business logic. The business objects then manipulate data in a database 2716 through a database management system (DBMS) 2714. Alternatively, the database 2716 can be part of the client 2702 or directly connected to it, rather than communicating / obtaining information from the database 2716 across the network 2704. When developers encapsulate business functionality into objects, the system can be referred to as a Component Object Model (COM) system. Thus, scripts executed on the web server 2710 (and / or application 2712) call COM objects that implement business logic. Furthermore, the server 2706 can utilize Microsoft's Transaction Server (MTS) to access required data stored in the database 2716 via interfaces such as ADO (Active Data Objects), OLE DB (Object Linking and Embedding), or ODBC (Open Database Connectivity).

[0183] Generally speaking, these components 2700 to 2716 include logic and / or data embedded in and / or retrievable from a device, medium, signal, or carrier (e.g., a data storage device, a data communication device, a remote computer or a device connected to a computer via a network or via another data communication device, etc.) Further, such logic and / or data, when read, executed, and / or interpreted, causes the steps required to implement and / or use the present invention to be performed.

[0184] Although the terms "user computer," "client computer," and / or "server computer" are referred to herein, it will be understood that such computers 2702 and 2706 may be interchangeable and may also include thin client devices with limited or full processing capabilities, portable devices (such as cell phones, notebook computers, pocket computers, multi-touch devices), and / or any other device with suitable processing, communication, and input / output capabilities.

[0185] Of course, those skilled in the art will recognize that any combination of the above components, or any number of different components, peripherals, and other devices, may be used with computers 2702 and 2706. Embodiments of the present invention are implemented as a software / CAD application on a client 2702 or server computer 2706. Furthermore, as described above, the client 2702 or server computer 2706 may comprise a thin client device or a portable device with a multi-touch based display.

[0186] in conclusion

[0187] This concludes the description of the preferred embodiments of the present invention. Some alternative embodiments for implementing the present invention are described below. For example, any type of computer (such as a mainframe, minicomputer, or personal computer) or computer configuration (such as a time-sharing mainframe, a local area network, or a stand-alone personal computer) may be used with the present invention.

[0188] In summary, embodiments of the present invention provide a smart block capability that enhances block management in drawings. Smart blocks simplify the creation, editing, and manipulation of blocks. By analyzing drawing context and patterns using ML algorithms, smart blocks provide a range of intelligent features. These include automatic block placement, block replacement, block conversion, and object detection. This provides a valuable tool that increases efficiency and productivity by automating repetitive tasks and streamlining block processing. Furthermore, it enhances workflow and saves time, whether users are working on architectural design or other types of projects.

[0189] The foregoing description of the preferred embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. It is intended that the scope of the present invention be limited not by this specific embodiment, but rather by the appended claims.

Claims

1. A computer-implemented method for providing similar computer-aided design (CAD) blocks, comprising: (a) Training a shape similarity machine learning (ML) model based on labeled images, where: (i) the labeled image includes blocks from the construction and engineering industry; (ii) Each labeled image includes a class label; (iii) the shape similarity model groups blocks together based on shape using the class labels to guide learning; (b) fine-tuning a name similarity ML model based on block metadata information extracted from one or more drawing files, wherein the name similarity model groups blocks together based on their names; (c) receiving a block query for a block, wherein the block query comprises: (i) raster-based block images; and (ii) metadata information associated with the block; (d) processing the raster-based patch image with the shape similarity model to output a shape embedding; (e) processing the metadata information associated with the chunk by the name similarity model to output a name embedding; (f) a suggestion engine combining the shape embedding and the name embedding to identify similar blocks; and (g) providing the identified similar blocks in response to the block query.

2. The computer-implemented method of claim 1 , wherein the metadata comprises: Block name; Drawing name; as well as File path.

3. The computer-implemented method of claim 1 , wherein: generating the shape similarity model using a convolutional neural network (CNN); and The name similarity model is generated using a transformer model.

4. The computer-implemented method of claim 1, wherein the shape similarity model processes the raster-based patch image using an encoder that extracts features from the labeled image.

5. The computer-implemented method of claim 1 , wherein the suggestion engine: sorting the identified similar blocks based on similarity; selecting a limited number of most relevant identified similar blocks based on the order; and The defined number of most relevant identified similar blocks are provided in response to the block query.

6. A computer-implemented method for detecting a computer-aided design (CAD) object, comprising: (a) training machine learning (ML) to process a graphic representing the CAD object, wherein: (b) obtaining a CAD drawing including two-dimensional (2D) geometric primitives; (c) representing the CAD drawing as an input graphic, wherein: (i) the 2D geometric primitive geometry is represented by a geometry tag in a node of the input graph; (ii) spatial KNN (k-nearest neighbor) adjacency relationships between the 2D geometric primitives are represented as edges between the nodes of the input graph; (d) processing the input graph in the ML model, wherein the ML model outputs nodes representing predicted objects of the 2D geometric primitive geometry; (e) representing the 2D geometric primitive geometry using the predicted object; and (f) merging the predicted objects to create a new labeled drawing.

7. The computer-implemented method of claim 6, wherein: The ML model uses a sparse transformer model as an encoder; The sparse transformer model encodes the geometric labels and position embeddings into object representations and geometric figure representations.

8. The computer-implemented method of claim 6, wherein: The ML model is updated via a feedback loop based on confirmation of the predicted object by user input.

9. The computer-implemented method of claim 6, further comprising: preparing geometric feature data according to the predicted object; grouping the predicted objects into classes based on the geometric feature data; The classified predicted objects are converted into blocks.

10. The computer-implemented method of claim 9, wherein said preparing said geometric feature data comprises: Increase the number of sampled vertices on the geometry; Utilizing principal component analysis (PCA) calculation, eigenvalue calculation is performed using all the sampled vertices; as well as Includes the length of the 2D geometric primitive and the ratio of the perimeter to the circumference of the smallest enclosing circle.

11. The computer-implemented method of claim 9, wherein said grouping said predicted objects comprises: classifying the geometric feature data from the 2D geometric primitive geometry into categories using k-means; determining the optimal number of classes by evaluating the results in said classes for increasing values ​​of k; The silhouette algorithm was used to evaluate the quality of the classification; Use compactness-based judgment conditions to resolve premature iteration exit; as well as An enhanced Bikmeans algorithm is used to enhance stability, which eliminates the uncertainty of random sampling of initial classification centers.

12. The computer-implemented method of claim 9, further comprising: Fuzzy logic is used to mitigate position deviations due to the absence or presence of geometric figures in the geometric feature data.

13. The computer-implemented method of claim 12, wherein said utilizing fuzzy logic comprises: sorting the predicted objects by length; scaling the predicted object to a target; comparing geometry types and lengths between the targets and scaling the results of the comparison in order; as well as Based on the order, the best matching geometry is selected to participate in the alignment coordinate calculation.

14. A computer-implemented method for placing blocks in a computer-aided design (CAD) drawing, comprising: (a) determining the block to be placed in the computer-aided design (CAD) drawing, wherein the block comprises a set of geometric objects; (b) a content extractor obtains a block context from an existing instance of the block in the CAD drawing, wherein the block context comprises: (i) a set of objects associated with the block; (ii) geometric data of the set of objects, which have fixed relative positions and orientations with respect to the block; (iii) a background pattern of geometric data of the set of objects; (c) a background matcher that searches for the background pattern around a cursor position where the block is to be placed; and (d) placing the block in the CAD drawing based on a relationship between the existing instances of the block and the background pattern.

15. The computer-implemented method of claim 14 , wherein the background extractor obtains the block background by: For all existing instances of the block, sampling geometric data of the object, wherein the sampling encodes and records the sampled geometric data for subsequent context analysis and extraction; analyzing the sampled geometric data by merging the sampled geometric data having sample points having the same distribution; as well as Determining the background pattern includes a set of sample points having a same position and geometric data of an object having the set of sample points.

16. The computer-implemented method of claim 14, wherein the background matcher uses geometry-based matching to find the background pattern, the method comprising: Taking the cursor position as the center of the circle; Using the radius of the circle as the detection range; Searching for an entity combination around the cursor position within the detection range that can match the geometric data in the background pattern; After a match is found, geometric data of the current background is sampled, and the sampled geometric data is compared with the set of sample points in the background pattern to determine whether the background pattern can be applied to the cursor position; Once it is determined that the background mode is applicable to the cursor position, the coordinate system of the current background is calculated; as well as A rotation angle and a position offset are calculated according to the difference between the coordinate system of the current background and the background pattern coordinate system.

17. The computer-implemented method of claim 14, wherein the context matcher uses location-based matching to find the context pattern, the method comprising: Reversely calculate the center of the block to place it starting from the position of a sampling point on the geometric data of an object in the set of objects that is closest to the cursor position; Using the reverse calculated center as the center of the circle; Sampling the cursor position using a sampling method; as well as Whether the background mode is applicable to the current background is determined by comparing whether the sampled points of the cursor positions match.

Images