Generating 3D articles and model-based definitions from 2D maps using artificial intelligence

Abstract

The invention discloses generation of 3D articles and model-based definitions from 2D maps using artificial intelligence. Generating a 3D model from a 2D graph is provided. The method includes extracting, by a design parser, content from a 2D engineering drawing of the component and comparing the extracted content to a bill of materials corresponding to a 3D computer-aided design (CAD) model of the component to identify missing components from the 3D CAD model. In response to identifying the missing component, a 3D representation of the missing component is modeled based on the 2D engineering drawing and metadata and textual information related to the 2D engineering drawing. The 3D representation of the missing component is incorporated into the 3D CAD model of the component to create a complete 3D CAD model. And then the manufacturing process of the assembly is controlled according to the complete 3D CAD model.

Description

Technical Field

[0001] This disclosure generally relates to computer modeling, and more specifically to generating 3D representations from 2D graphs. Background Technology

[0002] One of the key requirements for model-based engineering manufacturing and production systems is having fully defined model-based design (MBD), where each computer-aided design (CAD) artifact is a true representation of the actual physical object. Manufacturing and production companies are striving to shift from traditional drawing and installation / production instructions to MBD and model-based instruction (MBI). Summary of the Invention

[0003] An illustrative embodiment provides a computer-implemented method for generating a 3D model from 2D drawings. The method includes extracting content from 2D engineering drawings of a component by a design parser and comparing the extracted content with a bill of materials corresponding to a 3D computer-aided design (CAD) model of the component to identify missing parts from the 3D CAD model. In response to identifying the missing parts, a 3D representation of the missing parts is modeled based on the 2D engineering drawings and metadata and textual information associated with the 2D engineering drawings. The 3D representation of the missing parts is incorporated into the 3D CAD model of the component to create a complete 3D CAD model. The manufacturing process of the component is then controlled based on the complete 3D CAD model.

[0004] Another illustrative embodiment provides a system for generating a 3D model from a 2D drawing. The system includes a storage device storing program instructions and one or more processors operatively connected to the storage device and configured to execute the program instructions to cause the system to: extract content from a 2D drawing of a component via a design parser; compare the extracted content with a bill of materials corresponding to a 3D computer-aided design (CAD) model of the component to identify missing parts from the 3D CAD model; in response to identifying missing parts, model a 3D representation of the missing parts based on the 2D drawing and metadata and textual information associated with the 2D drawing; incorporate the 3D representation of the missing parts into the 3D CAD model of the component to create a complete 3D CAD model; and control the manufacturing process of the component based on the complete 3D CAD model.

[0005] An illustrative embodiment provides a computer program product for generating a 3D model from a 2D drawing. The computer program product includes a computer-readable storage medium having program instructions embodied thereon to perform the following operations: extracting content from a 2D engineering drawing of a component by a design parser; comparing the extracted content with a bill of materials corresponding to a 3D computer-aided design (CAD) model of the component to identify missing parts from the 3D CAD model; in response to identifying missing parts, modeling a 3D representation of the missing parts based on the 2D engineering drawing and metadata and textual information associated with the 2D engineering drawing; merging the 3D representation of the missing parts into the 3D CAD model of the component to create a complete 3D CAD model; and controlling the manufacturing process of the component based on the complete 3D CAD model.

[0006] Features and functions may be implemented independently in various embodiments of this disclosure, or may be combined in other embodiments, wherein further details can be seen with reference to the following description and accompanying drawings. Attached Figure Description

[0007] The appended claims set forth novel features that are considered characteristics of the illustrative embodiments. However, the illustrative embodiments, their preferred modes of use, further objects, and features will be best understood by referring to the following detailed description of the illustrative embodiments of this disclosure, in conjunction with the accompanying drawings, in which:

[0008] Figure 1 A block diagram of a 3D representation generator according to an illustrative embodiment is depicted;

[0009] Figure 2 An example of a 2D engineering drawing depicting a component, including parts missing from the 3D CAD model;

[0010] Figure 3 An example of a 3D CAD model for adding missing parts according to an illustrative embodiment is depicted;

[0011] Figure 4 A method for generating a 3D representation from a 2D drawing using CAD macros, according to an illustrative embodiment, is described;

[0012] Figure 5 A method for generating a 3D representation from a 2D drawing using CAD UV mapping map unfolding, according to an illustrative embodiment, is described;

[0013] Figure 6 The first stage of training a 3D semi-generative AI model according to an illustrative embodiment for generating 3D representations from 2D graphs is described.

[0014] Figure 7The second stage of training a 3D semi-generative AI model according to an illustrative embodiment for generating 3D representations from 2D graphs is described.

[0015] Figure 8 The third stage of training a 3D semi-generative AI model according to an illustrative embodiment for generating 3D representations from 2D graphs is described.

[0016] Figure 9 A method for generating 3D representations from 2D graphs using a 3D semi-generative AI model, according to an illustrative embodiment, is described;

[0017] Figure 10 A flowchart illustrating the process of generating a 3D model from a 2D drawing, according to an illustrative embodiment, is provided.

[0018] Figure 11 The process of modeling a 3D representation of a missing part based on a 2D engineering drawing according to an illustrative embodiment is described;

[0019] Figure 12 An alternative process for modeling a 3D representation of a missing component based on a 2D engineering drawing, according to an illustrative embodiment, is described;

[0020] Figure 13 An alternative process for modeling a 3D representation of a missing component based on a 2D engineering drawing, according to an illustrative embodiment, is described;

[0021] Figure 14 A flowchart illustrating the process of training a semi-generative AI model according to an illustrative embodiment is provided.

[0022] Figure 15 A flowchart illustrating a process, according to an illustrative embodiment, of modeling a 3D representation of a missing part based on a 2D engineering drawing using a trained semi-generative AI model; and

[0023] Figure 16 This is an illustration of a block diagram of a data processing system according to an illustrative embodiment. Detailed Implementation

[0024] The illustrative embodiments recognize and consider that one of the key requirements for having a model-based engineering manufacturing and production system is to have a fully defined model-based design (MBD), which means that each computer-aided design (CAD) artifact is a true representation of the actual physical object.

[0025] The exemplary embodiments also recognize and take into account that most aerospace and manufacturing parts and standards do not have 3D representations and / or realistic images of the parts. Operations teams need visual references to target parts or components referenced on 2D drawings.

[0026] Exemplary embodiments provide a method for using generative artificial intelligence to digest disparate production artifacts and documentation (such as 2D drawings, installation steps, specifications, standards, and requirements) to generate MBD artifacts for each part number and sub-assembly / component.

[0027] The illustrative embodiments also provide methods for teaching AI to understand installation steps and requirements and to visualize them on MBD articles.

[0028] Figure 1 This is a block diagram of a 3D representation generator depicted according to an illustrative embodiment. The 3D representation generator 100 compares a 3D CAD model 112 with a 2D drawing 102 to determine if any missing parts 118 exist, including those in the 2D drawing 102 that are not present in the 3D CAD model 112. (See...) Figure 2 ).

[0029] Each 2D drawing 104 in 2D drawing 102 includes several parts 106. Each 2D drawing 104 may also include metadata 108 and text information 110, which can be used to help generate a 3D representation 120 of the missing parts.

[0030] Metadata 108 can be extracted from 2D drawing 102 by design parser 154. Design parser 154 reads and interprets information from 2D drawing 102 regarding constraints on the target object (missing part 118) to be generated. Design parser 154 is also able to cross-reference text information 110.

[0031] The 3D representation generator 100 can use several alternative artificial intelligence (AI) models to generate a 3D representation 120 of the missing part 118 for inclusion in the 3D CAD model 112. (See...) Figure 3 ).

[0032] One model is a CAD macro AI model 122. This model is trained based on commands 124 historically executed by humans when manually generating 3D models from 2D drawings. Based on these historically executed commands 124, the CAD macro AI model 122 generates automatic CAD macros that can be run to generate a 3D representation 120 of the missing part 118 from the 2D drawing 102. (See...) Figure 4 ).

[0033] Another model is the UV unwrapping model 128, which utilizes several trained UV mapping maps 130. UV mapping projects the surfaces of the 3D model onto a 2D image. In UV mapping and unwrapping, U represents the horizontal axis, and V represents the vertical axis in two dimensions, since X, Y, and Z are used to represent axes in 3D modeling. (See...) Figure 5 ).

[0034] Another model is a 3D semi-generative AI model 132, which includes a low-resolution autoencoder 134 and a high-resolution autoencoder 140. The low-resolution autoencoder 134 includes a coarse transformer 136, which can generate a low-resolution code 138 from the masked low-resolution code 146 with the help of embeddings provided by the metadata embedder 150 and the image embedder 152.

[0035] The high-resolution autoencoder 140 includes a fine converter 142 that can generate a high-resolution code 144 from the masked high-resolution code 148 with the aid of the low-resolution code 138 generated by the low-resolution autoencoder 134. (See...) Figures 6-9 ).

[0036] The 3D representation generator 100 can be implemented using software, hardware, firmware, or a combination thereof. When using software, the operations performed by the 3D representation generator 100 can be implemented in program code configured to run on hardware, such as a processor unit. When using firmware, the operations performed by the 3D representation generator 100 can be implemented in program code and data and stored in persistent memory for execution on a processor unit. When using hardware, the hardware can include circuitry that operates to perform the operations in the 3D representation generator 100.

[0037] In the illustrative example, the hardware may take the form of at least one of the following: a circuit system, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware configured to perform certain operations. In the case of a programmable logic device, the device may be configured to perform certain operations. The device may be reconfigured at a later time or may be permanently configured to perform certain operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field-programmable logic arrays, field-programmable gate arrays, and other suitable hardware devices. Additionally, these processes may be implemented in organic components integrated with inorganic components and may consist entirely of organic components (excluding human elements). For example, these processes may be implemented as circuits in organic semiconductors.

[0038] Computer system 160 is a physical hardware system and includes one or more data processing systems. When there is more than one data processing system in computer system 160, these data processing systems communicate with each other using a communication medium. The communication medium may be a network. The data processing system may be selected from at least one of the following: a computer, a server computer, a mobile device such as a tablet computer, or some other suitable data processing system.

[0039] As depicted, computer system 160 includes a plurality of processor units 162 capable of executing program code 164 implementing the process in the illustrative example. As used herein, the processor units among the plurality of processor units 162 are hardware devices and consist of hardware circuitry, such as those on integrated circuits, which respond to and process instructions and program code that operate the computer. When the plurality of processor units 162 execute the program code 164 for the process, the plurality of processor units 162 are one or more processor units that may be on the same computer or different computers. In other words, the process may be distributed among processor units on the same or different computers in the computer system. Furthermore, the plurality of processor units 162 may be processor units of the same type or different types. For example, the plurality of processor units may be selected from at least one of the following: a single-core processor, a dual-core processor, a multi-processor core, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0040] Figure 2 This example depicts a 2D engineering drawing of a component that includes parts missing from the 3D CAD model. In this example, bracket 202 is part of the 2D engineering drawing 200, but is currently missing from the corresponding 3D CAD model and must be added to the 3D CAD model to complete the component's model-based definition (MBD), such as... Figure 3 As shown.

[0041] The 2D drawing 200 is compared with the Bill of Materials (BOM) of the corresponding 3D CAD model of the component in question to identify any parts missing from the MBD in the CAD model. The 3D representation generator of the illustrative embodiment uses Automated Data Processing (ADP) to extract content from the 2D drawing 200. The extracted content may include, for example, markings, sub-components, dimensions, GD&T (geometric dimensions and tolerances) symbols, etc. If all parts extracted from the 2D drawing are present in the BOM, the MBD of the component is complete. However, if any part in the 2D drawing 200 is missing from the BOM of the 3D CAD model, the MBD is incomplete, and these missing parts must be added to the MBD.

[0042] JSON file 204 contains metadata extracted by ADP, listing the parts found in 2D engineering drawing 200 and their corresponding labels. JSON file 204 represents the Manufacturing Bill of Materials (MBOM), which can be compared with the Electronic Bill of Materials (EBOM) (such as...). Figure 3The MBD is compared with EBOM 304. If MBOM 204 and EBOM 304 match, then MBD is complete. In this example, bracket 202 is missing from the 3DCAD model and must be added.

[0043] Figure 3 An example of a 3D CAD model to which a missing part is to be added, according to an illustrative embodiment, is depicted. After the missing part is discovered, the 3D representation generator delves deeper into the corresponding 2D engineering drawings, engineering references, and specifications to model the missing part 302, which is then added to the 3D CAD model 300.

[0044] In addition to modeling the missing part 302 in 3D, the 3D representation generator also generates model-based instructions (MBIs) that describe the correct installation sequence for adding the parts to the 3D CAD model 300 based on textual information in the engineering specifications.

[0045] Figure 4 A method for generating a 3D representation from a 2D drawing using CAD macros, according to an illustrative embodiment, is described. Figure 4 The methods shown document numerous examples of manually creating 3D objects from 2D diagrams.

[0046] These recorded operations are used to learn the types of actions human users take when using a CAD system to generate 3D models of various shapes. In this example, a hollow cylinder with a through hole 410 is generated from an initial circle 402. Starting from the initial circle 402, a solid cylinder 404 is extruded into three dimensions. Next, a 2D silhouette 406 for making cuts is added to the cylinder. The cuts are then extruded to produce a hollow cylinder 408. Next, the through hole 410 is drilled into the hollow cylinder. This sequence can be recorded into a macro. Similar processes can be performed on many other shapes with varying degrees of complexity and special features, such as cones, cubes, etc., all of which can be recorded into macros.

[0047] A large number of such CAD macros can be used to train an AI model to generate an automatic macro 414 when presented with a 2D drawing 412. This automatic macro 414 is then used to generate a 3D representation of the part specified in the 2D drawing 412.

[0048] Figure 5A method for generating a 3D representation from a 2D model using CAD UV mapping unfolding, according to an illustrative embodiment, is described. UV unfolding is the process of flattening the surfaces of a 3D model 502 onto a 2D plane to create a UV mapping 504. This flattening allows the accurate application of 2D images (textures) to the 3D model. This process is similar to peeling an orange and flattening its peel, or representing a globe as a flat map. Specialized algorithms help minimize stretching and deformation during this step.

[0049] The term "UV" refers to the axes of 2D texture coordinates (U for horizontal and V for vertical), distinguishing them from the X, Y, and Z axes of a 3D model.

[0050] AI models can be trained on several such unfolded UV maps to infer how to reverse the process to build a 3D model from a 2D representation.

[0051] Figure 6 A first phase of training a 3D semi-generative AI model according to an illustrative embodiment for generating 3D representations from 2D graphs is depicted. The first phase of training involves training a low-resolution autoencoder 602 and a high-resolution autoencoder 604 to reconstruct a 3D model 606 from reduced-resolution representations 608 and 610.

[0052] In this example, the first down-resolution representation 608 of the full-resolution 3D model 606 includes a voxel resolution of 32x32x32, which is fed as input data into the low-resolution autoencoder 602. The second down-resolution representation 610 has a higher relative resolution of 64x64x64 than the first down-resolution representation 608, but is still lower than the resolution of the original 3D model 606. Both autoencoders could be, for example, vector quantization variational autoencoders (VQ-VAE).

[0053] The low-resolution encoder 602 encodes the first down-resolution representation 608 into a low-resolution code 612, which is then decoded to produce a reconstruction 616 of the original 3D model 606. The high-resolution autoencoder 604 encodes the second down-resolution representation 610 into a high-resolution code 614, which is then decoded to produce a second reconstruction 618 of the original 3D model 606. Through numerous iterations, both autoencoders are trained to reconstruct the original 3D model 606 at full resolution, although starting from the corresponding down-resolution representations 608 and 610 of that model.

[0054] Figure 7The second stage of training a 3D semi-generative AI model according to an illustrative embodiment for generating 3D representations from 2D graphs is described. After training a low-resolution autoencoder to reconstruct the original full-resolution model from the low-resolution code, the low-resolution autoencoder is then trained to predict the low-resolution code 720 from the partially masked low-resolution code 718.

[0055] To aid this reconstruction, the AI ​​design parser 704 extracts metadata 706 from the 2D drawing 702. The metadata embedder 708 embeds this extracted metadata 706 into the mapping network 714. Similarly, the 2D drawing 702 may include multiple images 710 of the object or component in question, such as, for example, top view, bottom view, right side view, left side view, etc. These different views 710 are embedded in an image embedding, which is also fed into the mapping network for cross-referencing with the metadata 706.

[0056] The coarse transformer 716 in the low-resolution autoencoder applies these metadata and image embeddings from the mapping network 714 to the partially masked low-resolution code 718 to learn to predict the unmasked low-resolution code 720.

[0057] Figure 8 A third stage is depicted, according to an illustrative embodiment, of training a 3D semi-generative AI model for generating 3D representations from 2D graphs. A coarse transformer has been trained to predict an unmasked low-resolution code 720 from a partially masked low-resolution code, and the predicted low-resolution code 720 can then be used to train a fine transformer 804 in a high-resolution autoencoder to predict an unmasked high-resolution code 806 from a partially masked high-resolution code 802.

[0058] Figure 9 A method for generating 3D representations from 2D graphs using a 3D semi-generative AI model, according to an illustrative embodiment, is described. After training a coarse transformer and a fine transformer as described above, they can be combined into a single process flow to generate a high-resolution 3D model 926 from a 2D drawing 902.

[0059] and Figure 7 The process is similar, with metadata embeddings 904 and image embeddings 906 generated from the 2D drawing 902 fed into the mapping network 910. However, in this application, at least one cue 908 is also fed into the mapping network 910. The cue 908 may include, for example, specific materials for constructing objects or components modeled in 3D, which act as constraints when constructing the 3D model 926.

[0060] Metadata embedding 904, image embedding 906, and cue 908 help the trained coarse transformer 912 predict the unmasked low-resolution code 916 from the fully masked low-resolution code 914. Then, the trained fine transformer 918 uses the predicted unmasked low-resolution code 916 to predict the unmasked high-resolution code 922 from the fully masked high-resolution code 920.

[0061] The voxel decoder 924 then decodes the predicted unmasked high-resolution code 922 to generate a 3D model 926.

[0062] Figure 10 A flowchart illustrating a process for generating a 3D model from a 2D drawing, according to an illustrative embodiment, is provided. Process 1000 can be performed in... Figure 1 The 3D representation generator 100 is implemented in the system.

[0063] Process 1000 begins with the design parser extracting content from the component's two-dimensional (2D) engineering drawing (operation 1002).

[0064] The extracted content is compared with the bill of materials (BOM) of the corresponding three-dimensional (3D) computer-aided design (CAD) model of the component to identify missing parts from the 3D CAD model (Operation 1004).

[0065] In response to the identification of missing parts, process 1000 models a 3D representation of the missing parts based on 2D engineering drawings and metadata and text information associated with the 2D engineering drawings (operation 1006).

[0066] The 3D representations of missing parts are merged into the 3D CAD model of the component to create a complete 3D CAD model (Operation 1008).

[0067] Process 1000 can generate the assembly sequence of parts within the 3D CAD model of the component based on the text information associated with the 2D engineering drawing (operation 1010).

[0068] Process 1000 controls the manufacturing process of the component based on the complete 3D CAD model (Operation 1012). Then, Process 1000 ends.

[0069] Figure 11 The process of modeling a 3D representation of a missing part based on a 2D engineering drawing, according to an illustrative embodiment, is described. Process 1100 is... Figure 10 A detailed example of the implementation of operation 1006 in the example.

[0070] Process 1100 begins by recording macros in the CAD system to document and mimic the standard commands historically executed by humans to generate 3D object files from 2D drawings (Operation 1102).

[0071] Process 1100 trains an artificial intelligence (AI) model to generate automated CAD macros, which are used to generate 3D object files from 2D drawings based on the recorded macros (Operation 1104). Then, process 1100 ends.

[0072] Figure 12 An alternative process for modeling a 3D representation of a missing part based on a 2D engineering drawing, according to an illustrative embodiment, is described. Process 1200 is... Figure 10 A detailed example of the implementation of operation 1006 in the example.

[0073] Process 1200 begins by training an artificial intelligence (AI) model based on UV mapping of several 3D models (Operation 1202).

[0074] Process 1200 uses a trained artificial intelligence model to generate a 3D representation of the missing part (operation 1204). Then, process 1200 ends.

[0075] Figure 13 An alternative process for modeling a 3D representation of a missing part based on a 2D engineering drawing, according to an illustrative embodiment, is described. Process 1300 is... Figure 10 A detailed example of the implementation of operation 1006 in the example.

[0076] Process 1300 includes using a 3D semi-generative artificial intelligence (AI) model to generate a 3D representation of the missing part, where metadata and textual information provide constraints for the 3D semi-generative AI model (operation 1302).

[0077] Figure 14 A flowchart illustrating the process of training a semi-generative AI model according to an illustrative embodiment is depicted. Process 1400 is an example of training a semi-generative AI model (such as the model used in process 1300).

[0078] Process 1400 begins by generating a first reduced-resolution representation of the 3D trained CAD model (operation 1402) and generates a second reduced-resolution representation of the 3D trained CAD model, wherein the second reduced-resolution representation has a higher relative resolution than the first reduced-resolution representation (operation 1404).

[0079] Train a first autoencoder to reconstruct the trained CAD model from a first down-resolution 3D representation (operation 1406). Train a second autoencoder to reconstruct the trained CAD model from a second down-resolution 3D representation (operation 1408).

[0080] Generate metadata embeddings in vector space from the 2D engineering drawing (Operation 1410). Generate image embeddings in the latent space of the 2D engineering drawing (Operation 1412).

[0081] A coarse transformer is trained based on metadata and image embeddings to reconstruct low-resolution codes from partially masked low-resolution codes, wherein the reconstructed low-resolution codes are used by a first autoencoder (operation 1414).

[0082] A fine transformer is then trained based on the generated low-resolution code to reconstruct the high-resolution code from the partially masked high-resolution code, where the reconstructed high-resolution code is used by the second encoder (operation 1416). Then, process 1400 ends.

[0083] Figure 15 A flowchart illustrating a process for modeling a 3D representation of a missing part based on a 2D engineering drawing using a trained semi-generative AI model, according to an illustrative embodiment, is provided. Process 1500 is a detailed example of the operation of a trained semi-generative AI model (such as the model used in process 1300).

[0084] Process 1500 begins by receiving input of metadata embedding based on 2D engineering diagram in vector space (operation 1502) and input of image embedding based on 2D engineering diagram in latent space (step 1504).

[0085] The semi-generative AI model also receives several prompts as input, which specify the desired material properties of the 3D representation of the missing part (Operation 1506).

[0086] The coarse transformer generates unmasked low-resolution code from the masked low-resolution code based on metadata embedding, image embedding, and cues (Operation 1508).

[0087] The fine transformer generates unmasked high-resolution codes from masked high-resolution codes based on unmasked low-resolution codes (operation 1510).

[0088] The voxel decoder generates a 3D representation of the missing part based on the unmasked high-resolution code (operation 1512). Then, procedure 1500 ends.

[0089] Now go to Figure 16 This illustration depicts a block diagram of a data processing system according to an illustrative embodiment. The data processing system 1600 can be used for implementation. Figure 1The computer system 160 is shown in the example. In this illustrative example, the data processing system 1600 includes a communication framework 1602 that provides communication between a processor unit 1604, a memory 1606, a persistent storage device 1608, a communication unit 1610, an input / output (I / O) unit 1612, and a display 1614. In this example, the communication framework 1602 takes the form of a bus system.

[0090] Processor unit 1604 is used to execute instructions for software that can be loaded into memory 1606. Depending on the specific implementation, processor unit 1604 may be a plurality of processors, a multiprocessor core, or some other type of processor. In one embodiment, processor unit 1604 includes one or more conventional general-purpose central processing units (CPUs). In an alternative embodiment, processor unit 1604 includes one or more graphics processing units (GPUs).

[0091] Memory 1606 and persistent storage device 1608 are examples of storage device 1616. A storage device is any hardware capable of storing information (such as, but not limited to, at least one of, data, program code in functional form, or other suitable information) on a temporary, permanent, or both temporary and permanent basis. In these illustrative examples, storage device 1616 may also be referred to as a computer-readable storage device. In these examples, memory 1606 may be, for example, random access memory or any other suitable volatile or non-volatile storage device. Persistent storage device 1608 may take various forms depending on the specific implementation.

[0092] For example, persistent storage device 1608 may include one or more components or devices. For example, persistent storage device 1608 may be a hard disk drive, flash memory, rewritable optical disk, rewritable magnetic tape, or some combination thereof. The media used in persistent storage device 1608 may also be removable. For example, a removable hard disk drive may be used in persistent storage device 1608. In these illustrative examples, communication unit 1610 provides communication with other data processing systems or devices. In these illustrative examples, communication unit 1610 is a network interface card.

[0093] The input / output unit 1612 allows data input and output to other devices that can be connected to the data processing system 1600. For example, the input / output unit 1612 can provide a connection for user input via at least one of a keyboard, mouse, or some other suitable input device. Furthermore, the input / output unit 1612 can send output to a printer. The display 1614 provides a mechanism for displaying information to the user.

[0094] Instructions for at least one of an operating system, application, or program may reside in storage device 1616, which communicates with processor unit 1604 via communication frame 1602. Processes in different embodiments may be executed by processor unit 1604 using computer-implemented instructions, which may reside in memory (e.g., memory 1606).

[0095] These instructions are referred to as program code, computer-usable program code, or computer-readable program code, which can be read and executed by the processor in processor unit 1604. The program code in different embodiments may be embodied on different physical or computer-readable storage media (such as memory 1606 or persistent storage device 1608).

[0096] Program code 1618 is functionally located on a computer-readable medium 1620 (which may be selectively removable) and may be loaded or transferred to a data processing system 1600 for execution by a processor unit 1604. In these illustrative examples, program code 1618 and computer-readable medium 1620 form a computer program product 1622. In one example, computer-readable medium 1620 may be a computer-readable storage medium 1624 or a computer-readable signal medium 1626.

[0097] In these illustrative examples, computer-readable storage medium 1624 is a physical or tangible storage device for storing program code 1618, and not a medium for propagating or transmitting program code 1618. As used herein, computer-readable storage medium 1624 should not be construed as a transient signal, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0098] Alternatively, program code 1618 may be transmitted to data processing system 1600 using computer-readable signal medium 1626. Computer-readable signal medium 1626 may be, for example, a propagated data signal containing program code 1618. For example, computer-readable signal medium 1626 may be at least one of electromagnetic signals, optical signals, or any other suitable type of signal. These signals may be transmitted via at least one of a communication link (e.g., wireless communication link, fiber optic cable, coaxial cable, wire, or any other suitable type of communication link).

[0099] The different components shown for data processing system 1600 do not imply an architectural limitation on how different embodiments can be implemented. Different illustrative embodiments may be implemented in a data processing system that includes components other than those shown for data processing system 1600 or components that replace those shown for data processing system 1600. Figure 16 Other components shown may differ from the illustrative example illustrated. Different embodiments may be implemented using any hardware device or system capable of running program code 1618.

[0100] As used in this article, when used with a list of items, the phrase "at least one of..." means that different combinations of one or more of the listed items can be used, and only one of each item in the list is required. In other words, "at least one of..." means that any number of items from any combination and item in the list can be used, but not all items in the list are required. Items can be specific objects, things, or categories.

[0101] For example, but not limited to, "at least one of Item A, Item B, or Item C" can include Item A; Item A and Item B; or Item B. This example could also include Item A, Item B, and Item C; or Item B and Item C. Of course, any combination of these items can exist. In some illustrative examples, "at least one of..." can be, for example, but not limited to, two of Item A; one of Item B; and ten of Item C; four of Item B and seven of Item C; or other suitable combinations.

[0102] As used in this article, when referring to projects, "several" means one or more projects. For example, "several different types of networks" means one or more different types of networks. In illustrative examples, "group / set," as used with reference projects, means one or more projects. For example, a set of metrics is one or more metrics.

[0103] The descriptions of various illustrative embodiments have been presented for purposes of illustration and description and are not intended to be exhaustive or limited to the embodiments of the disclosed forms. The various illustrative examples describe components that perform actions or operations. In the illustrative embodiments, components may be configured to perform the described actions or operations. For example, a component may have a configuration or design for a structure that provides the component with the ability to perform the actions or operations described in the illustrative examples as being performed by the component. Furthermore, within the scope of the terms “comprising,” “including,” “having,” “containing,” and variations thereof used herein, these terms are intended to be inclusive in a manner similar to the term “comprising” as an open transition word, without excluding any additional or other elements.

[0104] This application also includes embodiments pursuant to the following provisions:

[0105] Clause 1. A system for generating a 3D model from a 2D drawing, the system comprising:

[0106] Storage device (1616), its stored program instructions;

[0107] One or more processors (1604) operatively connected to the storage device and configured to execute the program instructions to cause the system to:

[0108] The parser (154) extracts the contents of (1002) from the 2D engineering drawing (102) of the component;

[0109] The extracted content is compared with the bill of materials (1004) of the 3D computer-aided design model (i.e., 3D CAD model) corresponding to the component to identify missing parts (118) from the 3D CAD model;

[0110] In response to the identification of the missing part, a 3D representation (120) of the missing part is modeled (1006) based on the 2D engineering drawing and metadata (108) and text information (110) associated with the 2D engineering drawing;

[0111] The 3D representation of the missing part is merged (1008) into the 3D CAD model of the component to create a complete 3D CAD model; and

[0112] The manufacturing process of the component is controlled (1012) based on the complete 3D CAD model.

[0113] Clause 2. The system according to Clause 1, wherein the processor further executes instructions to generate (1010) an assembly sequence of parts within the 3D CAD model of the component based on textual information associated with the 2D drawing.

[0114] Clause 3. The system according to Clause 1, wherein modeling the 3D representation of the missing part includes recording (1102) macros in the CAD system to record and mimic standard commands (124) historically executed by humans to generate 3D object files from 2D drawings.

[0115] Clause 4. The system according to Clause 3, wherein the processor further executes instructions to train (1104) an artificial intelligence model, i.e., an AI model (122), to generate an automatic CAD macro (126) for generating a 3D object file from a 2D drawing based on the recorded macro.

[0116] Clause 5. The system according to Clause 1, wherein modeling the 3D representation of the missing part includes:

[0117] The artificial intelligence model (AI model) is trained based on UV mapping (130) of several 3D models; and

[0118] The 3D representation of the missing part is generated using a trained artificial intelligence model (1204).

[0119] Clause 6. The system according to Clause 1, wherein modeling the 3D representation of the missing part includes using a (1302) 3D semi-generative artificial intelligence model, i.e., a 3D semi-generative AI model (132), to generate the 3D representation of the missing part, wherein the metadata and textual information provide constraints for the 3D semi-generative AI model.

[0120] Clause 7. The system according to Clause 6, wherein the 3D semi-generative artificial intelligence model is trained in the following manner:

[0121] Generate a first reduced-resolution representation of the (1402) 3D trained CAD model;

[0122] Generate (1404) a second reduced-resolution representation of the 3D trained CAD model, wherein the second reduced-resolution representation has a higher relative resolution than the first reduced-resolution representation;

[0123] Training (1406) the first autoencoder (134) to reconstruct the trained CAD model from the first down-resolution 3D representation; and

[0124] The second autoencoder (140) is trained (1408) to reconstruct the trained CAD model from the second reduced-resolution 3D representation.

[0125] Clause 8. A computer program product for generating a 3D model from a 2D drawing, said computer program product comprising:

[0126] A computer-readable storage medium (1624) having program instructions embodied thereon to perform the following operations:

[0127] The design parser (154) extracts the contents of (1002) from the 2D engineering drawing (102) of the component;

[0128] The extracted content is compared with the bill of materials (1004) of the 3D computer-aided design model (i.e., 3D CAD model) corresponding to the component to identify missing parts (118) from the 3D CAD model;

[0129] In response to the identification of the missing part, a 3D representation (120) of the missing part is modeled (1006) based on the 2D engineering drawing and metadata (108) and text information (110) associated with the 2D engineering drawing;

[0130] The 3D representation of the missing part is merged (1008) into the 3D CAD model of the component to create a complete 3D CAD model; and

[0131] The manufacturing process of the component is controlled (1012) based on the complete 3D CAD model.

[0132] Clause 9. The computer program product pursuant to Clause 8 further includes instructions for generating (1010) an assembly sequence of parts within the 3D CAD model of the component based on textual information associated with the 2D engineering drawing.

[0133] Clause 10. The computer program product pursuant to Clause 8, wherein modeling the 3D representation of the missing component includes recording (1102) macros in a CAD system to record and mimic standard commands (124) historically executed by humans to generate 3D object files from 2D drawings.

[0134] Clause 11. The computer program product pursuant to Clause 10 further includes instructions for training (1104) an artificial intelligence model, i.e., an AI model (122), to generate an automated CAD macro (126) for generating a 3D object file from a 2D drawing based on the recorded macro.

[0135] Clause 12. The computer program product according to Clause 8, wherein modeling the 3D representation of the missing part includes:

[0136] The artificial intelligence model (AI model) is trained based on UV mapping (130) of several 3D models; and

[0137] The 3D representation of the missing part is generated using a trained artificial intelligence model (1204).

[0138] Clause 13. The computer program product according to Clause 8, wherein modeling the 3D representation of the missing part includes using a (1302) 3D semi-generative artificial intelligence model, i.e., a 3D semi-generative AI model (132), to generate the 3D representation of the missing part, wherein the metadata and textual information provide constraints for the 3D semi-generative AI model.

[0139] Clause 14. The computer program product pursuant to Clause 13, wherein the 3D semi-generative artificial intelligence model is trained in the following manner:

[0140] Generate a first reduced-resolution representation of the (1402) 3D trained CAD model;

[0141] Generate (1404) a second reduced-resolution representation of the 3D trained CAD model, wherein the second reduced-resolution representation has a higher relative resolution than the first reduced-resolution representation;

[0142] Training (1406) the first autoencoder (134) to reconstruct the trained CAD model from the first down-resolution 3D representation; and

[0143] The second autoencoder (140) is trained (1408) to reconstruct the trained CAD model from the second reduced-resolution 3D representation.

[0144] Many modifications and variations will be apparent to those skilled in the art. Furthermore, different illustrative embodiments may provide different features compared to other desired embodiments. The selection and description of one or more embodiments are intended to best explain the principles of the embodiments, their practical application, and to enable others skilled in the art to understand the disclosure of various embodiments with various modifications suitable for the intended particular purpose.

Claims

1. A computer-implemented method for generating a 3D model from a 2D drawing, the method comprising: Execute using multiple processors: The design parser (154) extracts the contents of (1002) from the 2D engineering drawing (102) of the component; The extracted content is compared with the bill of materials (1004) of the 3D computer-aided design model (i.e., 3D CAD model) corresponding to the component to identify missing parts (118) from the 3D CAD model; In response to the identification of the missing part, a 3D representation (120) of the missing part is modeled (1006) based on the 2D engineering drawing and metadata (108) and text information (110) associated with the 2D engineering drawing; The 3D representation of the missing part is merged (1008) into the 3D CAD model of the component to create a complete 3D CAD model; and The manufacturing process of the component is controlled (1012) based on the complete 3D CAD model.

2. The method of claim 1, further comprising generating (1010) an assembly sequence of parts within the 3D CAD model of the component based on text information associated with the 2D engineering drawing.

3. The method of claim 1, wherein modeling the 3D representation of the missing part includes recording (1102) macros in a CAD system to record and mimic standard commands (124) historically executed by humans to generate 3D object files from 2D drawings.

4. The method of claim 3 further includes training (1104) an artificial intelligence model, i.e., an AI model (122), to generate an automatic CAD macro (126) for generating a 3D object file from a 2D drawing based on the recorded macro.

5. The method of claim 1, wherein modeling the 3D representation of the missing component comprises: The artificial intelligence model (AI model) is trained by UV mapping (130) of several 3D models. and The 3D representation of the missing part is generated using a trained artificial intelligence model (1204).

6. The method of claim 1, wherein modeling the 3D representation of the missing part comprises generating the 3D representation of the missing part using a (1302) 3D semi-generative artificial intelligence model, i.e., a 3D semi-generative AI model (132), wherein the metadata and text information provide constraints for the 3D semi-generative AI model.

7. The method of claim 6, wherein the 3D semi-generative artificial intelligence model is trained in the following manner: Generate a first reduced-resolution representation of the (1402) 3D trained CAD model; Generate (1404) a second reduced-resolution representation of the 3D trained CAD model, wherein the second reduced-resolution representation has a higher relative resolution than the first reduced-resolution representation; Training (1406) the first autoencoder (134) to reconstruct the trained CAD model from the first down-resolution 3D representation; and The second autoencoder (140) is trained (1408) to reconstruct the trained CAD model from the second reduced-resolution 3D representation.

8. The method according to claim 7, further comprising: Metadata embedding in vector space is generated from the 2D engineering drawing (1410); A (1412) image embedding is generated in the latent space of the 2D engineering drawing; as well as A coarse transformer (136) is trained (1414) based on the metadata embedding and image embedding to reconstruct a low-resolution code (138) from a partially masked low-resolution code (146), wherein the reconstructed low-resolution code is used by the first autoencoder.

9. The method according to claim 8, further comprising: A fine transformer (142) is trained (1416) based on the generated low-resolution code to reconstruct a high-resolution code (144) from the partially masked high-resolution code (148), wherein the reconstructed high-resolution code is used by the second autoencoder.

10. The method of claim 6, wherein the 3D semi-generative AI model: Receive (1502) input based on the metadata embedding of the 2D engineering drawing in the vector space; Receive (1502) an input based on the image embedding of the 2D engineering drawing in the latent space; Receive (1506) input of several prompts, the prompts specifying the desired material properties of the 3D representation of the missing part; Based on the metadata embedding, image embedding, and cues, a coarse transformer is used to generate (1508) unmasked low-resolution code from the masked low-resolution code; Based on the unmasked low-resolution code, a fine transformer is used to generate an unmasked high-resolution code (1510) from the masked high-resolution code. as well as Based on the unmasked high-resolution code, the 3D representation of the missing part is generated (1512) using a voxel decoder.

Images