Using artificial intelligence to generate 3D artifacts and model based definition from 2d drawings

Abstract

To provide generating a 3D model from 2D drawings.SOLUTION: The method comprises extracting, by a design parser, content from 2D engineering drawings of an assembly, and comparing the extracted content to a bill of materials corresponding to a 3D computer assisted design (CAD) model of the assembly to identify missing components from the 3D CAD model. In response to identifying missing components, 3D representations of the missing components are modeled based on the 2D engineering drawings, and metadata and textual information related to the 2D engineering drawings. The 3D representations of the missing components are incorporated into the 3D CAD model of the assembly to create a complete 3D CAD model. A manufacturing process for the assembly is then controlled according to the complete 3D CAD model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates generally to computer modeling, and more particularly to generating 3D representations from 2D drawings. [Background technology]

[0002] One of the key requirements for having a model-based technology manufacturing and production system is to have a fully defined Model-Based Design (MBD), where each Computer Aided Design (CAD) artifact is a true representative of the actual physical object. Manufacturing and production companies are struggling to switch from traditional technical drawings and installation / production instructions to MBD and Model-Based Instruction (MBI). Summary of the Invention [Means for solving the problem]

[0003] An exemplary embodiment provides a computer-implemented method for generating a 3D model from a 2D drawing. The method includes extracting, by a design analyzer, content from a 2D engineering drawing of an assembly and comparing the extracted content with a bill of materials corresponding to a 3D computer-aided design (CAD) model of the assembly to identify missing components from the 3D CAD model. In response to identifying the missing components, 3D representations of the missing components are modeled based on the 2D engineering drawing and metadata and text information associated with the 2D engineering drawing. The 3D representations of the missing components are incorporated into the 3D CAD model of the assembly to create a complete 3D CAD model. A manufacturing process for the assembly is then controlled according to the complete 3D CAD model.

[0004] Another exemplary embodiment provides a system for generating a 3D model from a 2D drawing, comprising: a storage device that stores program instructions; and one or more processors operably connected to the storage device that execute the program instructions to cause the system to: extract, with a design analyzer, content from a 2D engineering drawing of an assembly; compare the extracted content with a bill of materials corresponding to a 3D computer-aided design (CAD) model of the assembly to identify missing components from the 3D CAD model; model a 3D representation of the missing component based on the 2D engineering drawing or metadata and text information associated with the 2D engineering drawing in response to identifying the missing component; incorporate the 3D representation of the missing component into the 3D CAD model of the assembly to create a complete 3D CAD model; and control a manufacturing process of the assembly according to the complete 3D CAD model.

[0005] An exemplary embodiment provides a computer program product for generating a 3D model from a 2D drawing. The computer program product includes a computer-readable storage medium having program instructions embodied thereon to perform the following operations: extracting, by a design analyzer, content from a 2D engineering drawing of an assembly; comparing the extracted content with a bill of materials corresponding to a 3D computer-aided design (CAD) model of the assembly to identify missing components from the 3D CAD model; modeling a 3D representation of the missing component based on the 2D engineering drawing and metadata and text information associated with the 2D engineering drawing in response to identifying the missing component; incorporating the 3D representation of the missing component into the 3D CAD model of the assembly to create a complete 3D CAD model; and controlling a manufacturing process of the assembly according to the complete 3D CAD model.

[0006] The forms and functions may be achieved independently in various embodiments of the present disclosure or may be combined in yet other embodiments, further details of which can be seen with reference to the following description and drawings.

[0007] The novel features believed characteristic of the exemplary embodiments are set forth in the appended claims. However, the exemplary embodiments, as well as their preferred modes of use, further objects and features, will best be understood by reference to the following detailed description of exemplary embodiments of the present disclosure, when considered in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a 3D representation generator in accordance with an illustrative embodiment; [Figure 2] FIG. 1 illustrates an example of a 2D technical drawing of an assembly that includes components missing from the 3D CAD model. [Figure 3] FIG. 1 illustrates an example of a 3D CAD model with missing components added, in accordance with an illustrative embodiment. [Figure 4] FIG. 1 illustrates a method for generating a 3D representation from a 2D drawing using a CAD macro, in accordance with an exemplary embodiment. [Figure 5] FIG. 1 illustrates a method for generating a 3D representation from a 2D drawing using CAD UV map unwrapping, according to an example embodiment. [Figure 6] FIG. 1 illustrates a first stage of training a 3D semi-generative AI model to generate 3D representations from 2D drawings, according to an exemplary embodiment. [Figure 7] FIG. 10 illustrates a second stage of training a 3D semi-generative AI model to generate 3D representations from 2D drawings, according to an exemplary embodiment. [Figure 8] FIG. 10 illustrates a third stage of training a 3D semi-generative AI model to generate 3D representations from 2D drawings, according to an exemplary embodiment. [Figure 9]FIG. 1 illustrates a method for generating a 3D representation from a 2D drawing using a 3D semi-generative AI model, according to an exemplary embodiment. [Figure 10] 1 is a flowchart illustrating a process for generating a 3D model from a 2D drawing in accordance with an illustrative embodiment; [Figure 11] FIG. 1 illustrates a process for modeling a 3D representation of a missing component based on a 2D technical drawing, in accordance with an illustrative embodiment. [Figure 12] FIG. 10 illustrates an alternative process for modeling a 3D representation of a missing component based on a 2D technical drawing, in accordance with an illustrative embodiment. [Figure 13] FIG. 10 illustrates an alternative process for modeling a 3D representation of a missing component based on a 2D technical drawing, in accordance with an illustrative embodiment. [Figure 14] 1 is a flowchart illustrating a process for training a semi-generative AI model, in accordance with an illustrative embodiment. [Figure 15] 10 is a flowchart illustrating a process for modeling a 3D representation of a missing component based on a 2D technical drawing with a trained semi-generative AI model, in accordance with an illustrative embodiment. [Figure 16] 1 is an illustration of a block diagram of a data processing system in accordance with an illustrative embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0009] The illustrative embodiments recognize and take into account that one of the key requirements for having a model-based engineering manufacturing and production system is to have a fully defined model-based design (MBD), meaning that each computer-aided design (CAD) artifact actually represents a real physical object.

[0010] The illustrative embodiments also recognize and take into account that the majority of aerospace and manufacturing parts and standards do not have 3D representations and / or actual images of the parts, and that the operations team needs to have a visual reference of the target part or assembly referenced on a 2D drawing.

[0011] The illustrative embodiments provide a method for using generative artificial intelligence to understand scattered manufacturing artifacts and documents such as 2D drawings, installation steps, specifications, standards and requirements to generate MBD artifacts per part number and subassembly / assembly.

[0012] The illustrative embodiments also provide a method for teaching an AI to understand installation steps and requirements and visualize them on MBD artifacts.

[0013] 1 is a block diagram of a 3D representation generator depicted in accordance with an exemplary embodiment. The 3D representation generator 100 compares a 3D CAD model 112 with a 2D technical drawing 102 to determine whether there are any missing components 118 present in the 3D CAD model 112 included in the 2D technical drawing 102 (see FIG. 2).

[0014] Each 2D technical drawing 102 of the 2D technical drawings 104 includes several components 106. Each 2D technical drawing 104 may also include metadata 108 and text information 110 that can be used to assist in generating a 3D representation 120 of the missing components.

[0015] The metadata 108 can be extracted from the 2D engineering drawing 102 by a design analyzer 154. The design analyzer 154 reads and interprets information about the limitations of the target object (missing component 118) to be generated from the 2D engineering drawing 102. The design analyzer 154 can also cross-reference the text information 110.

[0016] The 3D representation generator 100 may use several alternative artificial intelligence (AI) models to generate a 3D representation 120 of the missing component 118 for inclusion in the 3D CAD model 112. (See FIG. 3)

[0017] One model is a CAD macro AI model 122. This model is trained according to historically executed commands 124 made by humans when manually generating a 3D model from a 2D drawing. Based on these historically executed commands 124, the CAD macro AI model 122 generates an automated CAD macro that can be executed to generate a 3D representation 120 of the missing component 118 from the 2D technical drawing 102 (see FIG. 4).

[0018] Another model is a UV unwrapping model 128 that utilizes several training UV maps 130. UV mapping projects the surface of a 3D model onto a 2D image for mapping. In UV mapping and unwrapping, X, Y, and Z are used to indicate the axes of 3D modeling, so U represents the horizontal axis and V represents the vertical axis in 2D (see Figure 5).

[0019] Another model is a 3D semi-generative AI model 132 that comprises a low-resolution autoencoder 134 and a high-resolution autoencoder 140. The low-resolution autoencoder 134 comprises a coarse transformer 136 that can generate low-resolution code 138 from masked low-resolution code 146 with the help of embeddings provided by a metadata embedder 150 and an image embedder 152.

[0020] The high-resolution autoencoder 140 comprises a fine converter 142 that can generate a high-resolution code 144 from a masked high-resolution code 148 with the help of the low-resolution code 138 generated by the low-resolution autoencoder 134 (see Figures 6 to 9).

[0021] The 3D representation generator 100 can be implemented in software, hardware, firmware, or a combination thereof. When software is used, the operations performed by the 3D representation generator 100 can be implemented in program code running on hardware, such as a processor unit. When firmware is used, the operations performed by the 3D representation generator 100 can be implemented in program code and data and stored in persistent memory running on a processor unit. When hardware is employed, the hardware can include circuitry that operates to perform the operations in the 3D representation generator 100.

[0022] In illustrative examples, the hardware can take the form of at least one of a circuit system, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, or other suitable type of hardware that performs multiple operations. Using a programmable logic device, the device can be configured to perform multiple operations. The device can be later reconfigured or permanently configured to perform multiple operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field programmable logic arrays, field programmable gate arrays, and other suitable hardware devices. Additionally, the process can be implemented in organic components integrated with inorganic components, or can be comprised entirely of organic components, excluding humans. For example, the process can be implemented as a circuit of organic semiconductors.

[0023] Computer system 160 is a physical hardware system and includes one or more data processing systems. When multiple data processing systems are present in computer system 160, the data processing systems communicate with each other using a communication medium. The communication medium may be a network. The data processing systems may be selected from at least one of a computer, a server computer, a mobile device such as a tablet computer, or other suitable data processing system.

[0024] As shown, computer system 160 includes multiple processor units 162 capable of executing program code 164 that implements processes in the illustrative example. As used herein, a processor unit of multiple processor units 162 is a hardware device, comprised of hardware circuitry, such as that on an integrated circuit, that processes in response to instructions and program code that cause a computer to operate. When multiple processor units 162 execute program code 164 for a process, multiple processor units 162 are one or more processor units that may be on the same computer or different computers. In other words, a process may be distributed among processor units on the same or different computers of a computer system. Furthermore, the multiple processor units 162 may be the same or different types of processor units. For example, the multiple processor units may be selected from at least one of a single-core processor, a dual-core processor, a multi-processor core, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0025] Figure 2 illustrates an example of a 2D engineering drawing of an assembly that includes a component missing from a 3D CAD model. In this example, a bracket 202 is part of the 2D engineering drawing 200 but is currently missing from the corresponding 3D CAD model and must be added to the 3D CAD model to complete the model-based definition (MBD) of the assembly, as shown in Figure 3.

[0026] The 2D technical drawing 200 is compared with the bill of materials (BOM) of the corresponding 3D CAD model of the assembly in question to identify components missing from the MBD of the CAD model. The 3D representation generator of the exemplary embodiment uses automated data processing (ADP) to extract content from the 2D technical drawing 200. The extracted content may include, for example, flag notes, subassemblies, dimensions, GD&T (geometric dimensioning and tolerancing) symbols, etc. If all components extracted from the 2D technical drawing are present in the BOM, the MBD of the assembly is complete. However, if components in the 2D technical drawing 200 are missing from the BOM of the 3D CAD model, the MBD is incomplete, and those missing components must be added to the MBD.

[0027] JSON file 204 contains ADP extracted metadata that lists the components and their respective labels found in 2D technical drawing 200. JSON file 204 represents a manufacturing bill of materials (MBOM), which can be compared to an electronic bill of materials (EBOM), such as EBOM 304 in FIG. 3. If MBOM 204 and EBOM 304 match, the MBD is complete. In this example, bracket 202 is missing from the 3D CAD model and must be added.

[0028] 3 is a diagram illustrating an example of a 3D CAD model with missing components added, according to an example embodiment. When a missing component is discovered, the 3D representation generator dives deeper into the corresponding 2D technical drawings and technical references and specifications to model the missing component 302, which is then added to the 3D CAD model 300.

[0029] In addition to modeling the missing component 302 in 3D, the 3D representation generator also generates model-based instructions (MBIs) that describe the appropriate installation sequence for adding that component to the 3D CAD model 300 based on textual information provided in the technical specification.

[0030] 4 is a diagram illustrating a method for generating a 3D representation from a 2D drawing using a CAD macro, according to an example embodiment. The approach shown in FIG. 4 documents numerous examples of manually creating a 3D object from a 2D drawing.

[0031] These recorded actions are used to generate 3D models of many types of shapes using a CAD system to learn the types of actions human users take. In this example, a hollow cylinder with a through hole 410 is generated from an initial circle 402. Starting from the initial circle 402, a solid cylinder 404 is extruded into three dimensions. Next, a 2D silhouette 406 of a cut is added to the cylinder. The cut is then extruded to generate a hollow cylinder 408. Next, a through hole 410 is drilled through the hollow cylinder. This sequence can be recorded into a macro. A similar process can be performed for numerous other shapes, such as cones, cubes, and other shapes of various levels of complexity with special features, all of which can be recorded into a macro.

[0032] A large number of such CAD macros can be used to train an AI model that, when presented with a 2D technical drawing 412, generates an automated macro 414. This automated macro 414 is then used to generate a 3D representation of the part specified in the 2D technical drawing 412.

[0033] 5 illustrates how CAD UV map unwrapping can be used to generate a 3D representation from a 2D drawing, according to an example embodiment. UV unwrapping is the process of flattening the surface of a 3D model 502 onto a 2D plane to create a UV map 504. This flattening allows for accurate application of 2D images (textures) to the 3D model. This process is similar to peeling an orange and laying the peel flat, or representing a sphere as a flat map. Special algorithms help minimize stretching and distortion during this step.

[0034] The term "UV" refers to the axes of the 2D texture coordinates (U for horizontal, V for vertical), distinguishing them from the X, Y, and Z axes of a 3D model.

[0035] An AI model can be trained on several such unwrapped UV maps to estimate how to reverse the process to build a 3D model from a 2D representation.

[0036] 6 illustrates a first stage of training a 3D semi-generative AI model for generating 3D representations from 2D drawings, according to an example embodiment. This first stage of training involves training a low-resolution autoencoder 602 and a high-resolution autoencoder 604 to reconstruct a 3D model 606 from reduced-resolution representations 608 and 610.

[0037] In this example, a first reduced resolution representation 608 of a full resolution 3D model 606 has a voxel resolution of 32x32x32 that is fed as input data to a low-resolution autoencoder 602. A second reduced resolution representation 610 has a higher relative resolution, 64x64x64, than the first reduced resolution representation 608, but is still a lower resolution than the original 3D model 606. Both autoencoders may be, for example, vector quantization variational autoencoders (VQ-VAE).

[0038] The low-resolution autoencoder 602 encodes a first reduced resolution representation 608 into a low-resolution code 612, which is then decoded to generate a reconstruction 616 of the original 3D model 606. The high-resolution autoencoder 604 encodes a second reduced resolution representation 610 into a high-resolution code 614, which is then decoded to generate a second reconstruction 618 of the original 3D model 606. Over many iterations, both autoencoders are trained to reconstruct the full resolution of the original 3D model 606, despite starting from their respective reduced-resolution representations 608, 610 of that model.

[0039] 7 illustrates the second stage of training a 3D semi-generative AI model for generating 3D representations from 2D drawings, according to an example embodiment. After being trained to reconstruct the original full-resolution model from the low-resolution code, the low-resolution autoencoder is trained to predict low-resolution code 720 from the partially masked low-resolution code 718.

[0040] To aid in this reconstruction, AI design analyzer 704 extracts metadata 706 from 2D technical drawing 702. Metadata embedder 708 embeds this extracted metadata 706 into mapping network 714. Similarly, 2D technical drawing 702 may contain multiple images 710 of the object or assembly in question, e.g., top view, bottom view, right side view, left side view, etc. These different views 710 are embedded in image embeddings that are also fed into the mapping network for cross-referencing with metadata 706.

[0041] The coarse transformer 716 in the low-resolution autoencoder applies these metadata and image embeddings from the mapping network 714 to the partially masked low-resolution code 718 to learn to predict the unmasked low-resolution code 720.

[0042] 8 illustrates a third stage of training a 3D semi-generative AI model for generating 3D representations from 2D drawings, according to an example embodiment. The coarse transformer is trained to predict unmasked low-resolution code 720 from partially masked low-resolution code, which can then be used to train a fine transformer 804 in a high-resolution autoencoder to predict unmasked high-resolution code 806 from partially masked high-resolution code 802.

[0043] 9 illustrates a method for generating a 3D representation from a 2D drawing using a 3D semi-generative AI model, according to an example embodiment. After training the coarse and fine transformers as described above, they can be combined into a single process flow to generate a high-resolution 3D model 926 from a 2D technical drawing 902.

[0044] 7 , the metadata embeddings 904 and image embeddings 906 generated from the 2D technical drawing 902 are provided to a mapping network 910. However, in the present application, at least one prompt 908 is also provided to the mapping network 910. The prompt 908 may include, for example, the particular materials used to construct the 3D modeled object or assembly, which act as constraints in constructing the 3D model 926.

[0045] The metadata embeddings 904, image embeddings 906, and prompts 908 help a trained coarse transformer 912 predict unmasked low-resolution codes 914 from fully masked low-resolution codes 916. The predicted unmasked low-resolution codes 916 are then used by a trained fine transformer 918 to predict unmasked high-resolution codes 922 from fully masked high-resolution codes 920.

[0046] A voxel decoder 924 then decodes the predicted unmasked high resolution code 922 to generate a 3D model 926 .

[0047] 10 is a flowchart illustrating a process for generating a 3D model from a 2D drawing, according to an example embodiment. Process 1000 can be implemented in 3D representation generator 100 of FIG.

[0048] Process 1000 begins by extracting content from a two-dimensional (2D) engineering drawing of an assembly by a design analyzer (act 1002).

[0049] The extracted content is compared to a bill of materials (BOM) corresponding to a three-dimensional (3D) computer-aided design (CAD) model of the assembly to identify missing components from the 3D CAD model (act 1004).

[0050] In response to identifying the missing component, process 1000 models a 3D representation of the missing component based on the 2D technical drawing and metadata and text information associated with the 2D technical drawing (act 1006).

[0051] The 3D representation of the missing component is incorporated into the 3D CAD model of the assembly to create a complete 3D CAD model (operation 1008).

[0052] Process 1000 may generate a sequence of assembly of components in a 3D CAD model of the assembly from textual information associated with the 2D technical drawings (operation 1010).

[0053] Process 1000 controls the manufacturing process of the assembly according to the complete 3D CAD model (operation 1012), after which process 1000 ends.

[0054] 11 illustrates a process for modeling a 3D representation of a missing component based on a 2D technical drawing, according to an example embodiment. Process 1100 is a detailed example of an implementation of operation 1006 of FIG.

[0055] Process 1100 begins by recording a macro in a CAD system to document and mimic standard commands historically performed by humans to generate 3D object files from 2D drawings (act 1102).

[0056] Process 1100 trains an artificial intelligence (AI) model to generate automated CAD macros for generating 3D object files from the 2D drawings based on the recorded macros (act 1104), after which process 1100 ends.

[0057] 12 illustrates an alternative process for modeling a 3D representation of a missing component based on a 2D technical drawing, according to an example embodiment. Process 1200 is a detailed example of an implementation of operation 1006 of FIG. 10.

[0058] Process 1200 begins by training an artificial intelligence (AI) model based on UV mapping of several 3D models (act 1202).

[0059] The process 1200 uses the trained artificial intelligence model to generate a 3D representation of the missing component (act 1204), after which the process 1200 ends.

[0060] 13 illustrates an alternative process for modeling a 3D representation of a missing component based on a 2D technical drawing, according to an example embodiment. Process 1300 is a detailed example of an implementation of operation 1006 of FIG. 10.

[0061] Process 1300 includes using a 3D semi-generative artificial intelligence (AI) model to generate a 3D representation of the missing component, with metadata and text information providing constraints to the 3D semi-generative AI model (operation 1302).

[0062] 14 is a flowchart illustrating a process for training a semi-generative AI model according to an example embodiment. Process 1400 is an example of training a semi-generative AI model such as that used in process 1300.

[0063] Process 1400 begins by generating a first reduced resolution representation of the 3D training CAD model (operation 1402) and generating a second reduced resolution representation of the 3D training CAD model, the second reduced resolution representation having a higher relative resolution than the first reduced resolution representation (operation 1404).

[0064] A first autoencoder is trained to reconstruct a training CAD model from the first reduced-resolution 3D representation (operation 1406), and a second autoencoder is trained to reconstruct a training CAD model from the second reduced-resolution 3D representation (operation 1408).

[0065] Metadata embeddings are generated in vector space from the 2D technical drawing (act 1410). Image embeddings are generated in a latent space of the 2D technical drawing (act 1412).

[0066] A coarse transformer is trained according to the metadata and image embedding to reconstruct the low-resolution code from the partially masked low-resolution code, and the reconstructed low-resolution code is used by the first autoencoder (operation 1414).

[0067] A fine transformer is then trained according to the generated low-resolution code to reconstruct a high-resolution code from the partially masked high-resolution code, and the reconstructed high-resolution code is used by a second autoencoder (operation 1416), after which process 1400 ends.

[0068] 15 is a flowchart illustrating a process for modeling a 3D representation of a missing component based on a 2D technical drawing with a trained semi-generative AI model, according to an example embodiment. Process 1500 is a detailed example of the operation of a trained semi-generative AI model such as that used in process 1300.

[0069] Process 1500 begins by receiving input of metadata embedding into a vector space based on a 2D technical drawing (act 1502) and receiving input of image embedding into a latent space based on the 2D technical drawing (act 1504).

[0070] The semi-generative AI model also receives input of several prompts specifying desired material properties of the 3D representation of the missing component (operation 1506).

[0071] A coarse transformer generates unmasked low-resolution code from the masked low-resolution code according to the metadata embedding, image embedding, and prompts (operation 1508).

[0072] A fine converter generates an unmasked high-resolution code from the masked high-resolution code based on the unmasked low-resolution code (operation 1510).

[0073] The voxel decoder generates a 3D representation of the missing components based on the unmasked high-resolution codes (act 1512), after which process 1500 ends.

[0074] Referring now to Figure 16, an illustration of a block diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system 1600 may be used to implement computer system 160 of Figure 1. In this illustrative example, data processing system 1600 includes a communications framework 1602 that communicates between a processor unit 1604, a memory 1606, persistent storage 1608, a communications unit 1610, an input / output (I / O) unit 1612, and a display 1614. In this example, communications framework 1602 takes the form of a bus system.

[0075] Processor unit 1604 is responsible for executing instructions for software that may be loaded into memory 1606. Processor unit 1604 may be multiple processors, a multi-processor core, or some other type of processor, depending on the particular implementation. In one example, processor unit 1604 comprises one or more conventional general-purpose central processing units (CPUs). In an alternative embodiment, processor unit 1604 comprises one or more graphical processing units (GPUs).

[0076] Memory 1606 and persistent storage 1608 are examples of storage device(s) 1616. A storage device is any piece of hardware that can store information, either temporarily, permanently, or both temporarily and permanently, such as, for example, but not limited to, data, program code in functional form, or other suitable information. Storage device 1616 may be referred to as a computer-readable storage device in these illustrative examples. Memory 1606, in these examples, may be, for example, a random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 1608 may take various forms depending on the particular implementation.

[0077] For example, persistent storage 1608 may include one or more components or devices. For example, persistent storage 1608 may be a hard drive, a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination thereof. The medium used by persistent storage 1608 may be removable. For example, a removable hard drive may be used for persistent storage 1608. In these illustrative examples, communications unit 1610 communicates with other data processing systems or devices. In these illustrative examples, communications unit 1610 is a network interface card.

[0078] Input / output unit 1612 allows for the input and output of data with other devices that may be connected to data processing system 1600. For example, input / output unit 1612 may provide a connection for user input via at least one of a keyboard, a mouse, or some other suitable input device. Further, input / output unit 1612 may send output to a printer. Display 1614 provides a mechanism for displaying information to a user.

[0079] Instructions for at least one of the operating system, applications, or programs may be located in storage devices 1616, which are in communication with processor unit 1604 through communications framework 1602. The processes of the different embodiments may be performed by processor unit 1604 using computer-implemented instructions, which may be located in a memory, such as memory 1606.

[0080] These instructions are referred to as program code, computer usable program code, or computer readable program code, which may be read and executed by a processor in processor unit 1604. The program code in different embodiments may be embodied on different physical or computer readable storage media, such as memory 1606 or persistent storage 1608.

[0081] Program code 1618 is located in a functional form on computer readable media 1620 that is selectively removable and may be loaded onto or transferred to data processing system 1600 for execution by processor unit 1604. Program code 1618 and computer readable media 1620 form computer program product 1622 in these depicted examples. In one example, computer readable media 1620 may be computer readable storage media 1624 or computer readable signal media 1626.

[0082] In these illustrative examples, computer readable storage medium 1624 is not a medium that propagates or transmits program code 1618, but rather a physical or tangible storage device used to store program code 1618. Computer readable storage medium 1624 as used herein should not be interpreted as being a transitory signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted through wires. Computer readable medium as used herein should not be interpreted as being a transitory signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted through wires.

[0083] Alternatively, program code 1618 may be transferred to data processing system 1600 using computer readable signal medium 1626. Computer readable signal medium 1626 may be, for example, a propagated data signal containing program code 1618. For example, computer readable signal medium 1626 may be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted over at least one of a communications link, such as a wireless communications link, an optical fiber cable, a coaxial cable, a wire, or any other suitable type of communications link.

[0084] The different components illustrated for data processing system 1600 are not meant to provide architectural limitations to the manner in which different embodiments may be implemented. The different illustrative embodiments may be implemented in a data processing system including components in addition to or instead of those illustrated for data processing system 1600. Other components illustrated in FIG. 16 may differ from the illustrated illustrative example. The different embodiments may be implemented using any hardware device or system capable of running program code 1618.

[0085] As used herein, the phrase "at least one of," when used in conjunction with a list of items, means that different combinations of one or more of the listed items can be used, and that only one of each item in the list may be required. In other words, "at least one of" means that any combination and number of items can be used from the list, but not all of the items in the list may be required. An item can be a specific object, thing, or category.

[0086] For example, "at least one of item A, item B, or item C" may include, but is not limited to, item A, item A and item B, or item B. Examples of this may include item A, item B, item C, or item B and item C. Of course, any combination of these items may be present. In some illustrative examples, "at least one of" may be, for example, but not limited to, two items A, one item B, and ten items C, four items B and seven items C, or other suitable combinations.

[0087] As used herein, "plurality," when used with reference to an item, means one or more items. For example, "plurality of different types of networks" is one or more different types of networks. In illustrative examples, a "set" used with a reference item means one or more items. For example, a set of metrics is one or more of the metrics.

[0088] The descriptions of different exemplary embodiments are presented for purposes of illustration and description and are not intended to be exhaustive or limited to the disclosed forms of embodiments. Various exemplary examples describe components that perform actions or operations. In the exemplary examples, a component may be configured to perform the described actions or operations. For example, a component may have a structural configuration or design that provides the component with the ability to perform the actions or operations described as being performed by the component in the exemplary examples. Furthermore, to the extent that the terms "include," "including," "has," "contain," and variations thereof are used herein, such terms are intended to be inclusive, similar to the open transitional term "comprise," without excluding any additional or other elements.

[0089] Many modifications and variations will be apparent to those skilled in the art. Furthermore, various exemplary embodiments may provide different configurations than other preferred embodiments. The selected embodiment or embodiments have been chosen and described in order to best explain the principles, practical applications of the embodiments, and to enable others skilled in the art to understand the disclosure of the various embodiments with various modifications suited to the particular use contemplated. [Explanation of symbols]

[0090] 100 3D representation generator, 102 2D technical drawing, 104 2D technical drawing, 106 components, 108 metadata, 110 text information, 112 CAD model, 118 missing components, 120 3D representation, 122 CAD macro AI model, 124 commands, 128 UV unwrapping model / UV unwrapping AI model, 130 training UV map, 132 3D semi-generative AI model, 134 low-resolution autoencoder, 136 coarse converter, 138 low-resolution code, 140 high-resolution autoencoder, 142 fine converter, 144 high-resolution code, 146 masked low-resolution code, 148 masked high-resolution code, 150 metadata embedder, 152 image embedder, 154 design analyzer, 160 computer system, 162 processor unit, 164 program code, 200 2D technical drawing, 202 bracket, 204 JSON file, 300 CAD model, 302 missing components, 402 initial circle, 404 solid cylinder, 406 2D silhouette, 408 hollow cylinder, 410 through hole, 412 2D technical drawing, 414 automation macro, 502 3D model, 504 UV map, 602 low-resolution autoencoder, 604 high-resolution autoencoder, 606 3D model, 608 first reduced-resolution representation, 610 second reduced-resolution representation, 612 low-resolution code, 614 high-resolution code, 616 reconstruction, 618 second reconstruction, 702 2D technical drawing, 704 AI design analyzer, 706 metadata, 708 metadata embedding, 710 image, 714 mapping network, 716 coarse transformer, 718 masked low-resolution code, 720 unmasked low-resolution code, 802 partially masked high-resolution code, 804 Fine converter, 806 Unmasked high-resolution code, 902 2D technical drawings, 904 Metadata embedding, 906 Image embedding, 908 Prompt, 910 Mapping network, 912 Coarse converter, 914 Unmasked low-resolution code, 916 Low-resolution code, 918 Converter, 920 Fully masked high-resolution code, 922 Unmasked high-resolution code, 924 Voxel decoder, 926 3D models, 1000Process, 1100 process, 1200 process, 1300 process, 1400 process, 1500 process, 1600 data processing system, 1602 communication framework, 1604 processor unit, 1606 memory, 1608 persistent storage, 1610 communication unit, 1612 input / output unit, 1614 display, 1616 storage device, 1618 program code, 1620 computer readable medium, 1622 computer program product, 1624 computer readable storage medium, 1626 computer readable signal medium

Claims

1. 1. A computer-implemented method for generating a 3D model from a 2D drawing, comprising: Using some processors, extracting (1002) content from a 2D engineering drawing (102) of an assembly by a design analyzer (154); comparing (1004) the extracted content with a bill of materials corresponding to the 3D Computer Aided Design (CAD) model (112) of the assembly to identify missing components (118) from the 3D CAD model; In response to identifying the missing component, modeling (1006) a 3D representation (120) of the missing component based on the 2D technical drawing and metadata (108) and text information (110) associated with the 2D technical drawing; Incorporating (1008) the 3D representation of the missing component into the 3D CAD model of the assembly to create a complete 3D CAD model; controlling (1012) a manufacturing process of the assembly according to the complete 3D CAD model; To carry out A method comprising:

2. The method of claim 1 , further comprising generating (1010) a sequence of assembly of components in the 3D CAD model of the assembly from textual information associated with the 2D technical drawing.

3. 2. The method of claim 1, wherein modeling the 3D representation of the missing component comprises recording (1102) a macro in a CAD system to document and mimic standard commands (124) historically performed by humans to generate a 3D object file from a 2D drawing.

4. 4. The method of claim 3, further comprising: training (1104) an artificial intelligence (AI) model (122) to generate an automated CAD macro (126) for generating a 3D object file from a 2D drawing based on the recorded macro.

5. modeling the 3D representation of the missing component, training (1202) an artificial intelligence (AI) model based on UV mapping (130) of several 3D models; generating (1204) the 3D representation of the missing component using the trained artificial intelligence model; 2. The method of claim 1, comprising:

6. 2. The method of claim 1, wherein modeling the 3D representation of the missing component comprises using (1302) a 3D semi-generative artificial intelligence (AI) (132) model to generate the 3D representation of the missing component, and wherein the metadata and text information provide constraints to the 3D semi-generative AI model.

7. The 3D semi-generative artificial intelligence model: generating a first reduced resolution representation of a 3D training CAD model (1402); generating (1404) a second reduced resolution representation of the 3D training CAD model, the second reduced resolution representation having a higher relative resolution than the first reduced resolution representation; training (1406) a first autoencoder (134) to reconstruct the 3D training CAD model from the first reduced resolution 3D representation; training (1408) a second autoencoder (140) to reconstruct the 3D training CAD model from the second reduced resolution 3D representation; The method of claim 6, wherein the training is performed by

8. generating (1410) a metadata embedding in a vector space from the 2D technical drawing; generating an image embedding in a latent space of the 2D technical drawing (1412); training (1414) a coarse transformer (136) according to the metadata and image embedding to reconstruct a low-resolution code (138) from the partially masked low-resolution code (146), wherein the reconstructed low-resolution code is used by the first autoencoder; 8. The method of claim 7, further comprising:

9. training (1416) a fine transformer (142) according to the generated low-resolution code to reconstruct a high-resolution code (144) from a partially masked high-resolution code (148), wherein the reconstructed high-resolution code is used by the second autoencoder. The method of claim 8.

10. the 3D semi-generative AI model: receiving (1502) an input of embedding metadata into a vector space based on the 2D technical drawing; receiving (1504) an input of an image embedding in a latent space based on the 2D technical drawing; receiving 1506 input of a number of prompts specifying desired material properties of the 3D representation of the missing component; generating 1508 unmasked low-resolution code from the masked low-resolution code using the metadata embedding, image embedding, and a coarse converter according to prompts; generating 1510 an unmasked high-resolution code from the masked high-resolution code using a fine converter based on the unmasked low-resolution code; generating 1512 the 3D representation of the missing component based on the unmasked high resolution code using a voxel decoder; The method of claim 6.

11. 1. A system for generating a 3D model from a 2D drawing, the system comprising: a storage device (1616) for storing program instructions; one or more processors (1604) operatively connected to the storage device, extracting (1002) content from a 2D engineering drawing (102) of an assembly by a design analyzer (154); comparing (1004) the extracted content with a bill of materials corresponding to the 3D Computer Aided Design (CAD) model (112) of the assembly to identify missing components (118) from the 3D CAD model; In response to identifying the missing component, modeling (1006) a 3D representation (120) of the missing component based on the 2D technical drawing and metadata (108) and text information (110) associated with the 2D technical drawing; Incorporating (1008) the 3D representation of the missing component into the 3D CAD model of the assembly to create a complete 3D CAD model; controlling (1012) a manufacturing process of the assembly according to the complete 3D CAD model; one or more processors (1604) that execute the program instructions to cause the system to Including, the system.

12. 12. The system of claim 11, wherein the processor further executes instructions for generating (1010) a sequence of assembly of components in the 3D CAD model of the assembly from textual information associated with the 2D technical drawing.

13. 12. The system of claim 11, wherein modeling the 3D representation of the missing component includes recording (1102) a macro in a CAD system to document and mimic standard commands (124) historically performed by humans to generate a 3D object file from a 2D drawing.

14. 14. The system of claim 13, wherein the processor further executes instructions for training (1104) an artificial intelligence (AI) model (122) to generate an automated CAD macro (126) for generating a 3D object file from a 2D drawing based on the recorded macro.

15. modeling the 3D representation of the missing component, training an artificial intelligence (AI) model (1202) based on UV mapping (130) of several 3D models; generating 1204 the 3D representation of the missing component using the trained artificial intelligence model; The system of claim 11 , comprising:

16. 12. The system of claim 11, wherein modeling the 3D representation of the missing component comprises using (1302) a 3D semi-generative artificial intelligence (AI) model (132) to generate the 3D representation of the missing component, and the metadata and text information provide constraints to the 3D semi-generative AI model.

17. The 3D semi-generative artificial intelligence model: generating 1402 a first reduced resolution representation of a 3D training CAD model; generating 1404 a second reduced resolution representation of the 3D training CAD model, the second reduced resolution representation having a higher relative resolution than the first reduced resolution representation; training (1406) a first autoencoder (134) to reconstruct the 3D training CAD model from the first reduced-resolution 3D representation; training (1408) a second autoencoder (140) to reconstruct the 3D training CAD model from the second reduced-resolution 3D representation; The system of claim 16, wherein the system is trained by

18. 1. A computer program for generating a 3D model from a 2D drawing, comprising: extracting (1002) content from a 2D engineering drawing (102) of an assembly by a design analyzer (154); comparing (1004) the extracted content with a bill of materials corresponding to the 3D Computer Aided Design (CAD) model (112) of the assembly to identify missing components (118) from the 3D CAD model; In response to identifying the missing component, modeling (1006) a 3D representation (120) of the missing component based on the 2D technical drawing and metadata (108) and text information (110) associated with the 2D technical drawing; Incorporating (1008) the 3D representation of the missing component into the 3D CAD model of the assembly to create a complete 3D CAD model; controlling (1012) a manufacturing process of the assembly according to the complete 3D CAD model; A computer program comprising instructions for causing a processor to execute the program.

19. 20. The computer program of claim 18, further comprising instructions for generating (1010) a sequence of assembly of components in the 3D CAD model of the assembly from textual information associated with the 2D technical drawing.

20. 20. The computer program product of claim 18, wherein modeling the 3D representation of the missing component comprises recording (1102) a macro in a CAD system to document and mimic standard commands (124) historically performed by humans to generate a 3D object file from a 2D drawing.

21. 21. The computer program of claim 20, further comprising instructions for training (1104) an artificial intelligence (AI) model (122) to generate an automated CAD macro (126) for generating a 3D object file from a 2D drawing based on the recorded macro.

22. modeling the 3D representation of the missing component, training (1202) an artificial intelligence (AI) model based on UV mapping (130) of several 3D models; generating (1204) the 3D representation of the missing component using the trained artificial intelligence model; 20. The computer program of claim 18, comprising:

23. 20. The computer program product of claim 18, wherein modeling the 3D representation of the missing component comprises using (1302) a 3D semi-generative artificial intelligence (AI) model (132) to generate the 3D representation of the missing component, and wherein the metadata and text information provide constraints to the 3D semi-generative AI model.

24. The 3D semi-generative artificial intelligence model: generating a first reduced resolution representation of a 3D training CAD model (1402); generating (1404) a second reduced resolution representation of the 3D training CAD model, the second reduced resolution representation having a higher relative resolution than the first reduced resolution representation; training (1406) a first autoencoder (134) to reconstruct the 3D training CAD model from the first reduced resolution 3D representation; training (1408) a second autoencoder (140) to reconstruct the 3D training CAD model from the second reduced resolution 3D representation; 24. The computer program of claim 23, wherein the computer program is trained according to

Images