Generating a prismatic cad model by machine learning

Abstract

Methods, systems, and devices (including media encoded computer program products) for computer-aided design and manufacturing of physical structures by generating a prismatic CAD model using machine learning include obtaining an input embedding encoding a representation of a target two-dimensional (2D) shape; processing the input embedding using a 2D decoder of a 2D autoencoder to obtain a decoded representation of the target 2D shape; determining a fitted 2D parametric sketch model for the input embedding, including finding a 2D parametric sketch model for the input embedding using a search in an embedding space of the 2D autoencoder and a database of sketch models associated with the 2D autoencoder, and fitting the 2D parametric sketch model to the decoded representation of the target 2D shape; and using the fitted 2D parametric sketch model in a computer modeling program.

Description

Background Technology

[0001] This specification relates to the computer-aided design and manufacture of physical structures.

[0002] Computer-aided design (CAD) software has been developed and used to generate three-dimensional (3D) representations of objects, and computer-aided manufacturing (CAM) software has been developed and used to manufacture the physical structures of those objects, for example, using computer numerical control (CNC) manufacturing technology. Typically, CAD software uses the Boundary Representation (B-Rep) format to store a 3D representation of the geometry of the object being modeled. A B-Rep model is a set of connected surface elements that specify the boundaries between the solid and non-solid parts of the 3D object being modeled. In a B-Rep model (often referred to as B-Rep), the geometry is stored in the computer using smooth and precise mathematical surfaces, which contrasts with the discrete and approximate surfaces of mesh models, which can be difficult to use in CAD programs.

[0003] Furthermore, CAD software has been designed to perform generative design processes, such as automatically generating 3D geometry for one or more parts in a larger system of parts to be manufactured. This automatic generation of 3D geometry is typically confined to a design space specified by the CAD software user, and the generation is often influenced by design objectives and constraints, which can be defined and imported into the CAD software by the user or another party. Design objectives (such as minimizing scrap or weight of the designed parts) can be used to drive the geometry generation process towards a better design. Design constraints can include both structural integrity constraints on individual parts (i.e., the requirement that parts should not fail under expected structural loads during their use) and physical constraints imposed on the larger system (i.e., the requirement that parts do not interfere with other parts in the system during their use).

[0004] However, 3D models generated directly from the generative design process may not be suitable for manufacturing. For example, models from generative design may have rounded edges and shapes that differ from human-designed sketches. Typically, CAD software users need to manually adjust the 3D model to generate a modified 3D model suitable for manufacturing. Summary of the Invention

[0005] This specification describes techniques for computer-aided design of physical structures using machine learning to generate prism CAD models.

[0006] Generally, one or more aspects of the subject matter described in this specification can be embodied in one or more methods (and one or more non-transitory computer-readable media that tangibly encode a computer program operable to cause a data processing device to perform operations), the methods comprising: obtaining an input embedding that encodes a representation of a target two-dimensional (2D) shape; processing the input embedding using a 2D decoder of a 2D autoencoder to obtain a decoded representation of the target 2D shape, wherein the 2D autoencoder includes a 2D encoder that processes a representation of a 2D object to generate an object embedding, and a decoder that processes the object embedding to generate a 2D object. A 2D decoder representing a code; determining a fitted 2D parametric sketch model for an input embedding, including: using a search in the embedding space of a 2D autoencoder and a sketch model database associated with the 2D autoencoder to find a 2D parametric sketch model for the input embedding, wherein the shape of the 2D parametric sketch model is determined by one or more parameter values ​​of the 2D parametric sketch model; and a decoded representation of fitting the 2D parametric sketch model to a target 2D shape by modifying one or more parameter values ​​of the 2D parametric sketch model to produce a fitted 2D parametric sketch model; and using the fitted 2D parametric sketch model in a computer modeling program.

[0007] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: obtaining a parameterized instance of a 2D parameterized sketch model; generating 2D training images from the parameterized instances of the 2D parameterized sketch model, wherein each of the 2D training images corresponds to a parameterized instance of the 2D parameterized sketch model; and training a 2D autoencoder on the 2D training images, including: for each of the 2D training images: processing the 2D training image using a 2D encoder to generate an embedding; and processing the embedding using a 2D decoder to generate a decoded 2D image; calculating a value of a loss function by comparing each of the 2D training images with its corresponding decoded 2D image; and updating the parameters of the 2D encoder and the 2D decoder based on the value of the loss function. Training a 2D autoencoder on 2D training images may include: generating a signed distance field image from the 2D training image; and processing the signed distance field image using a 2D encoder to generate an embedding.

[0008] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: obtaining an initial input embedding that encodes a representation of a target three-dimensional (3D) object; processing the initial input embedding using a sub-embedding decoder of a 3D autoencoder to obtain sub-embeddings including the input embeddings, wherein the sub-embeddings encode 2D shapes that define the representation of the target 3D object; generating a parametric sketch model, including: processing each of the sub-embeddings using one or more intermediate 2D decoders to obtain 2D shapes that define the representation of the target 3D object; generating each of the intermediate embeddings by processing each of the 2D shapes using a 2D encoder of a 2D autoencoder; and performing the determination of a corresponding parametric sketch model in the parametric sketch model for each of the intermediate embeddings, wherein the corresponding parametric sketch model is a fitted 2D parametric sketch model, wherein the decoded representation of the target 2D shape is each of the 2D shapes; generating a set of extrusion parameters from the sub-embeddings; and generating a 3D boundary representation (B-Rep) model of the target 3D object, wherein the generation includes constructing the 3D object by extruding it into 3D space using the fitted 2D parametric sketch model in a construction sequence. The B-Rep model, wherein the construction sequence includes the set of extrusion parameters.

[0009] The sub-embedding decoder may include a multilayer perceptron (MLP). One or more intermediate 2D decoders may include 2D decoders of a 2D autoencoder. One or more intermediate 2D decoders may include a second 2D decoder different from the 2D decoder of the 2D autoencoder. The 3D autoencoder may include: a 3D encoder that processes an input voxel model to generate 3D object embeddings; a sub-embedding decoder that processes the 3D object embeddings to generate sub-embeddings; a start envelope decoder that processes each sub-embedding in the sub-embeddings to generate a start envelope function; a finish envelope decoder that processes each sub-embedding in the sub-embeddings to generate a finish envelope function, wherein a set of extrusion parameters is generated from the start envelope function and the finish envelope function; and a differentiable distributed engine that generates a reconstructed model by extruding 2D shapes into 3D space using the start envelope function and the finish envelope function.

[0010] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: obtaining training examples, wherein each training example includes a training voxel model, a ground reality voxel model, and a ground reality 2D shape, wherein the ground reality voxel model is defined by extruding the ground reality 2D shape into 3D space using a set of ground reality extrusion parameters that define a ground reality envelope function, wherein the training voxel model is generated from the ground reality voxel model; and training a 3D autoencoder on the training examples, including: for each training example: processing the training voxel model using a 3D encoder to generate a 3D object embedding of the training voxel model; and processing the 3D object embedding using a sub-embedding decoder to generate a sub-embedding; and using a 2D decoder The process involves processing each sub-embedded element to generate a 2D shape within a 2D shape; processing each sub-embedded element using a start envelope decoder to generate a start envelope function for the 2D shape; processing each sub-embedded element using an end envelope decoder to generate an end envelope function for the 2D shape; generating a reconstructed voxel model of the training voxel model by constructing a reconstructed voxel model in 3D space using the 2D shape with a predicted construction sequence, wherein the predicted construction sequence includes operations defined in a differentiable distributed engine and start and end envelope functions for each 2D shape within the 2D shape; calculating the value of a first loss function by comparing each training voxel model with its corresponding reconstructed voxel model; and updating the parameters of the 3D autoencoder based at least on the value of the first loss function.

[0011] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: calculating the value of a second loss function by comparing a ground-based 2D shape with the 2D shape and comparing a start envelope function and an end envelope function with a ground-based envelope function; and updating the parameters of the 3D autoencoder based at least on the values ​​of the first and second loss functions. A training voxel model can be generated from the ground-based voxel model through morphological modifications. The 3D autoencoder may include decoding modules, and each of the decoding modules corresponds to a different set of predefined one or more extrusion directions and different predefined one or more Boolean operations, wherein each of the decoding modules may include a corresponding sub-embedding decoder, a corresponding start envelope decoder, and a corresponding end envelope decoder. Obtaining the initial input embedding may include: generating a first 3D object embedding from a first voxel model of a first 3D object; generating a second 3D object embedding from a second voxel model of a second 3D object; and generating the initial input embedding from the first and second 3D object embeddings.

[0012] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: obtaining an initial voxel model of a target 3D object; processing the initial voxel model by morphological modifications to generate a modified voxel model; processing the modified voxel model by a 3D encoder included in a 3D autoencoder to generate an initial input embedding; and using a construction sequence to generate a reconstructed 3D B-Rep model from a fitted 2D parametric sketch model by extruding it into 3D space, wherein the reconstructed 3D B-Rep model is similar to the initial voxel model.

[0013] The method (or operation performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: obtaining an initial voxel model of a target 3D object, wherein the initial voxel model is generated from generative design output; generating an initial input embedding by processing the initial voxel model using a 3D encoder included in a 3D autoencoder; and generating a 3D prism model of the target 3D object, wherein the 3D prism model of the target 3D object is a 3D B-Rep model.

[0014] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may further include: obtaining an initial voxel model of the target 3D object, wherein the initial voxel model is generated from generative design output; generating an initial input embedding by processing the initial voxel model using a 3D encoder included in a 3D autoencoder; and producing a fitted 3D B-Rep model by varying one or more parameters of the 3D B-Rep model to the initial voxel model of the target 3D object. Using the fitted 2D parametric sketch model may include: displaying the fitted 2D parametric sketch model in a user interface of a computer modeling program.

[0015] The method (or operations performed by a data processing device according to a computer program tangibly encoded in one or more non-transitory computer-readable media) may include: obtaining an input 2D image, wherein the input 2D image comprises two or more 2D shapes; generating sub-image portions from the input 2D image, wherein each sub-image portion depicts one of the two or more 2D shapes; generating a corresponding sub-image portion embedding for each sub-image portion; determining a fitted 2D parametric sketch model, including: performing the determination of each fitted 2D parametric sketch model for each sub-image portion embedding; and generating a combined 2D parametric sketch model by combining the fitted 2D parametric sketch models at corresponding locations in the sub-image portions.

[0016] One or more aspects of the subject matter described in this specification can also be embodied in one or more systems, said systems comprising: a non-transitory storage medium having instructions for a computer-aided design program stored thereon; and one or more data processing devices configured to execute the instructions of the computer-aided design program to perform operations specified by the instructions of the computer-aided design program; wherein said operations include: obtaining an input embedding that encodes a representation of a target two-dimensional (2D) shape; processing the input embedding using a 2D decoder of a 2D autoencoder to obtain a decoded representation of the target 2D shape, wherein the 2D autoencoder includes processing the representation of the 2D object to generate a representation of the target 2D shape. The method includes: a 2D encoder for object embedding and a 2D decoder for processing object embeddings to generate a decoded representation of a 2D object; determining a fitted 2D parametric sketch model for an input embedding, including: using a search in the embedding space of the 2D autoencoder and a sketch model database associated with the 2D autoencoder to find a 2D parametric sketch model for the input embedding, wherein the shape of the 2D parametric sketch model is determined by one or more parameter values ​​of the 2D parametric sketch model; and a decoded representation of the 2D parametric sketch model to fit the 2D parametric sketch model to a target 2D shape by modifying one or more parameter values ​​of the 2D parametric sketch model to produce a fitted 2D parametric sketch model; and using the fitted 2D parametric sketch model in a computer modeling program.

[0017] The operation may include: obtaining a parameterized instance of a 2D parameterized sketch model; generating 2D training images from the parameterized instances of the 2D parameterized sketch model, wherein each of the 2D training images corresponds to a parameterized instance of the 2D parameterized sketch model; and training a 2D autoencoder on the 2D training images, including: for each of the 2D training images: processing the 2D training image using a 2D encoder to generate an embedding; and processing the embedding using a 2D decoder to generate a decoded 2D image; calculating a loss function by comparing each of the 2D training images with its corresponding decoded 2D image; and updating the parameters of the 2D encoder and the 2D decoder based on the value of the loss function.

[0018] The operation may include: obtaining an initial input embedding that encodes a representation of a target 3D object; processing the initial input embedding using a sub-embedding decoder of a 3D autoencoder to obtain sub-embeddings that include the input embeddings, wherein the sub-embeddings encode 2D shapes that define the representation of the target 3D object; generating a parametric sketch model, including: processing each of the sub-embeddings using one or more intermediate 2D decoders to obtain 2D shapes that define the representation of the target 3D object; generating each of the intermediate embeddings by processing each of the 2D shapes using a 2D encoder of a 2D autoencoder; and performing the determination of a corresponding parametric sketch model in the parametric sketch model for each of the intermediate embeddings, wherein the corresponding parametric sketch model is a fitted 2D parametric sketch model, wherein the decoded representation of the target 2D shape is each of the 2D shapes; generating a set of extrusion parameters from the sub-embeddings; and generating a 3D boundary representation (B-Rep) model of the target 3D object, wherein the generation includes constructing a 3D B-Rep model by extruding into 3D space using the fitted 2D parametric sketch model in a construction sequence, wherein the construction sequence includes the set of extrusion parameters.

[0019] A 3D autoencoder may include: a 3D encoder that processes an input voxel model to generate a 3D object embedding; a sub-embedding decoder that processes the 3D object embedding to generate a sub-embedding; a start envelope decoder that processes each sub-embedding to generate a start envelope function; a stop envelope decoder that processes each sub-embedding to generate a stop envelope function, wherein a set of extrusion parameters is generated from the start envelope function and the stop envelope function; and a differentiable distributed engine that generates a reconstructed model by extruding a 2D shape into 3D space using the start envelope function and the stop envelope function.

[0020] The operation may include: obtaining training examples, wherein each training example includes a training voxel model, a ground reality voxel model, and a ground reality 2D shape, wherein the ground reality voxel model defines the ground reality 2D shape by extruding it into 3D space using a set of ground reality extrusion parameters that define a ground reality envelope function, wherein the training voxel model is generated from the ground reality voxel model; and training a 3D autoencoder on the training examples, including: for each training example: processing the training voxel model using a 3D encoder to generate a 3D object embedding of the training voxel model; and processing the 3D object embedding using a sub-embedding decoder to generate sub-embeddings; and processing each sub-embedding in the sub-embeddings using a 2D decoder to generate a 2D shape in the 2D shape. Shape; processing each sub-embedded in the sub-embedded using a start envelope decoder to generate a start envelope function for a 2D shape; processing each sub-embedded using an end envelope decoder to generate an end envelope function for a 2D shape; and generating a reconstructed voxel model of a training voxel model by using the 2D shape to expand the construction of the reconstructed voxel model in 3D space with a predicted construction sequence, wherein the predicted construction sequence includes operations defined in a differentiable distributed engine and a start envelope function and an end envelope function for each 2D shape; calculating the value of a first loss function by comparing each training voxel model with its corresponding reconstructed voxel model; and updating the parameters of the 3D autoencoder based at least on the value of the first loss function.

[0021] Specific implementations of the subjects described in this specification can be implemented to achieve one or more of the following advantages: Machine learning models (such as decoders of autoencoders) can take embedding vectors as input and can generate prism boundary representation (B-Rep) models of target 3D objects without providing the target geometry, and prism models can be easier to manufacture than other types of models (e.g., voxel models). Machine learning models can reconstruct 3D models from approximate data (such as point clouds or meshes). Machine learning models can generate 3D models that are interpolations between existing models. Machine learning models can perform style transfer, such as transferring local details between existing models. In some cases, machine learning models can automatically generate 3D prism B-Rep models that resemble human-designed geometry based on the output of generative design or topology optimization processes, which can reduce the time required to manually adjust 3D models to obtain models suitable for manufacturing. In some cases, machine learning models can automatically generate 3D models where design criteria are specified only for one or more parts, while the design of the entire 3D model is unspecified or unknown. For example, machine learning models can be used to create parts based on some approximate shapes.

[0022] 3D models generated by machine learning models are editable because they are generated using constrained sketches and an editable history of parametric features. Generating prism models as CAD construction sequences (e.g., a sequence of steps for generating CAD models) can be highly nonlinear or irregular, as very similar CAD sequences can produce very different shapes, while very different CAD sequences can produce nearly identical shapes. Machine learning models can process input embedding vectors, rather than encoding them in a sequence space, which encodes the voxel representation of the target 3D object in the embedding space of an autoencoder. Since similar shapes in the voxel representation may have similar embeddings generated by the autoencoder's encoder, changes in the autoencoder's embedding space can cause proportional and continuous changes in the corresponding output 3D shape. Therefore, small changes in the embedding vectors can correspond to small changes in the shape of the 3D model, thereby improving the reliability and predictability of the generated 3D model. In some implementations, complex shapes can be divided into smaller parts in image space, and a fitted parametric sketch model can be searched for each smaller part.

[0023] One or more machine learning models can be trained using a first loss function supervising the predicted 3D voxel model and a second loss function supervising the 2D shape (contour) used to construct the predicted 3D voxel model. By using both 3D and 2D supervision during training, the machine learning model can automatically generate 3D models with desired features (e.g., 2D contours or shapes that can be used for CAD extrusion) without needing to know the contour sequence and envelope function. Machine learning functions can be trained to process embeddings of signed distance field images generated using a signed distance function. Therefore, the machine learning function can generate more desirable 2D or 3D models because the signed distance field image provides an improved description of the 2D or 3D shape.

[0024] Details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the detailed description, the drawings, and the claims. Attached Figure Description

[0025] Figure 1 An example of a system that can be used to design and manufacture physical structures is shown.

[0026] Figure 2A An example of the process of generating a 3D model of a target 3D object using a machine learning algorithm is shown.

[0027] Figure 2B This is a flowchart illustrating an example of the process of generating a 3D model of a target 3D object using a machine learning algorithm.

[0028] Figure 3 This is a flowchart and corresponding system operation diagram illustrating an example of the process of generating a fitted 2D parametric sketch model of a target 2D shape using a machine learning algorithm.

[0029] Figure 4 This is a flowchart and corresponding system operation diagram illustrating an example of the process of training a 2D autoencoder that fits a 2D parametric sketch model to determine the 2D shape of a target.

[0030] Figure 5 An example of the process of training a 3D autoencoder that can be used to generate a 3D model of a target 3D object is shown.

[0031] Figure 6 This is a flowchart illustrating an example of the process of training a 3D autoencoder that can be used to generate a 3D model of a target 3D object.

[0032] Figure 7 An example of a neural network architecture for a 3D autoencoder that can be used to generate a 3D model of a target 3D object is shown.

[0033] Figure 8A An example of the process for determining a fitted 2D parametric sketch model for complex 2D shapes is shown.

[0034] Figure 8B An example of the process of generating sub-image portions from an input 2D image is shown.

[0035] Figure 9 It is a schematic diagram of a data processing system that can be used to implement the described system and technology.

[0036] In the various figures, similar figure labels and names indicate similar elements. Detailed Implementation

[0037] Figure 1 An example of a system 100 that can be used to design and manufacture a physical structure is shown. Computer 110 includes a processor 112 and memory 114, and computer 110 can be connected to a network 140, which can be a private network, a public network, a virtual private network, etc. Processor 112 can be one or more hardware processors, each of which can include multiple processor cores. Memory 114 can include both volatile and non-volatile memory, such as random access memory (RAM) and flash RAM. Computer 110 can include various types of computer storage media and devices that can include memory 114 to store instructions for programs running on processor 112.

[0038] Such programs include one or more 3D modeling, simulation, and / or manufacturing control programs, such as computer-aided design (CAD) program 116. Program 116 may run locally on computer 110, remotely on one or more remote computer systems 150 (e.g., one or more server systems from one or more third-party providers accessible from computer 110 via network 140), or both locally and remotely. Machine learning algorithm 134 may be stored in memory 114 (and / or in one or more remote computer systems 150) and may be accessible by CAD program 116.

[0039] The CAD program 116 presents a user interface (UI) 122 on the display device 120 of the computer 110, which can be operated using one or more input devices 118 of the computer 110 (e.g., keyboard and mouse). It should be noted that although in Figure 1 While shown as separate devices, display device 120 and / or input device 118 may also be integrated with each other and / or with computer 110, such as in a tablet computer or a virtual reality (VR) or augmented reality (AR) system. For example, input / output devices 118, 120 may include VR input gloves 118a and VR headset 120a.

[0040] User 190 can interact with program 116 to create and / or load representation 132 of target 3D object 136. The exact voxel model of target 3D object 136 is unknown, and only the idea or a portion of target 3D object 136 is known. Representation 132 can be a result of an initial design (e.g., a blob or blurred 3D voxel model) of target 3D object 136 generated from a generative design process. Figure 1 In some embodiments not shown, representation 132 may be a 2D image of the target 3D object 136. In some cases, representation 132 may be encoded using a 3D encoder to generate an embedding that encodes the representation of the target 3D object, such as an embedding vector or an embedding matrix. For example, when the target 3D object is a shape between two known shapes, representation 132 may be the result of interpolation between two embeddings of the two known shapes.

[0041] CAD program 116 can use machine learning algorithm 134 to generate a 3D model 135 (e.g., a 3D B-Rep model) of the target 3D object 136 from a representation 132 of the target 3D object 136. Machine learning algorithm 134 can automatically decode representation 132 to generate 2D shapes or outlines as fitted 2D parametric sketch models 133, including 2D shapes 133(a), 133(b), and 133(c). The fitted 2D parametric sketch model 133 can be obtained by performing a search in a database 130 of sketch models, which may be local to computer 110 (as shown) or part of a remote computer system 150. CAD program 116 can automatically generate a 3D model 135 of the target 3D object 136, such as a 3D B-Rep model, by extruding the fitted 2D parametric sketch model 133 into 3D space in a construction sequence. The automated process of decoding representation 132 and generating 3D model 135 can accelerate the 3D model generation process, and the generated 3D model 135 can be easily adapted for manufacture by CNC machine 170. Therefore, the automated process can reduce the time required to produce manufacturable structures, and similarly reduce the time required from conceptualizing new parts to the actual manufacturing of those new parts. Details of decoding representation to generate fitted 2D parametric sketch models are combined below. Figure 2A and Figure 2B Describe it.

[0042] Program 116 may include a series of menus in UI 122 that allow user 190 to accept or reject one or more 3D models 135 automatically generated by machine learning algorithm 134. In some implementations, program 116 may include a series of menus in UI 122 that allow user 190 to adjust one or more parts of the 3D model 135 until the user is satisfied with the 3D model. Once user 190 accepts the 3D model generated by the machine learning algorithm, program 116 can save the 3D model in 3D model document 160. Furthermore, user 190 can interact with program 116 to make additional modifications to the 3D model 135 before it is saved in 3D model document 160. For example, rounded corners may be fitted based on the 3D model 135.

[0043] The 3D model 135 generated by the machine learning algorithm of CAD program 116 can be used by a computer-controlled manufacturing system, such as a CNC machine 170 (e.g., a subtractive manufacturing (SM) machine), to manufacture the target 3D object 136. This can be accomplished using known graphical user interface tools. Various known 3D modeling formats can be used in the computer to define the 3D model 135, such as using solid models (e.g., voxels) or surface models (e.g., B-Rep, surface meshes).

[0044] Once the 3D model 135 of the target 3D object 136 is ready for manufacturing, the 3D model 135 can be prepared to manufacture the physical structure of the object 136 by generating toolpaths for use by a computer-controlled manufacturing system. For example, the 3D model 135 can be used to generate a toolpath specification document 165, which can be sent to a CNC machine 170 and used to control the operation of one or more milling cutters. This can be done upon request from user 190, or upon request from the user for another action, such as sending the 3D model document 160 to the CNC machine 170, or to other manufacturing machinery that can be directly connected to computer 110 or connected via network 140, as shown. This can involve a post-processing step, performed on a local computer 110 or a cloud service, to export the 3D model document 160 to an electronic document upon which manufacturing is based. It should be noted that an electronic document (hereinafter simply referred to as a document) can be a file, but does not necessarily correspond to a file. A document can be stored as a part of a file that stores other documents, in a single file dedicated to the document in question, or in multiple collaborative files.

[0045] In any case, program 116 can create one or more toolpaths in document 165 and provide document 165 (with appropriate format) to CNC machine 170 to create the physical structure of target 3D object 136. (Note that in some embodiments, computer 110 is integrated into CNC machine 170, so toolpath specification document 165 is created by the same computer that will use toolpath specification document 165 to manufacture object 136.) For example, CNC machine 170 may be a subtractive manufacturing machine that manufactures object 180 by removing stock material. CNC machine 170 can use toolpaths (e.g., stored in document 165) to control cutting tool 174.

[0046] While the examples above focus on subtractive manufacturing, various other manufacturing systems and processes can also be used, including additive manufacturing (AM) and hybrid AM+SM manufacturing. Furthermore, the systems and techniques described in this application can also be used with 3D modeling programs 116 that support construction management or video and film production. For example, as described in this application, interpolation between existing models can be used to generate new 3D characters (in a video / film production workflow) that are a hybrid of two existing 3D characters. Other applications are also possible. The systems and techniques described in this application can also be used with other forms of representation of the target 3D object. For example, the input to the system can include a point cloud representing the target 3D object, and the machine learning algorithm 134 can include a pointnet neural network (Qi, Charles R. et al., “Pointnet: Deep learning on point sets”, IEEE Conference Proceedings on Computer Vision and Pattern Recognition, 2017), which can be trained to process point cloud data. For example, the input to the system can include natural language text, which can be decoded using a pre-trained natural language processing machine learning model. In some implementations, the system input may include both a voxel representation 132 and a second representation (e.g., a 2D image, point cloud, text, etc.).

[0047] Figure 2A An example of a process 200 for generating a 3D model of a target 3D object using a machine learning algorithm is shown. Figure 2B This is a flowchart illustrating an example of a process for generating a 3D model of a target 3D object using a machine learning algorithm. Procedure 116 obtains initial input embeddings that encode 232 representations of the target 3D object. (See reference) Figure 2A Procedure 116 obtains an initial input embedding 206. The initial input embedding 206 may be the output of a 3D encoder 204 that encodes an initial design (e.g., an initial 3D voxel model 202 of a target 3D object). The initial input embedding 206 may be a one-dimensional vector having floating-point numbers or other types of numerical values, for example, a vector of integer values ​​of length 512, 1024, or 2048 bits, bytes, words, or double words. In some embodiments, the initial input embedding 206 may be an embedding matrix. In some embodiments, the initial input embedding 206 may be any embedding vector or matrix corresponding to an unknown 3D shape.

[0048] As used in this specification, an embedding is a digital representation of image features or 3D voxel model features of a target 3D object. Specifically, an embedding is a digital representation in an embedding space, i.e., an ordered set of a fixed number of numerical values, where the number of values ​​is equal to the dimension of the embedding space. The embedding space can be a high-dimensional space, and an embedding can be a point in that high-dimensional space. For example, an embedding can be a vector of floating-point or other types of numerical values. Generally, the dimension of the embedding space is much smaller than the number of numerical values ​​in an image or voxel model represented by a given embedding. A query embedding can be used to perform a search to find another embedding that is close to a query embedding in the embedding space, because similar shapes can have similar embeddings (e.g., embedding vectors) in the embedding space.

[0049] The 3D encoder 204 belongs to the category of 3D autoencoders. A 3D autoencoder includes a 3D encoder 204 and a sub-embedding decoder 208. The 3D autoencoder can be trained to encode a 3D voxel model into embeddings and decode the embeddings into 2D contours, which can be used to reconstruct the 3D voxel model using a construction sequence (e.g., a parameterized construction sequence). Details of training the 3D autoencoder are combined below. Figure 5 and Figure 6 The following description is provided. A 3D encoder may include a 3D convolutional neural network (CNN), which includes one or more 3D convolutional layers of configurable size, one or more fully connected layers, one or more activation layers, or skip connections between layers, etc. A sub-embedding decoder may include a CNN, which includes one or more fully connected layers, one or more activation layers, or skip connections between layers, etc. In some embodiments, the sub-embedding decoder may include a multilayer perceptron (MLP), which includes a series of multilayer perceptron network layers.

[0050] In some implementations, when the target 3D object is a shape between two 3D objects, an initial input embedding 206 can be generated from the two embeddings of the two 3D objects. Program 116 can generate a first 3D object embedding from a first voxel model of the first 3D object. Program 116 can generate a second 3D object embedding from a second voxel model of the second 3D object. Program 116 can generate the initial input embedding 206 from the first and second 3D object embeddings, for example, by interpolating the first and second 3D object embeddings.

[0051] Refer again Figure 2B Program 116 uses the sub-embedding decoder of a 3D autoencoder to process the initial input embedding 234 to obtain a sub-embedding, and the sub-embedding encodes the 2D shape that defines the representation of the target 3D object. (See reference) Figure 2AThe sub-embedding decoder 208 receives the initial input embedding 206 as input. The sub-embedding decoder 208 can process the initial input embedding 206 to generate sub-embeddings 210(a) and 210(b). The sub-embeddings can represent a set of 2D shapes, such as shapes 214(a) and 214(b), which define a representation of the target 3D object. In some implementations, each sub-embedding encodes a corresponding 2D shape in a predetermined 2D orientation. For example, sub-embedding 210(a) encodes 2D shape 214(a) in the xy-plane, and sub-embedding 210(b) encodes 2D shape 214(b) in the xy-plane.

[0052] Refer again Figure 2B Procedure 116 determines 236 parametric sketch models. Procedure 116 determines a corresponding parametric sketch model for each of the sub-embedded elements. The corresponding parametric sketch model for each of the sub-embedded elements is a fitted 2D parametric sketch model. The 2D parametric sketch model is a 2D model of a 2D shape. The shape of the 2D parametric sketch model can be determined by one or more parameter values ​​of the 2D parametric sketch model. Since modeling a 3D object as a sketch can be difficult, the 2D portions of the 3D object can be modeled as 2D parametric sketch models that can be extruded into 3D space. In some implementations, procedure 116 processes each of the sub-embedded elements using one or more intermediate 2D decoders to obtain a 2D shape that defines a representation of the target 3D object. Procedure 116 generates each of the intermediate embeddings by processing each of the 2D shapes using a 2D encoder of a 2D autoencoder. Procedure 116 determines a corresponding fitted 2D parametric sketch model for each of the intermediate embeddings.

[0053] refer to Figure 2A A fitted 2D parametric sketch model can be determined for each sub-embedded. For example, a fitted 2D parametric sketch model 218(a) can be determined for sub-embedded 210(a), and a fitted 2D parametric sketch model 218(b) can be determined for sub-embedded 210(b). The trained intermediate 2D decoder 212 takes sub-embedded 210(a) or 210(b) as input and generates decoded 2D shapes 214(a) or 214(b). Procedure 116 performs a search and fitting step 219 to generate a fitted 2D parametric sketch model 218(a) or 218(b) from the decoded 2D shapes 214(a) or 214(b).

[0054] In some implementations, the search and fitting step 219 may include the following. The 2D encoder 260 of the trained 2D autoencoder can generate an intermediate embedding 254 of the decoded 2D shape (e.g., decoded 2D shape 214(a)). Procedure 116 may search for a 2D parametric sketch model 258 with intermediate embeddings 254 in a sketch model database 256, and the search is performed in the embedding space of the trained 2D autoencoder. Procedure 116 can fit the 2D parametric sketch model 258 to the decoded 2D shape 214(a) by modifying one or more parameter values ​​to produce a fitted 2D parametric sketch model 218(a). Details of the process for determining the fitted 2D parametric sketch model based on the embeddings will be provided below. Figure 3 The following description is provided. In some implementations, each of the 2D shapes 214(a) and 214(b) can be a probability indicating the likelihood that a pixel is inside the 2D shape (e.g., generated from a score via a sigmoid function), and procedure 116 can use a signed distance field function, such as... Figure 4 A signed distance field function 408 is used to generate a signed distance field image for each 2D shape. The signed distance field image of the 2D shape can be processed by a 2D encoder 260 to generate an embedding, such as an intermediate embedding 254 for the 2D shape 214(a).

[0055] Refer again Figure 2B Program 116 generates a set of 237 extrusion parameters from the sub-embedded components. Program 116 determines a corresponding set of extrusion parameters for each sub-embedded component. (See reference...) Figure 2A The envelope decoder can take sub-embeddings as input and generate an envelope function as output. Program 116 can determine the set of extrusion parameters based on the envelope function.

[0056] The envelope decoder is previously trained to generate the envelope function. The envelope decoder can be implemented using an MLP or one or more one-dimensional deconvolutional neural network layers. For example, envelope decoder 217(a) can take sub-embedding 210(a) as input and generate envelope function 216(a). Envelope decoder 217(b) can take sub-embedding 210(b) as input and generate envelope function 216(b). The envelope function includes a score indicating the probability of a start or end position. By passing the score to a sigmoid function, the score can be converted into a probability value. For example, the gray levels in envelope functions 216(a) and 216(b) are probability values ​​indicating the probability of a start or end position.

[0057] In some implementations, the envelope decoder may include a start envelope decoder and an end envelope decoder. The start envelope decoder takes the sub-embedding as input and generates a start envelope function as output, while the end envelope decoder takes the sub-embedding as input and generates an end envelope function as output. The start and end envelope functions may be two 1D vectors of floating-point numbers or other types of numerical values. The start envelope function can be used to determine the starting position of the extrusion, and the end envelope function can be used to determine the ending position of the extrusion. The start function can be positive when the position is below the starting plane of the extrusion, and negative when the position is above the starting plane. The end function can be negative when the position is below the ending plane of the extrusion, and positive when the position is above the ending plane.

[0058] The set of extrusion parameters may include the extrusion start position and extrusion end position of the corresponding fitted 2D parametric sketch model generated from the respective sub-embedding. This set of extrusion parameters can be used to extrude the fitted 2D parametric sketch models 218(a) and 218(b) into 3D space. The extrusion start position and extrusion end position can be determined based on an envelope function. For example, based on envelope function 216(a), program 116 can determine the extrusion start position 213(a) and extrusion end position 211(a) of the fitted parametric sketch model 218(a). Based on envelope function 216(b), program 116 can determine the extrusion start position 213(b) and extrusion end position 211(b) of the fitted 2D parametric sketch model 218(b).

[0059] In some implementations, procedure 116 can determine the extrusion start position based on the start envelope function and the extrusion end position based on the end envelope function. In some implementations, the extrusion start position can be at a location where the start envelope function crosses zero, and the extrusion end position can be at a location where the end envelope function crosses zero. In some implementations, the start and end extrusion positions can be positions with a probability exceeding a predetermined threshold (e.g., 50%). If more than one position exceeds the predetermined threshold, the start position can be at the first intersection of the start envelope function, and the end position can be at the last intersection of the end envelope function. The extrusion direction can be predetermined or hard-coded. In this example, for both fitted 2D parametric sketch models 218(a) and 218(b), the extrusion direction is the z-direction. However, it should be noted that in some implementations, the orientation of the extrusion direction can be another extrusion parameter among the extrusion parameters.

[0060] In some implementations, an extrusion can begin from the end of a previous extrusion. Therefore, instead of using a start envelope decoder for the first 2D parametric sketch model and an end envelope decoder for the second 2D parametric sketch model, a single envelope decoder can be used to generate a start-end envelope function that defines a shared plane / position. For example, the end envelope function (e.g., used to determine the end position 211(b) of the lower extrusion (e.g., of 2D parametric sketch model 218(b)) can be (-1) times the start envelope function (e.g., used to determine the start position 213(a) of the upper extrusion (e.g., of 2D parametric sketch model 218(a))). This shared envelope function can be decoded by the same envelope decoder and can be shared by both extrusions.

[0061] Refer again Figure 2B For example, program 116 can provide the user with a fitted 2D parametric sketch model generated by a machine learning algorithm to determine whether design 238 is acceptable for the target 3D object. Program 116 may include UI elements in UI 122 that allow user 190 to accept or reject the fitted 2D parametric sketch model generated by the machine learning algorithm. In some implementations, program 116 uses the fitted 2D parametric sketch model to generate a 240 3D B-Rep model, and the 3D B-Rep model is displayed to the user, allowing the user to decide whether to accept or reject the 3D B-Rep model.

[0062] If the user determines that the generated fitted 2D parametric sketch model is unacceptable for constructing the target 3D object, program 116 can use a machine learning algorithm to generate an updated fitted 2D parametric sketch model. In some implementations, program 116 may include a UI element in UI 122 that allows user 190 to specify another set of extrusion directions. The machine learning algorithm can generate a fitted 2D parametric sketch model with another set of extrusion directions.

[0063] Once the user determines that the fitted 2D parametric sketch model generated by 238 is acceptable for constructing the target 3D object, program 116 generates a 3D boundary representation (B-Rep) model of the target 3D object by extrudement into 3D space using the fitted 2D parametric sketch model in a construction sequence (e.g., a parametric construction sequence). The construction sequence includes the set of extrusion parameters (e.g., start and end extrusion positions), a predetermined extrusion direction, and predetermined Boolean operations defined in the CAD Boolean engine 222.

[0064] refer to Figure 2AProcedure 116 can extrude each of the fitted 2D parametric sketch models 218(a) or 218(b) into 3D. Procedure 116 may include a CAD extrusion engine. The CAD extrusion engine can generate an extruded parametric sketch model from each fitted 2D parametric sketch model at a start extrusion position and an end extrusion position included in the set of extrusion parameters, and in a predetermined extrusion direction included in the build sequence. For example, the CAD extrusion engine can use the set of extrusion parameters, such as an extrusion start position 213(a) and an extrusion end position 211(a), to generate a first extruded parametric sketch model from the fitted 2D parametric sketch model 218(a). The CAD extrusion engine can use the set of extrusion parameters, such as an extrusion start position 213(b) and an extrusion end position 211(b), to generate a second extruded parametric sketch model from the fitted 2D parametric sketch model 218(b). Procedure 116 may include a CAD Boolean engine 222. The CAD Boolean engine 222 is non-differentiable. The CAD Boolean engine 222 can receive extruded parametric sketch models (e.g., a first extruded parametric sketch model generated from 218(a) and a second extruded parametric sketch model generated from 218(b)) as input, and can generate a 3D boundary representation (B-Rep) model 220 through a set of predetermined (or hard-coded) Boolean operations. The CAD Boolean engine 222 defines the set of predetermined (or hard-coded) Boolean operations, such as union, intersection, complement, or combinations thereof.

[0065] For example, a fitted 2D parametric sketch model 218(a) is extruded into 3D in the z-direction between the starting and ending extrusion positions determined by the envelope function 216(a), and a fitted 2D parametric sketch model 218(b) is extruded into 3D in the z-direction between the starting and ending extrusion positions determined by the envelope function 216(b). The CAD Boolean engine 222 may include predetermined union operations. Therefore, two 3D parametric sketch models extruded from the fitted 2D parametric sketch models 218(a) and 218(b) can be combined together via a union operation. In some cases, the CAD Boolean engine 222 may define an operation that is the product of an intersection operation and a complement operation (e.g., -1), which results in one 3D parametric sketch model being subtracted from another 3D parametric sketch model. In some cases, the CAD Boolean engine 222 may define a sequence of union operations, intersection operations, complement operations, or combinations thereof.

[0066] Program 116 can define a set of connected surface elements that specify the boundaries between solid and non-solid portions of the 3D B-Rep model 220. Therefore, the geometry of the 3D B-Rep model 220 can be stored in the computer using smooth and accurate mathematical surfaces. Program 116 can store the 3D B-Rep model 220 in a local computer or send it to a remote computer. Program 116 can display the 3D B-Rep model 220 in the UI 122 of the display device 120 of computer 110. Program 116 can generate toolpaths 165 for the 3D B-Rep model 160 or 220. Toolpaths 165 can be used by SM and / or AM machines 170 to manufacture a target 3D object. In some embodiments, program 116 can generate a 3D voxel model of the target 3D object from the 3D B-Rep model 220.

[0067] In some implementations, procedure 116 can use procedure 200 to undo morphological modifications to the 3D voxel model. Procedure 116 can obtain an initial voxel model of the target 3D object. Procedure 116 can process the initial voxel model by morphological modifications to generate a modified voxel model 202. Procedure 116 can process the modified voxel model 202 using a 3D encoder 204 to generate an initial input embedding 206. Procedure 116 can generate sub-embeddednesses 210(a) and 210(b) from the initial input embedding 206 of the modified voxel model 202. Procedure 116 can determine fitted 2D parametric sketch models 218(a) and 218(b) based on the sub-embeddednesses. Procedure 116 can use a construction sequence to generate a reconstructed 3D B-Rep model 220 from the fitted 2D parametric sketch models 218(a) and 218(b) by extruding them into 3D space. The reconstructed 3D B-Rep model 220 can resemble the initial voxel model, thereby undoing the morphological modifications.

[0068] In some implementations, program 116 can generate a prism CAD model from the output of a generative design process. Program 116 can obtain an initial voxel model 202 of the target 3D object, and the initial voxel model can be a generative design output. Program 116 can generate an initial input embedding 206 by processing the initial voxel model 202 using a 3D encoder 204. Program 116 can generate a 3D prism model of the target 3D object, wherein the 3D prism model is a 3D B-Rep model 220.

[0069] In some implementations, the 3D B-Rep model 220 can be post-processed by procedure 116 to increase its similarity to the initial 3D voxel model 202 of the target 3D object. Procedure 116 can obtain the initial 3D voxel model 202 of the target 3D object, which is generated from the output of a generative design. Procedure 116 can generate an initial input embedding 206 by processing the initial voxel model 202 using a 3D encoder 204. After obtaining the 3D B-Rep model 220, procedure 116 can perform post-processing on the 3D B-Rep model 220. Procedure 116 can fit the 3D B-Rep model 220 to the initial voxel model 202 of the target 3D object by changing one or more parameters of the 3D B-Rep model 220 to produce a fitted 3D B-Rep model. For example, rounded corners can be added to the 3D B-Rep model 220 to make the edges of the 3D B-Rep model 220 rounded. In some implementations, when the initial 3D voxel model 202 is unavailable, program 116 may use a 3D deconvolution voxel decoder (e.g., combined with...). Figure 7 The described 3D deconvolution voxel decoder 703 generates a decoded 3D voxel model from the input embedding 206, and the procedure 116 can perform rounded corner fitting on the 3D B-Rep model 220 based on the decoded 3D voxel model.

[0070] In some implementations, after generating fitted 2D parametric sketch models 218(a) and 218(b), program 116 can fit fillet arcs to the fitted 2D parametric sketch models 218(a) and 218(b) to better approximate the shapes of the 2D contours 214(a) and 214(b) decoded by the intermediate 2D decoder 212. Therefore, sketches in the sketch model database 256 can be defined primarily using lines, and it is easier to perform fitting 259 using a constraint solver for sketches defined using lines. The fillet arc radius can be obtained from the signed distance function by evaluating it at the vertex where the fillet is to be cut. If the 2D shapes 214(a) and 214(b) are not generated using the signed distance function, signed distance values ​​can be generated from the 2D shapes 214(a) and 214(b) using a fast-step method.

[0071] In some implementations, the parameters of the 2D parametric sketch model 258 can be fitted to better approximate the entire target voxel model 202 or the 3D voxel model generated by the 3D deconvolution voxel decoder. Procedure 116 can use an optimization algorithm that does not require derivatives. In some implementations, the set of extrusion parameters (e.g., extrusion start position and extrusion end position) can be fitted to better approximate the entire target voxel model 202 or the 3D voxel model generated by the 3D deconvolution voxel decoder.

[0072] In some implementations, a signed distance function can be used to determine the corner radius by evaluating the average value of the signed distance function along each edge and then constructing a rounded corner with that average value as the radius. This eliminates the need for optimization algorithms to create the rounded corners. If the available voxel model is not a signed distance function (e.g., when the voxel model is the output of a 3D deconvolutional voxel decoder), a fast marching method can be used to create the signed distance function.

[0073] Figure 3 The flowchart includes an example of a process that demonstrates the generation of a fitted 2D parametric sketch model of a target 2D shape using a machine learning algorithm. Given embedding vectors, the machine learning algorithm can find a 2D shape that represents the embedding vectors and resembles a human design (such as a sketch model previously designed by a human user).

[0074] Procedure 116 obtains 322 input embeddings that encode a representation of the target 2D shape. For example, procedure 116 may receive input embedding 302 that encodes a representation of the target 2D shape. The exact model of the target 2D shape is unknown, and only certain aspects or desired features of the target 2D shape are known. Therefore, input embedding 302 may represent the desired features of the target 2D shape. For example, input embedding 302 may be generated by encoding an initial design of the target 2D shape using a 2D encoder. As another example, the target 2D shape may be a shape associated with two 2D shapes, and input embedding 302 may be an interpolation of two embeddings of the two 2D shapes. In some 2D implementations, input embedding 302 may be an embedding of a 2D shape. In some 3D implementations, input embedding 302 may be each of the corresponding sub-embedded embeddings 210(a) and 210(b) generated from a 3D shape (e.g., a 3D voxel model 202).

[0075] The input embedding is processed 324 by the 2D decoder of the 2D autoencoder by procedure 116 to obtain a decoded representation of the target 2D shape. The 2D decoder 304 is previously trained and can decode the input embedding 302 to generate a 2D image 306, which is a decoded representation of the target 2D shape. The 2D decoder 304 belongs to a 2D autoencoder that includes a 2D encoder and a 2D decoder 304. The 2D autoencoder includes a 2D encoder that processes the representation of a 2D object to generate an object embedding, and a 2D decoder 304 that processes the object embedding to generate a decoded representation of the 2D object. Details of the process of training the 2D autoencoder are combined below. Figure 4 Describe it.

[0076] The 2D encoder may include a 2D convolutional neural network (CNN) comprising one or more convolutional layers of configurable size, one or more fully connected layers, one or more activation layers, or skip connections between layers. The 2D decoder 304 may include a CNN comprising one or more deconvolutional layers of configurable size, one or more transposed convolutional layers of configurable size, one or more fully connected layers, one or more activation layers, or skip connections between layers. The 2D decoder generates a 2D image 306 that approximates the shape of the target 2D shape. The value of each pixel in the 2D image is a score indicating the likelihood or probability that the pixel is inside the approximate shape. In some embodiments, the architecture of the 2D decoder 304 may be mirrored that of the 2D encoder. For example, if the 2D encoder includes three convolutional layers with embeddings of sizes 32, 64, and 512, the 2D decoder may include three deconvolutional layers receiving embeddings of sizes 512, 64, and 32.

[0077] Procedure 116 determines the fitted 2D parametric sketch model of the input embedding by performing operations 326 and 328. Procedure 116 uses a search in the embedding space of the 2D autoencoder and a sketch model database associated with the 2D autoencoder to find the 2D parametric sketch model of the input embedding 326. The shape of the 2D parametric sketch model (e.g., including translations and uniform scaling factors in the x and y directions) is determined by one or more parameter values ​​of the 2D parametric sketch model. Procedure 116 performs a search 310 in the embedding space of the 2D autoencoder using the sketch model database 308. The sketch model database includes embeddings previously calculated for parametric variations of multiple 2D parametric sketch models.

[0078] To generate a sketch model database, program 116 obtains multiple 2D parametric sketch models. Each 2D parametric sketch model has constraints controlling its dimensions and rough shape. The details of the shape of each 2D parametric sketch model are flexible and can be determined by one or more parameter values ​​of the 2D parametric sketch model. Program 116 changes one or more parameter values ​​of each 2D parametric sketch model, thereby creating numerous geometric shapes or parametric variations for the 2D parametric sketch models. In some implementations, the parametric variations of the 2D parametric sketch models can be mirrored, rotated, and / or translated. Each parametric variation of each of the 2D parametric sketch models can be encoded using a 2D encoder of a 2D autoencoder to obtain an embedding. The embeddings of the different parametric variations of the multiple 2D parametric sketch models are stored in sketch model database 308.

[0079] Program 116 can use a search in database 308 to find the 2D parametric sketch model 312 of the input embedding 302. In some implementations, program 116 can search in the embedding space of the 2D autoencoder for the embedding closest to the input embedding 302 stored in database 308, and program 116 can find the 2D parametric sketch model 312 corresponding to the closest embedding.

[0080] In some implementations, program 116 may use a transformer generator to perform the search. Examples of using a transformer generator to perform a search can be found in DeepCAD (Wu, Rundi, Chang Xiao and Changxi Zheng. "Deepcad: A deep generative network for computer-aided design models." IEEE / CVF International Conference on Computer Vision. 2021). Other examples include models described in the following literature: Engineering Sketch Generation for Computer-Aided Design, Karl DD Willis and Pradeep Kumar Jayaraman and Joseph G. Lambourne and Hang Chu and Yewen Pu, The 1st Workshop on Sketch-Oriented Deep Learning (SketchDL), CVPR 2021; Ganin, Yaroslav et al., "Computer-aided design as language." Advances in Neural Information Processing Systems 34 (2021); Para, Wamiq et al., "Sketchgen: Generating constrained cad sketches." Advances in Neural Information Processing Systems (Advances in Neural Information Processing Systems) 34 (2021); Seff, Ari et al., "Vitruvion: A Generative Model of Parametric CAD Sketches." arXiv preprint arXiv:2109.14124 (2021).

[0081] A 2D parametric sketch model is fitted 328 to a decoded representation of a target 2D shape by program 116 by modifying one or more parameter values ​​of the 2D parametric sketch model to produce a fitted 2D parametric sketch model. A 2D parametric sketch model retrieved from a search in database 308 can be further refined by modifying one or more parameter values ​​of the 2D parametric sketch model. Because database 308 stores a finite number of parameter variations of the 2D parametric sketch model 312, program 116 can perform a fitting 314 from the 2D parametric sketch model 312 to a 2D image 306. Program 116 can execute an optimization algorithm (e.g., a derivative-free multidimensional optimization algorithm) to modify one or more parameter values ​​of the 2D parametric sketch model 312 to obtain a fitted 2D parametric sketch model 316. Examples of optimization algorithms that can be applied here include simplex optimization (i.e., the Nelder-Mead method), Bayesian optimization, (adaptive) coordinate descent, cuckoo search, etc. Figure 3 As shown, compared to the 2D parametric sketch model 312, the diameter of the circle and the length and / or width of the rectangle of the fitted 2D parametric sketch model 316 are adjusted. Therefore, the fitted 2D parametric sketch model 316 more closely matches the shape of the 2D image 306.

[0082] In some implementations, in addition to fitting the intrinsic parameters of the parametric sketch model, procedure 116 may fit parameters defining the position of the parametric sketch model (e.g., translations in the x and y directions) and the scale of the parametric sketch model (e.g., a uniform scaling factor). In some implementations, the cost function used during fitting 314 may be based on the probability predicted by the 2D decoder 304 that each pixel is inside the contour. Let Pr(x,y) be the probability that a pixel at a position defined by integers x and y is inside the contour. Let Inside(x,y) be a value of +1 if the pixel is inside the contour and -1 if the pixel is outside the contour. The cost function used during fitting 314 may be...

[0083] cost = ∑ x，y (0.5-Pr(x,y))*Inside(x,y). (1)

[0084] The sum here is the sum of all pixels in the 2D outline image.

[0085] A fitted 2D parametric sketch model 316 is used in a computer modeling program (such as a CAD program). The program 116 can display the fitted 2D parametric sketch model in the user interface (UI) of the computer modeling program. For example, the fitted 2D parametric sketch model 316 can be displayed in a UI 320 on a computer display device 318, which can be operated using one or more input devices of the computer (e.g., a keyboard and mouse). A user of the computer modeling program can further modify the fitted 2D parametric sketch model 316. The user can incorporate the fitted 2D parametric sketch model 316 into a 3D B-Rep model being created. For example, the user can use CAD tools to extrude the fitted 2D parametric sketch model 316 into 3D space. In some embodiments, the process of generating the fitted 2D parametric sketch model 316 can be part of the automatic generation of the 3D B-Rep model. For example, input embedding 302 can be... Figure 2A The corresponding sub-embeddings 210(a) and 210(b) are, and the fitted 2D parametric sketch model 316 can be the corresponding fitted 2D parametric sketch models 218(a) and 218(b).

[0086] Figure 4 This includes a flowchart illustrating an example of the process of training a 2D autoencoder that can be used to determine the fit of a 2D parametric sketch model to a target 2D shape. Parametric instances of 2D parametric sketch models are obtained through procedure 116. Procedure 116 obtains 2D parametric sketch models (e.g., contours), and the shape of each 2D parametric sketch model is determined by one or more parameter values ​​of the 2D parametric sketch model. A parametric instance of a 2D parametric sketch model is a 2D shape of a 2D parametric sketch model having one or more defined parameter values. Procedure 116 can change one or more parameter values ​​of the 2D parametric sketch model and obtain multiple parametric instances of the 2D parametric sketch model. For example, 2D shape 402 is a parametric instance of a 2D parametric sketch model, for example, having one parameter value defining the size of a circular portion of the 2D parametric sketch model and two other parameter values ​​defining the width and height of a rectangular portion of the 2D parametric sketch model.

[0087] Procedure 116 generates 424 2D training images from a parameterized instance of the 2D parameterized sketch model. Each of the 2D training images corresponds to a parameterized instance of the 2D parameterized sketch model. For example, procedure 116 can generate a binary mask 406 for a parameterized instance 402 of the 2D parameterized sketch model, and binary mask 406 is one of the 2D training images that can be used to train a 2D autoencoder.

[0088] Following operations 426, 428, 430, and 432, procedure 116 trains a 2D autoencoder on 2D training images. The 2D autoencoder includes a 2D encoder 412 and a 2D decoder 416. For each of the 2D training images, procedure 116 processes 426 of the 2D training images using the 2D encoder to generate embeddings. During training, the 2D encoder 412 can receive a 2D training image 406 as input and can generate an embedding 414 for the 2D training image 406. The embedding 414 can be a one-dimensional embedding vector, for example, having a length of 512, 1024, or 2048 bits, bytes, words, or double words.

[0089] In some implementations, the signed distance field image is generated from a 2D training image, and the signed distance field image is processed using a 2D encoder to generate an embedding. Procedure 116 can process the 2D training image 406 using a signed distance function (SDF) 408 to obtain a signed distance field image 410 that provides an improved description of the 2D shape. An SDF is a function that takes a location as input and outputs the distance from that location to the nearest portion of the shape. The signed distance field image 410 can include pixel values ​​as floating-point numbers, rather than binary values ​​such as those in the 2D training image 406. Because a binary image only describes the shape at the boundary between black and white pixels, while a signed distance function can spread the signal across the entire image or pixel region, the signed distance field image 410 can provide a better description of the 2D shape 402 than the binary 2D training image 406, thereby improving the performance of the 2D autoencoder.

[0090] For example, pixels in a binary image only provide information about the shape's outline, but pixels in a signed distance field image can provide information about the pixel's location and the shape's thickness at that pixel. Because CNNs (such as 2D autoencoders) can be translation-invariant, the same shape at different locations in an image can produce the same signature for a binary image, but the same shape at different locations in an image can produce different signatures for a signed distance field image, thus providing a better description of the 2D shape. For example, the embedding 414 generated from the signed distance field image 410 can describe both the 2D shape 402 and the relative positions of pixels in the 2D training image 406. Experimental results demonstrate the performance improvement of using signed distance field images for 2D autoencoders.

[0091] Program 116 uses a 2D decoder to process the embedding 428 to generate a decoded 2D image. The 2D decoder 416 can generate a decoded 2D image 418, and each pixel of the decoded 2D image 418 can be a score, such as a floating-point number or other type of numerical value, indicating the probability that the pixel is inside the 2D shape. In some embodiments, program 116 can pass the pixel values ​​of the decoded 2D image 418 through a sigmoid function to obtain the probability that the pixel is inside the 2D shape. The sigmoid function converts negative numbers to probabilities below a threshold (e.g., 50%) and positive numbers to probabilities above a threshold (e.g., 50%). In some embodiments, a signed function can be applied to the decoded 2D image 418 before applying the sigmoid function, and the signed function can be negative inside the 2D shape and positive outside the 2D shape.

[0092] Procedure 116 computes the value of loss function 430 by comparing each of the 2D training images with its corresponding decoded 2D image. Procedure 116 can use loss function 420 to compare each 2D training image 406 with its corresponding decoded 2D image 418. Loss functions can compare each predicted probability with the actual class indicated in the 2D training image. Examples of loss functions include (binary) cross-entropy loss, mean squared error, Huber loss, and hinge loss.

[0093] The parameters of the 2D encoder and 2D decoder are updated by procedure 116 based on the value of the loss function 432. Procedure 116 can compute the value of the loss function for a batch of 2D training images selected from the 2D training images. Procedure 116 can use an optimizer (such as stochastic gradient descent) to update the parameters of the 2D autoencoder, such as the parameters of the 2D encoder and 2D decoder, through an iterative optimization process. Procedure 116 can iteratively update the parameters of the 2D autoencoder on batches of 2D training images.

[0094] The convergence of the optimization (e.g., the completion of training a 2D autoencoder) can be checked 434. In some implementations, convergence can be determined if the parameter update is less than a threshold. In some implementations, convergence occurs when the update has reached the accuracy limit supported by the computer or CAD program. In some implementations, optimization converges after a fixed number of iterations have been performed.

[0095] After training is complete, program 116 can use a 436 2D autoencoder. Program 116 can use a 2D encoder 412 to create an embedding database for parameter variations of multiple parametric sketch models. For example, the 2D encoder 412 can be used to generate embeddings stored in... Figure 3 Sketch model database 308 and / or Figure 1 The embeddings in the sketch database 130 are, in some embodiments, the two databases can be the same. Program 116 can use the signed distance field function 408 to convert each parametric variation of the parametric sketch model into a signed distance field image. Program 116 can then use the 2D encoder 412 to generate embeddings of the parametric variations of the parametric sketch model. The embeddings and mappings under the parametric variations of the parametric sketch model can be stored in the sketch model database 308 and / or the sketch database 130.

[0096] Procedure 116 can use the trained 2D decoder 416 and the trained 2D encoder 412 in process 200 to generate a 3D model of the target 3D object. Specifically, the trained 2D decoder 416 can be... Figure 2A The intermediate 2D decoder 212 or a portion thereof, and the trained 2D decoder can be used to determine fitted 2D parametric sketch models 218(a) and 218(b) for sub-embeddednesses 210(a) and 210(b). The trained 2D encoder 412 can be Figure 2A The 2D encoder 260 or a portion thereof. Procedure 116 may also use a trained 2D decoder 416 in process 500 to train a 3D autoencoder that can be used to generate a 3D model of the target 3D object.

[0097] Figure 5 An example of the process 500 for training a 3D autoencoder that can be used to generate a 3D model of a target 3D object is shown. Figure 6 This is a flowchart illustrating an example of a process for training a 3D autoencoder that can be used to generate a 3D model of a target 3D object. The 3D autoencoder includes: a 3D encoder 508 that processes an input voxel model to generate a 3D object embedding 510; a sub-embedding decoder 512 that processes the 3D object embedding 510 to generate sub-embeddings (e.g., 514(a), 514(b), and 514(c)); an envelope decoder 517 that processes each of the sub-embeddings to generate a corresponding envelope function; and a differentiable distributed engine that generates a reconstructed voxel model by extending a 2D shape into 3D space using the envelope function. In some embodiments, the envelope decoder 517 may include: a start envelope decoder that processes each of the sub-embeddings to generate a start envelope function, and a finish envelope decoder that processes each of the sub-embeddings to generate a finish envelope function. A set of extrusion parameters (e.g., Figure 2A The starting and ending extrusion positions can be generated from the starting and ending envelope functions. For example, the set of extrusion parameters can be... Figure 2A The extrusion parameters 211 and 213 are used to extrude a 2D parametric sketch model into a 3D B-Rep model 220.

[0098] Refer again Figure 6 602 training examples are obtained through procedure 116. Each training example includes a training voxel model and a ground reality voxel model. The training voxel model is generated from the ground reality voxel model. When historical information for generating the ground reality voxel model is available, each training example may also include a ground reality 2D shape and a set of ground reality extension parameters (e.g., the starting and ending planes for each extrusion) that define the ground reality envelope function. The ground reality voxel model is generated by extending the ground reality 2D shape into 3D space using the set of ground reality extension parameters. When historical information for generating the ground reality voxel model is not available, the ground reality 2D shape and the set of ground reality extension parameters are not included in the training examples. In some implementations, the 3D autoencoder can be trained directly on the ground reality voxel model. Therefore, each training example includes a ground reality voxel model, and procedure 116 does not need to generate the training voxel model from the ground reality voxel model.

[0099] For example, refer to Figure 5 Procedure 116 can obtain multiple training examples, and each training example can include a training voxel model 506, a ground reality voxel model 502, and a ground reality 2D shape 528, including three 2D shapes 528(a), 528(b), and 528(c). To generate multiple training examples, procedure 116 can obtain multiple ground reality voxel models. Procedure 116 can generate one or more training voxel models 506 from each ground reality voxel model 502 through data augmentation. For example, procedure 116 can perform various morphological modifications 504 on the ground reality voxel 502 to obtain multiple training voxel models, such as training voxel models 506 with speckled or blurred shapes and additional holes 507. Through morphological operations, training voxel models 506 can have rounded edges and can have shapes similar to various models created through topology optimization or generative design. In some implementations, the training voxel model can be a training signed distance field voxel model generated in a 3D mesh using a 3D signed distance function.

[0100] Using a ground reality voxel model 502 as the ground reality, a 3D autoencoder can be trained using supervised learning, such as a first loss function 524. For example, a 3D autoencoder can be trained to undo morphological modifications 504 by generating a 3D voxel model 522 from a blob or blurred input voxel model 506. Thus, a 3D autoencoder can be trained to generate a 3D prism B-Rep CAD model from a model created by topology optimization or generative design.

[0101] In some implementations, when historical information for generating ground reality voxel models is available, each ground reality voxel model 502 can be defined by a ground reality 2D shape 528. That is, the ground reality voxel model 502 is generated by extending the ground reality 2D shape 528 using a set of ground reality extension parameters that define a ground reality envelope function. The ground reality envelope function can be the maximum or intersection of a ground reality start envelope function and a ground reality end envelope function. The ground reality start envelope function and the ground reality end envelope function can be two 1D vectors of binary values. Therefore, when the ground reality 2D shape and the corresponding ground reality start and end envelope functions are available to procedure 116, the ground reality 2D shape and the corresponding ground reality extension parameters can be used as additional ground reality labels to train a 3D autoencoder using supervised learning (e.g., using a second loss function 526 in addition to the first loss function 524). Specifically, the ground-based start and end envelope functions can be used to train the envelope decoder 517 using supervised learning (e.g., including the start envelope decoder and the end envelope decoder).

[0102] For example, 2D shapes 528(a), 528(b), and 528(c) are located in the xy plane, xy plane, and yz plane, respectively. The ground reality voxel model 502 can be generated by generating three intermediate 3D shapes in the following manner: extending 2D shape 528(a) in the z-direction at a position defined by the ground reality start and end envelope functions of 2D shape 528(a); extending 2D shape 528(b) in the z-direction at a position defined by the ground reality start and end envelope functions of 2D shape 528(b); and extending 2D shape 528(c) in the x-direction at a position defined by the ground reality start and end envelope functions of 2D shape 528(b). The three intermediate 3D shapes can be combined using union (e.g., minimum operation), intersection (e.g., maximum operation), and complement (e.g., negative 1). For example, the ground reality voxel model 502 can be generated by taking the union of the 3D shapes corresponding to the 2D shapes 528(a) and 528(b), and then subtracting the 3D shape corresponding to the 2D shape 528(c), for example, by intersection and complement operations.

[0103] Refer again Figure 6 The 3D autoencoder follows operations 604, 606, 608, 610, 612, 614, and 616 to train on the training examples. For each of the training examples, procedure 116 processes the 604 training voxel model using the 3D encoder of the 3D autoencoder to generate a 3D object embedding of the training voxel model. (See reference...) Figure 5 The 3D encoder 508 receives the training voxel model 506 as input and generates a 3D object embedding 510 of the training voxel model 506. The 3D object embedding 510 can be a one-dimensional vector with a predetermined length (e.g., 512, 1024, or 2048 bits, bytes, words, or double words).

[0104] Refer again Figure 6 Program 116 uses the sub-embedding decoder of a 3D autoencoder to process 606 3D object embeddings to generate sub-embeddings. (See reference) Figure 5 The sub-embedding decoder 512 receives the 3D object embedding 510 as input and generates sub-embeddings 514(a), 514(b), and 514(c). The number of sub-embeddings and the orientation of the 2D shapes corresponding to the sub-embeddings can be predetermined by the sub-embedding decoder. For example, the sub-embedding decoder 512 can be configured to generate three sub-embeddings corresponding to shapes in the yz plane (e.g., 518(a)), xy plane (e.g., 518(b)), and xy plane (e.g., 518(c)).

[0105] Refer again Figure 6 Program 116 processes each of the 608 sub-embedded elements to generate a 2D shape and start and end envelope functions. Program 116 uses a 2D decoder 516 to process each sub-embedded element to generate a 2D shape. Program 116 uses an envelope decoder 517 to process each sub-embedded element to generate an envelope function for the 2D shape. In some embodiments, the envelope decoder 517 includes a start envelope decoder and an end envelope decoder. Program 116 may use the start envelope decoder to process each sub-embedded element to generate a start envelope function. Program 116 may use the end envelope decoder to process each sub-embedded element to generate an end envelope function.

[0106] refer to Figure 5 The 2D decoder 516 decodes each sub-embedded element to generate the corresponding 2D shape 518. For example, the 2D decoder 516 decodes sub-embedded element 514(a) to generate 2D shape 518(a). The 2D decoder 516 decodes sub-embedded element 514(b) to generate 2D shape 518(b). The 2D decoder 516 decodes sub-embedded element 514(c) to generate 2D shape 518(c).

[0107] In some implementations, the 2D decoder 516 may be the same intermediate 2D decoder 212 used to determine the fitted 2D parametric sketch model at inference time, and the 2D decoder may belong to a previously implemented model by combining... Figure 4 The described process trains a 2D autoencoder. Procedure 116 can freeze the parameters of the trained 2D decoder during the training of the 3D autoencoder. That is, the parameters of the trained 2D decoder are not updated during the training of the 3D autoencoder. Therefore, the 3D autoencoder can be trained such that the sub-embedded embeddings 514(a), 514(b), and 514(c) generated from the sub-embedded decoder 512 are in the same embedding space of the intermediate 2D decoder 212 used during inference. Therefore, during inference, the reference... Figure 2A In process 200, the trained intermediate 2D decoder 212 can determine the fitted 2D parametric sketch models 218(a) and 218(b) from the sub-embedded models 210(a) and 210(b) generated by the sub-embedded decoder 208. Figure 2A The intermediate 2D decoder 212, Figure 5 2D decoder 516 Figure 3 The 2D decoder 304 can be used Figure 4 The same 2D decoder is trained using the process described in [the document].

[0108] In some implementations, the 2D decoder 516 can be different from the combination Figure 4 The 2D decoder 416 of the 2D autoencoder described herein can be used to determine a fitted 2D parametric sketch model for the input embedding. That is, the 2D decoder 516 has not yet been trained to determine a fitted 2D parametric sketch model for the input embedding. In some cases, the 2D decoder 516 may have the same architecture as the 2D decoder 416 but may have different parameter values. In some cases, the 2D decoder 516 may have a different architecture than the 2D decoder 416. The parameters of the 2D decoder 516 can be trained together with the parameters of the 3D autoencoder. Therefore, the sub-embeddednesses 514(a), 514(b), and 514(c) directly generated from the sub-embeddedness decoder 512 are in different... Figure 2A The intermediate 2D decoder 212 defines the embedding space in the embedding space.

[0109] Therefore, during reasoning, reference Figure 2AThe process 200 may use additional processing, as shown in box 219. Procedure 116 may process the decoded 2D shape 214(a) using a 2D encoder 260, which is identical to the 2D encoder 412 of the trained 2D autoencoder used to generate embeddings of sketch models in the sketch model database 256. Therefore, procedure 116 can search the embedding space of the trained 2D autoencoder for the 2D parametric sketch model 258 with intermediate embedding 254.

[0110] Return to reference Figure 5 In some embodiments, envelope decoder 517 generates an envelope function from each sub-embedding. In some embodiments, envelope decoder 517 may include a start envelope decoder and an end envelope decoder. The start envelope decoder can generate a start envelope function from the sub-embedding. The end envelope decoder can generate an end envelope function from the sub-embedding. The start function and the envelope function can determine the envelope function. In some embodiments, the maximum value or intersection of the start envelope function and the end envelope function can be the envelope function. The envelope function can be used to generate envelope extrusion of 2D shapes (e.g., 521(a), 521(b), or 521(c)).

[0111] Refer again Figure 6 Procedure 116 generates a reconstructed voxel model of 610 trained voxel models by constructing a reconstructed voxel model using 2D shapes extended in 3D space based on a predicted construction sequence. (See reference) Figure 5 Program 116 uses the predicted construction sequence to generate a reconstructed voxel model 522 from the 2D shape 518. The predicted construction sequence includes an envelope function, operations defined in the differentiable distribution engine 520, and predefined expansion directions.

[0112] In some implementations, procedure 116 may include a differential expansion engine 519. The differential expansion engine 519 can generate three intermediate 3D shapes by expanding each 2D shape 518(a), 518(b), or 518(c) in a predetermined expansion direction. For example, the differentiable 2D shapes 518(a), 518(b), or 518(c) can be copied along a predetermined or hard-coded axis of the 3D mesh. Procedure 116 can process each of the intermediate 3D shapes using a corresponding envelope function to generate an envelope expansion. For example, the maximum value or intersection of the envelope function and the intermediate 3D shapes can be the envelope expansion. The envelope function can be applied to the expanded 2D shapes orthogonal to the predetermined or hard-coded expansion axis, using the maximum function to define the intersection. 3D shape 521(a) is an envelope extension of 2D shape 518(a), and 3D shape 521(b) is an envelope extension of 2D shape 518(b), and 3D shape 521(c) is an envelope extension of 2D shape 518(c). The differentiable distributed engine 520 can process the envelope extensions to generate a reconstructed 3D voxel model 522. For example, envelope extensions 521(a), 521(b), and 521(c) can be combined using union, intersection, complement, or combinations thereof operations defined in the differentiable distributed engine 520.

[0113] Refer again Figure 6 Procedure 116 calculates the value of the first loss function 612 by comparing each trained voxel model with its corresponding reconstructed voxel model. (See reference) Figure 5 Procedure 116 can use a first loss function 524 to measure the difference between the reconstructed voxel model 522 and the ground reality voxel model 502. Examples of the first loss function include (binary) cross-entropy loss, mean squared error, Huber loss, and hinge loss. The first loss function allows training machine learning models without requiring training examples from sketch-based history.

[0114] In some implementations, the value of the second loss function 614 is calculated by comparing the ground-based 2D shape with the 2D shape and by comparing the start and end envelope functions with the ground-based envelope function. Procedure 116 can use the second loss function 526 to measure the difference between the 2D shapes 518(a), 518(b), and 518(c) and the ground-based 2D shape 528. Examples of the second loss function include cross-entropy loss, mean squared error, Huber loss, and hinge loss. In some implementations, procedure 116 can also use the second loss function 526 to measure the difference between the start and end envelope functions and the ground-based envelope function. The second loss function allows the 3D autoencoder to learn to decompose the 3D shape into CAD-like 2D contours. Therefore, with the second loss function supervised, the 3D autoencoder can be trained to generate 2D contours or shapes that can be used for CAD extrusion.

[0115] Refer again Figure 6 Procedure 116 updates the parameters of the 3D autoencoder based at least on the value of the first loss function. In some implementations, the parameters of the 616 3D autoencoder are updated based at least on the values ​​of the first and second loss functions. (See reference...) Figure 5 The parameters of the 3D encoder 508, sub-embedded decoder 512, and envelope decoder 517 can be updated based on the value of the first loss function 524. In some embodiments, the parameters of the 3D encoder 508 and sub-embedded decoder 512 can be updated based on the sum (or weighted sum) of the values ​​of the first loss function 524 and the second loss function. In some embodiments, the parameters of the 2D decoder 516 can also be updated during the training of the 3D autoencoder. The parameters of the 3D autoencoder can be updated based on the values ​​of the first and / or second loss functions through an iterative optimization process using an optimizer (such as stochastic gradient descent). Procedure 116 can iteratively update the parameters of the 3D autoencoder on batches of training examples.

[0116] The convergence of the optimization (e.g., the completion of training a 3D autoencoder) can be checked 618. In some implementations, convergence can be determined if the parameter update is less than a threshold. In some implementations, convergence occurs when the update has reached the accuracy limit supported by the computer or CAD program. In some implementations, optimization converges after a fixed number of iterations.

[0117] After training is complete, procedure 116 can use the trained 3D autoencoder 620. Procedure 116 can use the trained sub-embedding decoder and generate a 3D model of the target 3D object in process 200. Specifically, the trained sub-embedding decoder 512 can be... Figure 2A The sub-embedded decoder 208, and the trained sub-embedded decoder 512 can be used to decode the embedding 206 into sub-embedded embeddings 210(a) and 210(b). The trained envelope decoder 517 can be Figure 2A The envelope decoders 217(a) and 217(b) can be used to generate start and end envelope functions from the sub-embeddedness. Procedure 116 can also use a trained 3D encoder 508 as the 3D encoder 204 in procedure 200 to generate embeddings 206 from the representation 202 of the target 3D object.

[0118] Figure 7An example of a neural network architecture 700 for a 3D autoencoder that can be used to generate a 3D model of a target 3D object is shown. The 3D autoencoder includes a 3D encoder 702 and multiple decoding modules, such as sub-decoders 706(A), 706(B), 706(C), etc. Each of the decoding modules corresponds to a different set of predefined expansion directions. Each decoding module includes a corresponding sub-embedding decoder and a corresponding differentiable distributed engine to generate a 3D voxel model. A 3D object can be expanded from a 2D shape using a build sequence that includes expansion / extrusion directions and Boolean operations. Although many different 3D objects exist, many 3D objects can be constructed using a finite number of build sequences. The 3D autoencoder can determine a finite number of combinations of expansion / extrusion directions. The 3D autoencoder can determine a finite number of Boolean operations for each combination of extrusion directions. Therefore, the 3D autoencoder can include a finite number (e.g., 18) of decoding modules, and each decoding module can have a predetermined set of extrusion / expansion directions and a predetermined set of Boolean operations in a corresponding differentiable distributed engine. The same 2D decoder 712 can be shared among all decoding modules.

[0119] For example, a 3D autoencoder architecture 700 may include two or more decoding modules. Decoding module 706(A) includes a sub-embedding decoder 708, which can be trained to generate sub-embeddings 710(a) and 710(b) that encode 2D shapes in the yz plane (e.g., shape 714(a)) and in the xz plane (e.g., shape 714(b)). The extension directions of the 2D shapes 714(a) and 714(b) are predefined, for example, in the x and y directions, respectively. The extensions of the 2D shapes 714(a) and 714(b) can be combined using a differentiable distributed engine that defines predefined union, intersection, complement, or combinations thereof operations to generate a 3D voxel model 716. Decoding module 706(B) includes a sub-embedding decoder 720, which can be trained to generate sub-embeddings 722(a), 722(b), and 722(c) that encode three 2D shapes. The three 2D shapes may include, for example, shape 726(a) in the yz plane, shape 726(b) in the xy plane, and shape 726(c) in the xy plane. The extension directions of the three 2D shapes are predefined, for example, in the x, z, and z directions, respectively. The extensions of 2D shapes 726(a), 726(b), and 726(c) can be combined using a differentiable distributed engine that defines predefined union, intersection, complement, or combinations thereof operations to generate a 3D voxel model 728. The decoding module 706(C) includes a sub-embedding decoder 730, which can be trained to generate sub-embeddings 732 that encode the 2D shapes. The 2D shapes may include, for example, shape 734 in the xz plane. The extension direction of 2D shape 734 is predefined, for example, in the y direction. The extension of 2D shape 734 can produce a 3D voxel model 736.

[0120] During the training of a 3D autoencoder, for example, Figure 5 In process 500, procedure 116 can determine the expansion direction and Boolean operation for each ground reality voxel model and classify the ground reality voxel models into different categories. Training examples of ground reality voxel models with each category can be selected to train the corresponding decoding modules with the corresponding expansion direction and Boolean operation.

[0121] In inference, for example Figure 2A In process 200, program 116 can obtain a representation 701 of the target 3D object and determine one or more possible build sequences. Each build sequence can include a set of extrusion directions and Boolean operations. Program 116 can select a decoding module of the 3D autoencoder architecture 700 that operates on the extrusion directions and Boolean operations, and can use the selected decoding module to generate a 3D B-Rep model. The selection of the decoding module can be performed automatically by a computer program or by a user through a UI.

[0122] For example, given an input 3D shape 701, program 116 may select a decoding module 706(A) that extrudes a 2D shape 714(a) in the x-direction and a 2D shape 714(b) in the y-direction. Alternatively, for the input 3D shape 701, program 116 may also select a decoding module 706(B) that extrudes two xy-plane 2D shapes in the z-direction and one yz-plane 2D shape in the x-direction. Either decoding module can generate a satisfactory 3D B-Rep model. Program 116 may determine that decoding module 706(C) cannot generate a satisfactory 3D B-Rep model that extrudes only a single xz-plane 2D shape in the y-direction, for example, it cannot generate holes. In some implementations, the user may use two or more decoding modules to generate 3D B-Rep models and may select a preferred 3D B-Rep model from the results.

[0123] In some implementations, during inference, for example Figure 2A In process 200, procedure 116 may receive an initial input embedding 704 that encodes a representation of a target 3D object, without receiving the target geometry. Procedure 116 may generate a decoded 3D voxel model 701 from the initial input embedding using a 3D deconvolutional voxel decoder 703, which has been previously trained to decode the initial input embedding 206 into a 3D shape. The 3D deconvolutional voxel decoder 703 may include a CNN comprising: one or more 3D deconvolutional layers of configurable size, one or more 3D transposed convolutional layers of configurable size, one or more fully connected layers, one or more activation layers, or skip connections between layers, etc. Each voxel in the decoded 3D voxel model 701 may be a score indicating the likelihood that the voxel will be filled with material. Procedure 116 may then determine possible construction sequences based on the decoded 3D voxel model and may select a decoding module based on the possible construction sequences. For example, procedure 116 may receive the initial embedding 704, without receiving the target 3D geometry of the 3D shape. Program 116 can use a 3D deconvolutional voxel decoder 703 to process the initial embedding 704 as input and generate a decoded 3D voxel model 701. Based on the shape of the decoded 3D voxel model 701, program 116 can determine whether decoding module 706(A) or decoding module 706(B) can generate a satisfactory 3D B-Rep model.

[0124] Figure 8AAn example of the process for determining a fitted 2D parametric sketch model for complex 2D shapes is shown. An input 2D image is obtained via procedure 116. The input 2D image comprises two or more 2D shapes. For example, an input 2D image 802 is obtained via procedure 116. The input 2D image 802 comprises eight rectangles located at eight different positions and with eight different orientations in image 802. One of the rectangles is rectangle 804.

[0125] Sub-image portions are generated from the input 2D image by procedure 116. Each sub-image portion depicts a 2D shape among two or more 2D shapes. For example, eight image patches comprising each of eight rectangles can be generated, for example, through connected component analysis. For example, image patch 805 can be generated for rectangle 804. Procedure 116 generates a corresponding sub-image portion embedding for each sub-image portion. Procedure 116 can be used by, for example, in... Figure 4 The 2D encoder 412 of the 2D autoencoder trained in the process described in the article encodes eight sub-image parts to generate eight sub-image part embeddings.

[0126] The fitted 2D parametric sketch model is determined by procedure 116. That is, the fitted 2D parametric sketch model is embedded for each sub-image portion. For example, a fitted 2D parametric sketch model 806 can be embedded for the sub-image portions of image block 805. Procedure 116 generates a combined 2D parametric sketch model by combining the fitted 2D parametric sketch models at corresponding positions in the sub-image portions. For example, eight fitted 2D parametric sketch models can be combined at corresponding positions in the sub-image portions to generate a combined 2D parametric sketch model 808.

[0127] Figure 8BAn example of a process for generating sub-image portions (e.g., inner and outer loops) from an input 2D image is illustrated. Procedure 116 can obtain an input 2D image 812. The input 2D image includes a circular 2D object having a large hole at its center and six smaller holes. Procedure 116 can, for example, fill the large and small holes in the input 2D image 812 using morphological operations to obtain a 2D image 814. Procedure 116 can, for example, generate a difference image 816 between the 2D image 814 and the input 2D image 812 using subtraction. The difference image 816 includes seven sub-portions, such as the large central circle and six smaller circles. Procedure 116 can individually determine a fitted 2D parametric sketch model for each of the seven circles. Procedure 116 can generate a combined 2D parametric sketch model of the difference image 816 by combining the fitted 2D parametric sketch models of the seven circles. Program 116 can generate a 2D parametric sketch model for the input 2D image 812 by combining a fitted 2D parametric sketch model of the 2D image 814 with a complementary combination of a 2D parametric sketch model of the difference image 816, since the input 2D image 812 can be generated by subtracting seven circles from the 2D image 814.

[0128] Figure 9 This is a schematic diagram of a data processing system including a data processing device 900, which can be programmed as a client or server. The data processing device 900 is connected to one or more computers 990 via a network 980. Although in Figure 9 Only one computer is shown as data processing device 900, but multiple computers can be used. Data processing device 900 includes various software modules that can be distributed between the application layer and the operating system. These can include executable and / or interpretable software programs or libraries, including tools and services for one or more 3D modeling programs 904 that implement the systems and technologies described above. Thus, 3D modeling program 904 can be CAD program 904, and can implement the generation of prism B-Rep CAD models using machine learning algorithms. Furthermore, program 904 can potentially implement manufacturing control operations (e.g., generating and / or applying toolpath specifications to achieve the manufacturing of the designed object). In some cases, program 904 can potentially implement construction management or video and film production. The number of software modules used can vary depending on the implementation. Furthermore, software modules can be distributed across one or more data processing devices connected by one or more computer networks or other suitable communication networks.

[0129] The data processing device 900 also includes hardware or firmware means, including one or more processors 912, one or more auxiliary devices 914, a computer-readable medium 916, a communication interface 918, and one or more user interface devices 920. Each processor 912 is capable of processing instructions for execution within the data processing device 900. In some embodiments, the processor 912 is a single-threaded or multi-threaded processor. Each processor 912 is capable of processing instructions stored on the computer-readable medium 916 or a storage device such as one of the auxiliary devices 914. The data processing device 900 uses the communication interface 918 to communicate with one or more computers 990, for example, via a network 980. Examples of user interface devices 920 include displays, cameras, speakers, microphones, haptic feedback devices, keyboards, mice, and VR and / or AR devices. The data processing device 900 may store instructions for performing operations associated with the above-described programs on, for example, the computer-readable medium 916 or one or more auxiliary devices 914 (e.g., one or more of hard disk drives, optical disk drives, magnetic tape drives, and solid-state storage devices).

[0130] The embodiments of the subject matter and functional operation described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of these. Embodiments of the subject matter described in this specification can be implemented using one or more computer program instruction modules encoded on a non-transitory computer-readable medium for execution by a data processing device or for controlling the operation of a data processing device. The computer-readable medium can be an manufactured product, such as a hard disk drive in a computer system, or an optical disc sold through retail channels, or an embedded system. The computer-readable medium can be obtained separately, or it can be later encoded with one or more computer program instruction modules, such as by delivery via a wired or wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of these.

[0131] The term "data processing device" encompasses all devices, apparatuses, and machines used for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the device may also include code that generates the execution environment for the computer program in question, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, runtime environments, or combinations thereof. Furthermore, the device can employ a variety of different computing model infrastructures, such as network services, distributed computing, and grid computing infrastructures.

[0132] Computer programs (also called programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages, declarative or procedural languages) and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file storing one or more modules, subroutines, or code sections). Computer programs can be deployed to execute on one computer or on multiple computers (located in one place or distributed across multiple locations and interconnected via a communication network).

[0133] The processes and logic flows described in this specification can be executed by one or more programmable processors, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by special-purpose logic circuitry (e.g., FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit)), and the device can also be implemented as said special-purpose logic circuitry.

[0134] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or operatively coupled to receive data from or transfer data to said mass storage devices, or both. However, a computer does not need to have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example: semiconductor memory devices, such as EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or integrated into dedicated logic circuitry.

[0135] To provide interaction with a user, embodiments of the subject matter described herein can be implemented on a computer having a display device (e.g., an LCD (liquid crystal display), an OLED (organic light-emitting diode) display device, or another monitor) for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input.

[0136] A computing system may include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network. The client-server relationship arises from computer programs running on respective computers and having a client-server relationship with each other. Embodiments of the subject matter described in this specification may be implemented in a computing system including back-end components (e.g., as a data server); or in a computing system including middleware components (e.g., an application server); or in a computing system including front-end components (e.g., a client computer having a graphical user interface or web browser through which a user can interact with embodiments of the subject matter described in this specification); or in a computing system including any combination of one or more such back-end components, middleware components, or front-end components. Components of the system may be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0137] While this specification contains numerous details of implementation, these should not be construed as limiting the scope of the content protected by or potentially protected by the claims, but rather as descriptions of features specific to particular embodiments of the disclosed subject matter. Specific features described in this specification within the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually in multiple embodiments or in any suitable sub-combination. Furthermore, although features may be described above as functioning in certain combinations and even initially protected by the claims as such, one or more features from a claimed combination may, in some cases, be separable from said combination, and a claimed combination may involve sub-combinations or variations thereof.

[0138] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or in an ordered sequence, or that all shown operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged in multiple software products.

[0139] Several specific embodiments of the invention have been described. Other embodiments are within the scope of the following claims.

Claims

1. A method comprising: Obtain the input embedding that encodes the representation of the target's two-dimensional (2D) shape; The input embedding is processed by a 2D decoder of a 2D autoencoder to obtain a decoded representation of the target 2D shape, wherein the 2D autoencoder includes a 2D encoder that processes the representation of a 2D object to generate an object embedding, and a 2D decoder that processes the object embedding to generate the decoded representation of the 2D object. Determining a fitted 2D parametric sketch model for the input embedding includes: A search is used in the embedding space of the 2D autoencoder and in a sketch model database associated with the 2D autoencoder to find a 2D parametric sketch model of the input embedding, wherein the shape of the 2D parametric sketch model is determined by one or more parameter values ​​of the 2D parametric sketch model; and The 2D parametric sketch model is fitted to the decoded representation of the target 2D shape by modifying one or more parameter values ​​of the 2D parametric sketch model to generate the fitted 2D parametric sketch model; and The fitted 2D parametric sketch model is used in a computer modeling program.

2. The method of claim 1, further comprising: Obtain a parametric instance of a 2D parametric sketch model; 2D training images are generated from the parametric instances of the 2D parametric sketch model, wherein each of the 2D training images corresponds to a parametric instance of the 2D parametric sketch model. as well as Training the 2D autoencoder on the 2D training images includes: For each of the 2D training images: The 2D training images are processed using the 2D encoder to generate embeddings; and The 2D decoder is used to process the embedding to generate a decoded 2D image; The loss function is calculated by comparing each of the 2D training images with its corresponding decoded 2D image; and The parameters of the 2D encoder and the 2D decoder are updated based on the value of the loss function.

3. The method of claim 2, wherein training the 2D autoencoder on the 2D training image comprises: Generate a signed distance field image from the 2D training image; as well as The 2D encoder is used to process the signed range field image to generate the embedding.

4. The method of claim 1, further comprising: Obtain the initial input embedding for encoding the representation of the target three-dimensional (3D) object; The initial input embedding is processed using a sub-embedding decoder of a 3D autoencoder to obtain a sub-embedding that includes the input embedding, wherein the sub-embedding encodes the 2D shape of the representation that defines the target 3D object; Generate a parametric sketch model, including: Each of the sub-embeddings is processed using one or more intermediate 2D decoders to obtain the 2D shape that defines the representation of the target 3D object; Each of the intermediate embeddings is generated by processing each of the 2D shapes using the 2D autoencoder; and The execution determines a corresponding parametric sketch model for each of the intermediate embeddings, wherein the corresponding parametric sketch model is the fitted 2D parametric sketch model, and wherein the decoded representation of the target 2D shape is each of the 2D shapes; Generate a set of extrusion parameters from the sub-embedding; and Generate a 3D boundary representation B-Rep model of the target 3D object, wherein the generation includes constructing the 3D B-Rep model by extruding it into 3D space using the fitted 2D parametric sketch model in a construction sequence, wherein the construction sequence includes the set of extrusion parameters.

5. The method of claim 4, wherein the sub-embedded decoder comprises a multilayer perceptron (MLP).

6. The method of claim 4, wherein the one or more intermediate 2D decoders include the 2D decoder of the 2D autoencoder.

7. The method of claim 4, wherein the one or more intermediate 2D decoders include a second 2D decoder that is different from the 2D decoder of the 2D autoencoder.

8. The method of claim 4, wherein the 3D autoencoder comprises: A 3D encoder that processes an input voxel model to generate a 3D object embedding. The sub-embedding decoder processes the 3D object embedding to generate the sub-embedding. A start envelope decoder processes each of the sub-embedded elements to generate a start envelope function. An ending envelope decoder processes each sub-embedded element in the sub-embedded elements to generate an ending envelope function, wherein a set of extrusion parameters is generated from the starting envelope function and the ending envelope function, and A differentiable distributed engine generates a reconstructed model by extruded the 2D shape into the 3D space using the starting envelope function and the ending envelope function.

9. The method of claim 8, further comprising: Obtain training examples, wherein each training example includes a training voxel model, a ground reality voxel model, and a ground reality 2D shape, wherein the ground reality voxel model is defined by extruding the ground reality 2D shape into 3D space using a set of ground reality extrusion parameters that define a ground reality envelope function, wherein the training voxel model is generated from the ground reality voxel model. as well as Training the 3D autoencoder on the training example includes: For each of the training examples: The training voxel model is processed using the 3D encoder to generate the 3D object embedding of the training voxel model; and The 3D object embedding is processed using the sub-embedded decoder to generate the sub-embedding; The 2D decoder is used to process each of the sub-embedded elements to generate the 2D shape in the 2D shape; The starting envelope decoder is used to process each of the sub-embedded elements to generate the starting envelope function of the 2D shape; The termination envelope decoder is used to process each sub-embedded in the sub-embedded to generate the termination envelope function of the 2D shape; and The reconstructed voxel model of the training voxel model is generated by using the 2D shape with a predicted construction sequence to construct a reconstructed voxel model in the 3D space by expanding the construction sequence, wherein the predicted construction sequence includes operations defined in the differentiable distributed engine and the start envelope function and the end envelope function of each of the 2D shapes. The value of the first loss function is calculated by comparing each trained voxel model with its corresponding reconstructed voxel model; and The parameters of the 3D autoencoder are updated based at least on the value of the first loss function.

10. The method of claim 9, further comprising: The value of the second loss function is calculated by comparing the ground reality 2D shape with the 2D shape and by comparing the starting envelope function and the ending envelope function with the ground reality envelope function; as well as The parameters of the 3D autoencoder are updated based at least on the values ​​of the first loss function and the second loss function.

11. The method of claim 9, wherein the training voxel model is generated from the ground-based voxel model through morphological modifications.

12. The method of claim 8, wherein the 3D autoencoder includes decoding modules and each of the decoding modules corresponds to a different set of predefined one or more extrusion directions and different predefined one or more Boolean operations, wherein each of the decoding modules includes a corresponding sub-embedding decoder, a corresponding start envelope decoder, and a corresponding end envelope decoder.

13. The method of claim 4, wherein obtaining the initial input embedding comprises: Generate a first 3D object embedding from a first voxel model of a first 3D object; Generate a second 3D object embedding from the second voxel model of the second 3D object; as well as The initial input embedding is generated from the first 3D object embedding and the second 3D object embedding.

14. The method of claim 4, further comprising: Obtain the initial voxel model of the target 3D object; The modified voxel model is generated by morphologically modifying the initial voxel model. The initial input embedding is generated by processing the modified voxel model using a 3D encoder included in the 3D autoencoder; as well as Using the construction sequence, a reconstructed 3D B-Rep model is generated from the fitted 2D parametric sketch model by extruding it into the 3D space, wherein the reconstructed 3D B-Rep model is similar to the initial voxel model.

15. The method of claim 4, further comprising: Obtain an initial voxel model of the target 3D object, wherein the initial voxel model is generated from the output of a generative design. The initial input embedding is generated by processing the initial voxel model using a 3D encoder included in the 3D autoencoder; as well as Generate a 3D prism model of the target 3D object, wherein the 3D prism model of the target 3D object is the 3DB-Rep model.

16. The method of claim 4, further comprising: Obtain an initial voxel model of the target 3D object, wherein the initial voxel model is generated from the output of a generative design. The initial input embedding is generated by processing the initial voxel model using a 3D encoder included in the 3D autoencoder; as well as The 3D B-Rep model is fitted to the initial voxel model of the target 3D object by changing one or more parameters of the 3D B-Rep model to produce a fitted 3D B-Rep model.

17. The method of claim 1, wherein the fitted 2D parametric sketch model comprises: The fitted 2D parametric sketch model is displayed in the user interface of the computer modeling program.

18. The method of claim 1, further comprising: Obtain an input 2D image, wherein the input 2D image comprises two or more 2D shapes; Sub-image portions are generated from the input 2D image, wherein each sub-image portion depicts one of the two or more 2D shapes; Generate a corresponding sub-image portion embedding for each sub-image portion in the sub-image portion; Determining the fitted 2D parametric sketch model includes: performing the determination of each fitted 2D parametric sketch model for each sub-image partial embedding; as well as A combined 2D parametric sketch model is generated by combining the fitted 2D parametric sketch models at corresponding positions in the sub-image portion.

19. A system comprising: A non-transitory storage medium, wherein instructions for a computer-aided design program are stored on the non-transitory storage medium; as well as One or more data processing devices, the one or more data processing devices being configured to run the instructions of the computer-aided design program to perform the operations specified by the instructions of the computer-aided design program; The operation includes Obtain the input embedding that encodes the representation of the target's two-dimensional (2D) shape; The input embedding is processed by a 2D decoder of a 2D autoencoder to obtain a decoded representation of the target 2D shape, wherein the 2D autoencoder includes a 2D encoder that processes the representation of a 2D object to generate an object embedding, and a 2D decoder that processes the object embedding to generate the decoded representation of the 2D object. Determining a fitted 2D parametric sketch model for the input embedding includes: A search is used in the embedding space of the 2D autoencoder and in a sketch model database associated with the 2D autoencoder to find a 2D parametric sketch model of the input embedding, wherein the shape of the 2D parametric sketch model is determined by one or more parameter values ​​of the 2D parametric sketch model; and The 2D parametric sketch model is fitted to the decoded representation of the target 2D shape by modifying one or more parameter values ​​of the 2D parametric sketch model to generate the fitted 2D parametric sketch model; and The fitted 2D parametric sketch model is used in a computer modeling program.

20. The system of claim 19, wherein the operation includes: Obtain a parametric instance of a 2D parametric sketch model; 2D training images are generated from the parametric instances of the 2D parametric sketch model, wherein each of the 2D training images corresponds to a parametric instance of the 2D parametric sketch model. as well as Training the 2D autoencoder on the 2D training images includes: For each of the 2D training images: The 2D training images are processed using the 2D encoder to generate embeddings; and The 2D decoder is used to process the embedding to generate a decoded 2D image; The loss function is calculated by comparing each of the 2D training images with its corresponding decoded 2D image; and The parameters of the 2D encoder and the 2D decoder are updated based on the value of the loss function.

21. The system of claim 19, wherein the operation includes: Obtain the initial input embedding for encoding the representation of the target three-dimensional (3D) object; The initial input embedding is processed using a sub-embedding decoder of a 3D autoencoder to obtain a sub-embedding that includes the input embedding, wherein the sub-embedding encodes the 2D shape of the representation that defines the target 3D object; Generate a parametric sketch model, including: Each of the sub-embeddings is processed using one or more intermediate 2D decoders to obtain the 2D shape that defines the representation of the target 3D object; Each of the intermediate embeddings is generated by processing each of the 2D shapes using the 2D autoencoder; and The execution determines a corresponding parametric sketch model for each of the intermediate embeddings, wherein the corresponding parametric sketch model is the fitted 2D parametric sketch model, and wherein the decoded representation of the target 2D shape is each of the 2D shapes; Generate a set of extrusion parameters from the sub-embedding; and Generate a 3D boundary representation B-Rep model of the target 3D object, wherein the generation includes constructing the 3D B-Rep model by extruding it into 3D space using the fitted 2D parametric sketch model in a construction sequence, wherein the construction sequence includes the set of extrusion parameters.

22. The system of claim 21, wherein the 3D autoencoder comprises: A 3D encoder that processes an input voxel model to generate a 3D object embedding. The sub-embedding decoder processes the 3D object embedding to generate the sub-embedding. A start envelope decoder processes each of the sub-embedded elements to generate a start envelope function. An ending envelope decoder processes each sub-embedded element in the sub-embedded elements to generate an ending envelope function, wherein a set of extrusion parameters is generated from the starting envelope function and the ending envelope function, and A differentiable distributed engine generates a reconstructed model by extruded the 2D shape into the 3D space using the start envelope function and the end envelope function; And the operation described therein includes: Obtain training examples, wherein each training example includes a training voxel model, a ground reality voxel model, and a ground reality 2D shape, wherein the ground reality voxel model is defined by extruded the ground reality 2D shape into 3D space using a set of ground reality extrusion parameters that define a ground reality envelope function, and wherein the training voxel model is generated from the ground reality voxel model; and Training the 3D autoencoder on the training example includes: For each of the training examples: The training voxel model is processed using the 3D encoder to generate the 3D object embedding of the training voxel model; and The 3D object embedding is processed using the sub-embedded decoder to generate the sub-embedding; The 2D decoder is used to process each of the sub-embedded elements to generate the 2D shape in the 2D shape; The starting envelope decoder is used to process each of the sub-embedded elements to generate the starting envelope function of the 2D shape; The termination envelope decoder is used to process each sub-embedded in the sub-embedded to generate the termination envelope function of the 2D shape; and The reconstructed voxel model of the training voxel model is generated by using the 2D shape with a predicted construction sequence to construct a reconstructed voxel model in the 3D space by expanding the construction sequence, wherein the predicted construction sequence includes operations defined in the differentiable distributed engine and the start envelope function and the end envelope function of each of the 2D shapes. The value of the first loss function is calculated by comparing each trained voxel model with its corresponding reconstructed voxel model; and The parameters of the 3D autoencoder are updated based at least on the value of the first loss function.

23. A non-transitory computer-readable medium encoding instructions operable to cause a data processing device to perform operations, said operations including: Obtain the input embedding that encodes the representation of the target's two-dimensional (2D) shape; The input embedding is processed by a 2D decoder of a 2D autoencoder to obtain a decoded representation of the target 2D shape, wherein the 2D autoencoder includes a 2D encoder that processes the representation of a 2D object to generate an object embedding, and a 2D decoder that processes the object embedding to generate the decoded representation of the 2D object. Determining a fitted 2D parametric sketch model for the input embedding includes: A search is used in the embedding space of the 2D autoencoder and in a sketch model database associated with the 2D autoencoder to find a 2D parametric sketch model of the input embedding, wherein the shape of the 2D parametric sketch model is determined by one or more parameter values ​​of the 2D parametric sketch model; and The 2D parametric sketch model is fitted to the decoded representation of the target 2D shape by modifying one or more parameter values ​​of the 2D parametric sketch model to generate the fitted 2D parametric sketch model; and The fitted 2D parametric sketch model is used in a computer modeling program.

24. The non-transitory computer-readable medium of claim 23, wherein the operation comprises: Obtain a parametric instance of a 2D parametric sketch model; 2D training images are generated from the parametric instances of the 2D parametric sketch model, wherein each of the 2D training images corresponds to a parametric instance of the 2D parametric sketch model. as well as Training the 2D autoencoder on the 2D training images includes: For each of the 2D training images: The 2D training images are processed using the 2D encoder to generate embeddings; and The 2D decoder is used to process the embedding to generate a decoded 2D image; The loss function is calculated by comparing each of the 2D training images with its corresponding decoded 2D image; and The parameters of the 2D encoder and the 2D decoder are updated based on the value of the loss function.

25. The non-transitory computer-readable medium of claim 23, wherein the operation comprises: Obtain the initial input embedding for encoding the representation of the target three-dimensional (3D) object; The initial input embedding is processed using a sub-embedding decoder of a 3D autoencoder to obtain a sub-embedding that includes the input embedding, wherein the sub-embedding encodes the 2D shape of the representation that defines the target 3D object; Generate a parametric sketch model, including: Each of the sub-embeddings is processed using one or more intermediate 2D decoders to obtain the 2D shape that defines the representation of the target 3D object; Each of the intermediate embeddings is generated by processing each of the 2D shapes using the 2D autoencoder; and The execution determines a corresponding parametric sketch model for each of the intermediate embeddings, wherein the corresponding parametric sketch model is the fitted 2D parametric sketch model, and wherein the decoded representation of the target 2D shape is each of the 2D shapes; Generate a set of extrusion parameters from the sub-embedding; and Generate a 3D boundary representation B-Rep model of the target 3D object, wherein the generation includes constructing the 3D B-Rep model by extruding it into 3D space using the fitted 2D parametric sketch model in a construction sequence, wherein the construction sequence includes the set of extrusion parameters.

Images