Systems, methods, and data structures for mapping 3D objects to 2D shadow profile rendering

Abstract

Embodiments described herein relate to a system for real-time conversion of a 3D model to a 2D shadow profile rendering. The system includes a processor in communication with a memory. The memory stores executable instructions, and when the processor executes the executable instructions, the system is configured to receive a 3D model input corresponding to a physical object and to generate a data structure including one or more features of the physical object and one or more 2D renders of the physical object based on the 3D model input. The processor further configures the system to associate the one or more features with the one or more 2D renders of the physical object and determine a shadow profile rendering of the physical object based on the one or more features (301). The system is configured to transmit a shadow profile rendering of a physical object to a display device.

Description

Cross Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 502,416, filed May 16, 2023, under 35 U.S.C. § 119(e), the contents of which are incorporated herein by reference. BACKGROUND

[0002] Contour and surface shading renderings are a fundamental component of product design documentation, requiring clear presentation of design features in formats that comply with industry and manufacturing standards. Current methods of creating contour and surface shading renderings are fraught with challenges. Manual drafting is not only time consuming, but also prone to human error, leading to potential inconsistencies in the interpretation of complex CAD models. Some algorithmic approaches are inaccurate and do not provide predictable results. These methods require significant time and resource investment, which is cost prohibitive for entities that render frequently. SUMMARY

[0003] Accordingly, some embodiments include a system for converting a 3D model to a 2D shaded contour rendering in real-time. The system includes a processor in communication with a memory. The memory stores executable instructions that, when executed by the processor, configure the system to receive a 3D model input corresponding to a physical object (200).

[0004] In some embodiments, the system is configured to generate a data structure including one or more features of the physical object and one or more 2D renderings of the physical object based on the 3D model input. In some embodiments, the system associates the one or more features with the one or more 2D renderings of the physical object and determines a shaded contour rendering of the physical object based on the one or more features. The system is configured to transmit the shaded contour rendering of the physical object to a display device.

[0005] Some embodiments include a computer-implemented method that includes receiving a three-dimensional model of a physical object and determining one or more two-dimensional views of the physical object based on the three-dimensional model. In some embodiments, the method includes associating one or more feature vectors of the three-dimensional object with the one or more two-dimensional views and outputting a data structure including the one or more features. In some embodiments, the method includes training a first predictive model with the data structure and training a second predictive model with the data structure. In some embodiments, the method includes generating a predicted rendering with the data structure. BRIEF DESCRIPTION OF DRAWINGS

[0006] For a more detailed understanding of the above-mentioned features of the present disclosure, reference can be made to the following more detailed description of the present disclosure taken in conjunction with the accompanying drawings, in which:

[0007] For the purposes of the present disclosure, the term "coupled" can be used in various contexts and can mean one or more direct connections, one or more indirect connections, or a combination thereof. Certain terms can be used for describing particular elements. These terms, as well as their derivatives, can be used herein to refer to the structural characteristics of an element together with other elements denoted by a common term. For example, the term "connected" can be used to describe two elements that are either directly connected, indirectly connected via one or more intermediate elements, or a combination thereof. It is also contemplated that one element can be "connected" to multiple other elements by using one or more connecting elements.

[0008] Figure 1 A schematic diagram of a system for mapping 3D objects to silhouette renderings is shown, in accordance with some embodiments;

[0009] Figure 2 A schematic diagram of a 3D object in relation to a 2D silhouette rendering is shown, in accordance with some embodiments;

[0010] Figures 3A-3G A schematic diagram of a 3D object in relation to a 2D view is shown, in accordance with some embodiments;

[0011] Figure 4 A data structure for training a neural network and generating silhouette renderings is depicted, in accordance with some embodiments;

[0012] Figures 5-7 A flowchart of an example method for training a neural network and generating silhouette renderings is depicted, in accordance with some embodiments; and

[0013] Figure 8 A neural network schematic for 3D object association is depicted, in accordance with some embodiments. DETAILED DESCRIPTION

[0014] The present disclosure will now be described in detail by way of specific examples, with reference to the attached drawings.

[0015] As used herein, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. As used herein, the statement that two or more parts or components are "coupled" shall mean that the parts are joined or operate together either directly or indirectly, such that forces other than forces caused by gravity act upon them. As used herein, "directly coupled" means that two elements are in contact with each other. As used herein, "fixedly coupled" or "fixed" means that two components are coupled so as to move as one while maintaining a constant orientation relative to each other. As used herein, "operatively coupled" means that two elements are coupled in a manner that one is able to act upon the other. It should be understood that "operatively coupled" does not require that two elements directly contact each other. As used herein, "substantially" means any variation is negligible or insubstantial, or the variation is within acceptable manufacturing tolerances for the relevant art, and provides the desired performance and / or functionality described in one or more of the embodiments herein. Descriptions of ranges include endpoints.

[0016] As used herein, the term "unitary" means a component is created as a single piece or unit. That is, the component is not created in multiple pieces and then coupled together as a unit. As employed herein, the statement that two or more parts or components "engage" one another shall mean that the parts exert a force against one another either directly or through one or more intermediate parts or components. As employed herein, the term "number" shall mean one or an integer greater than one (i.e., a plurality).

[0017] The described hardware- implemented embodiments are not to be limited to that specifically described herein, which can include software- or software and hardware-combined implementations, and vice versa, as will be apparent to those skilled in the art, unless otherwise indicated herein. In the exemplary embodiments described herein, embodiments showing a single component should not be taken as limiting; rather, multiple components including like components can be employed, and vice versa, unless otherwise explicitly stated herein. Also, applicant does not intend to convey any of the terms in the specification or claims as having any special or in common meaning, unless explicitly set forth herein. Moreover, the present invention encompasses current and future known equivalents of the known components referred to herein by way of example. Nothing in the specification should be construed as indicating any special or in common meaning for any term.

[0018] The embodiments described herein relate generally to systems, components thereof, and methods of using the same for real-time rendering of 3D objects via neural networks and predictive models to generate silhouette and surface shading renderings. For example, rendering of silhouette features by surface shading is an integral part of many industrial design and manufacturing applications. For example, for injection molding manufacturing, such products are rendered in three dimensions (3D) via CAD programs. However, when shown in 2D, these renderings lose depth and silhouette information. Accordingly, embodiments herein leverage computer-aided design (CAD) and artificial intelligence (AI) by employing machine learning (ML) models to predict the conversion of CAD models to 2D renderings, including real-time silhouette and surface shading renderings on a network, such as the Internet.

[0019] Accordingly, some embodiments herein provide a system, method, and data structure for an AI-driven solution designed to address these and other inefficiencies. By integrating advanced neural networks including Convolutional Neural Networks (CNNs), U-Net, and Generative Adversarial Networks (GANs), embodiments herein provide real-time conversion of 3D CAD models to 2D silhouette and surface shading renderings. The AI-driven approach of embodiments herein ensures high precision and consistency, significantly reduces time and cost associated with manual rendering, and minimizes or eliminates errors. In some embodiments, the AI models of embodiments herein are trained on a dataset of CAD models and corresponding silhouette and surface shading renderings, enabling the predictive models to learn and replicate necessary styles and regulatory requirements in real-time over a communication network. Accordingly, embodiments herein provide a scalable, efficient, and human-free alternative to other methods of preparing silhouette and surface shading renderings, thereby improving productivity and reducing operational costs.

[0020] Reference is now made to Figure 1 , Figure 1 An exemplary system 10 (hereinafter “system 10”) for mapping three-dimensional (3D) objects to silhouette shading renderings is shown in actual use in a network environment. As shown in FIG. 1, the system 10 includes a CAD model 12, a predictive model 14, and a network 16. Figure 1As shown, in some embodiments, system 10 can include server 102, resources 120, and user device 140 with GUI 142. In some embodiments, user device 140 can include a smartphone, a laptop, a desktop computer, and / or any computing device that serves as an endpoint for a user (e.g., an engineer or an illustrator) to interact with system 10. User device 140 facilitates a user to upload an inputted 3D CAD model and download outputted generated silhouette feature renderings, which can be displayed on GUI 142. As Figure 1 As shown, server 102, resources 120, and user device 140 can communicate with each other via network 130. The architecture of system 10 is configured to handle the process of converting a 3D CAD model of a physical object into silhouette feature renderings, which will be discussed in more detail below.

[0021] In some embodiments, server 102 includes processor 104 in communication with memory 106. Memory 106 can include software code 105. Processor 104 is configured to receive and execute software code 105 to implement one or more embodiments described herein. For example, server 102 can execute code 105 and cause system 10 to output a predicted rendering of a silhouette feature rendering based on an inputted 3D CAD model file. Such output can be transmitted to a remote client or user in real-time via network 130. In some embodiments, server 102 includes one or more modules for performing respective functions of the embodiments described herein. For example, in some embodiments, server 102 includes data management 108, training 110, feature extraction 112, image generation 114, output and delivery 116, monitoring and logging 118, and / or integrated data association 119, which will be described in more detail below.

[0022] In some embodiments, server 102 can communicate with external resources 120 via network 130. In some embodiments, network 130 may, for example, include a LAN / WAN connection configured to provide internet connectivity via a hybrid fiber optic (HFC) transmission network (e.g., an Ethernet twisted pair shielded CAT-5, WiFi, premises coaxial cable network, or any other connection capable of establishing an internet connection). In some embodiments, network 130 can include a wireless network (e.g., 5G, LTE, 4G, CDMA, etc.) capable of establishing an internet connection. Network 130 facilitates utilizing external resources 120 to implement various functions, which will be described in more detail below.

[0023] In some embodiments, external resources 120 can include remote databases and / or access to third-party API services, facilitating integration and interaction between system server 102 and remote clients at user devices 140 with external systems and resources to enable enhanced functionality. For example, resources 120 can facilitate establishing connections with various third-party API services, enabling the system to leverage external tools and data sources. Such API services can include, but are not limited to, platforms providing advanced AI processing capabilities and predictive analytics tools, which are described in greater detail below. In some embodiments, external resources 120 can establish connections with one or more remote databases (not shown), which can be beneficial in enhancing data processing and processing capabilities of system 10. By implementing external resources 120, system 10 can advantageously expand the range of functionality, such as real-time data analysis, machine learning processes, and complex predictive modeling, which are discussed in greater detail below. These external resources 120 not only enrich the user experience by providing more accurate and efficient results, but also enhance overall system performance by integrating virtualization techniques to improve process efficiency, which are described in greater detail below.

[0024] One or more components of system 10 (e.g., devices 140, processors 104, and / or modules 108, 110, 112, 114, 116, 118, 119) can be implemented in digital electronic circuitry, in integrated circuitry, in specially designed application specific integrated circuits (ASICs), in field programmable gate arrays (FPGAs), in computer hardware, in firmware, in software, and / or in combinations thereof. These various aspects or features can include implementation in one or more computer programs (e.g., code 105) that are executable and / or interpretable on a programmable system including one or more programmable processors (e.g., 104), which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system (e.g., memory 106), at least one input device, and at least one output device. The programmable system or computing system can include clients (e.g., user devices 140) and servers (e.g., 102). The clients and the servers generally both run programs in their respective computer systems and have a client-server relationship with each other over a communication network (e.g., 130). The relationship between client and server programs is established by computer programs running on the respective computers and having a client-server relationship to each other.

[0025] Such computer programs (which can also be referred to or referred to as programs, software, software applications, applications, components, or code) include non-transitory machine-readable instructions for a programmable processor, and can be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logical programming language, and / or in assembly / machine language. As used herein, the term "machine-readable medium" (or "computer-readable medium") refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" (or "computer-readable signal") refers to any signal that can be used to provide machine instructions and / or data to a programmable processor. The machine-readable medium can store such machine instructions non-transitorily, such as a non-transitory solid-state memory or a magnetic hard drive or any other such medium, and / or can store such machine instructions in a transitory manner, such as a processor cache or other random access memory associated with one or more physical processor cores.

[0026] In some embodiments, the server 102 can include data management 108, training 110, feature extraction 112, image generation 114, output and delivery 116, monitoring and logging 118, and / or integrated data correlation 119. As described in greater detail below, these modules 108-119 can operate individually and / or in coordination with one another to implement the embodiments described herein. In embodiments described herein, functions performed by a certain module can be performed by another different module in other embodiments, or split among multiple modules. For example, functions described in one embodiment as being performed by the data management 108 can be performed in another embodiment by the integrated data correlation 119 and / or the monitoring and logging 118, and / or vice versa.

[0027] For example, in some embodiments, the data management 108 manages all data-related operations, including the receipt, storage, and preprocessing of CAD files and shadow contour renderings. The data management 108 ensures the integrity of the data and efficient access and reading of the storage scheme. In some embodiments, the data management 108 manages data-related operations within the system 10. For example, by receiving, securely storing, and systemically preprocessing CAD files and detailed shadow contour renderings.

[0028] In some embodiments, data management 108 automatically receives new CAD files, ensuring that these incoming files are properly formatted and error-free. In some embodiments, the receiving process can include verifying the data structure of incoming files against the requirements of system 10, and performing preliminary checks on data integrity and completeness.

[0029] In some embodiments, data management 108 pre-processes the CAD files to extract features required for subsequent image generation using the integrated CNN / U-Net architecture discussed further below. Such pre-processing includes applying morphological operations to enhance feature visibility and prepare data for efficient feature extraction and segmentation by feature extraction module 112.

[0030] In some embodiments, data management 108 organizes data storage to ensure that both raw and processed data are stored in a structured manner for quick retrieval. This organization can be achieved through the use of optimized data indexing and partitioning strategies that can improve the performance of data queries and reduce latency in data access. To ensure the security and integrity of data throughout its lifecycle, data management 108 implements robust encryption methods for both static and in-transit data. Data management 108 can also maintain data integrity checks to prevent and correct any data corruption or loss.

[0031] In some embodiments, data management 108 closely collaborates with feature extraction (112), image generation (114), and output and delivery (116) modules. Data management 108 ensures seamless data flow between these modules and maintains consistency in data format and structure to support the end-to-end process of generating shadow outline feature renderings. This integration facilitates maintaining high efficiency and accuracy in generating detailed and standardized shadow outline feature renderings. In one embodiment, data management 108 can optimize the retrieval process to ensure that data access is quick and does not become a bottleneck, for example, when handling large numbers of high-resolution CAD models and their corresponding illustrations.

[0032] In some embodiments, training module 110 utilizes historical data to manage the training of machine learning models to effectively learn the creation of shadow outline renderings. Training module 110 advantageously continuously improves the accuracy and processing efficiency of the models. Training module 110 employs sophisticated training algorithms and cross-validation techniques to optimize model parameters and prevent overfitting. Regular update and retraining cycles ensure that the models can adapt to new types of CAD designs and evolving illustration standards, enhancing the adaptability and long-term performance of system 10.

[0033] In some embodiments, feature extraction 112 extracts essential feature vectors from the CAD model necessary to generate accurate silhouette rendering. This process involves complex algorithms designed to accurately identify and delineate edges, textures, and other geometric features critical for detailed rendering. Advanced image processing techniques such as edge detection, texture analysis, and morphological transformations are utilized to enhance feature visibility and distinguishability, which is advantageous for subsequent stages of image generation.

[0034] In some embodiments, image generation 114 utilizes trained models to convert processed CAD data into high-quality silhouette renderings. Image generation module 114 integrates artificial intelligence techniques, including GANs, to synthesize images. Through iterative optimization and learning, image generation module 114 continually improves its ability to generate outputs that accurately represent the design attributes and specifications of the original CAD model.

[0035] In some embodiments, output and delivery 116 is responsible for the final stage, where the silhouette renderings are formatted and delivered to user devices 140. The output and delivery module ensures that each image is rendered in the correct format, adhering to precise specifications required for industry submission, and maintaining the highest quality standards. The output and delivery module also manages distribution channels to ensure that the illustration renderings are delivered to user devices 140 safely and efficiently, while supporting various output formats and user-specific customization options.

[0036] In some embodiments, monitoring and logging 118 oversees the performance of the system and the operational integrity of system 10 by continuously monitoring and recording the activities of system 10. Monitoring and logging 118 has advantages for proactive troubleshooting, performance optimization, and maintaining the reliability of system 10. For example, by collecting and analyzing log data to detect anomalies, predict potential system failures, and initiate preventive measures. The insights gained from this data are beneficial for ongoing system maintenance, audits, and compliance with technical and regulatory standards.

[0037] In some embodiments, integrated data correlation (IDC) 119 serves as the central point for integrating 3D CAD data with 2D views and depth maps. This ensures that all parts of the system have access to the same unified data format, maintaining consistency across processes. In some embodiments, IDC 119 manages data preprocessing via normalization and standardization to standardize the format and scale of 3D and 2D data, ensuring that inputs to machine learning models are consistent and performance is optimized.

[0038] In some embodiments, the IDC 119 facilitates data caching by implementing a caching mechanism to store processed data, thereby reducing redundancy in data processing between different modules (e.g., 108, 110, 112, 114, 116, 117, 118, and / or 119) (e.g., when both a training model and a prediction model need to access the same data). The IDC 119 provides resource optimization by ensuring that data is processed and stored in a manner that minimizes memory usage and computational load, which is particularly important in cloud-based architectures. In some embodiments, the IDC 119 facilitates sharing of extracted features and deep information between different machine learning models, thereby improving the efficiency and effectiveness of the system by leveraging information learned across tasks.

[0039] Reference is now made to Figure 2 -3 and in conjunction with Figure 1 , Figure 2 A 3D CAD model of a physical object 200 is depicted along with a corresponding silhouette rendering 201 of the physical object 200. As shown in the CAD model, the object 200, shown in the form of a so-called CAD wireframe, does not convey depth information well. In contrast, the silhouette rendering 201 displays such depth information via hatching lines 204, which, although displayed in 2D form, distinguish the contours of the object 200. The contour lines 204 in the 3D model can be mapped to surface hatching in an output file. Hatching is particularly important when displaying three-dimensional items, where the hatching lines 204 are beneficial for delineating planar, concave, convex, raised, and / or indented surfaces of the object 200, and for distinguishing between open and solid surfaces. Transparent and translucent surfaces can be represented by diagonal hatching (not shown). Figure 2

[0040] Thus, in some embodiments, the training module 110 can perform a method of training a machine learning model for generating predictive silhouette renderings with hatching lines 204. The method can contrast a CAD model (e.g., object 200) without shading with a silhouette rendering counterpart (e.g., silhouette rendering 201) annotated with labels of various surface types. In some embodiments, the method of training a machine learning model can include classifying surface types (e.g., convex spherical surface, concave surface, flat surface, inner cylindrical surface, etc.) and applying appropriate shading techniques, which are discussed in detail below.

[0041] Data preparation and image classification via CNNs

[0042] ​In some embodiments, system 10 provides data preparation and annotation. Input data for training the model includes raw images or processed features from 3D CAD models. For each input image, the corresponding output should be an annotated image, for example, including all surface and shadow annotations, as shown in Figure 2 For example, by accurately annotating a CAD model with the types of surfaces and their respective shadow characteristics, as shown in shadow image 201. Each surface type (e.g., convex spherical surface, concave surface, flat surface, inner cylindrical surface, etc.) is clearly identified and labeled. In some embodiments, data preparation and annotation includes annotation. For example, annotation of 2D engineering drawings (B-2H) in a similar manner to the shadow image annotation in the example. These annotations include identification of curves, hatching, and linear surfaces. Accurate annotation will provide benchmark real data that the model can learn from. Figure 2

[0043] For example, object 200 corresponds to a 3D CAD model of a complex object containing detailed geometric data. The output is a 2D shadow contour feature rendering. As shown in Figure 2 A, object 201 includes data annotations. Each pair of input and output (200, 201) is prepared. In some embodiments, system 10 can train the model with thousands of pairs of input and output via resources 120. In some embodiments, tens of thousands or hundreds of thousands of pairs of input and output are implemented, which are not listed here for brevity. In some embodiments, for 3D CAD models, data preparation can include converting 3D CAD files to a format suitable for processing (e.g., raster images, simplified vector format). The corresponding output file needs to be annotated, for example, by indicating different features that the model should identify and convert.

[0044] Feature extraction

[0045] In some embodiments, the method for training the prediction model includes edge detection. For example, by using an algorithm capable of detecting edges and curves to distinguish different parts of the object. Such detection can be achieved through deeper CNN layers specifically trained to recognize such features in CAD drawings, which will be described in more detail below.

[0046] ​In some embodiments, the method includes surface analysis. For example, by implementing feature extraction techniques, texture and geometric features can be analyzed to classify different surface types. Such classification can include analyzing gradients in the image, curvature data derived from CAD models, or other morphological features. Morphological features can refer to characteristics within an image or 3D CAD rendering that relate to the shape and structure of an object. These features can be advantageous for tasks such as image segmentation, object detection, and feature classification in machine learning models. Some embodiments can include one or more edges, corner and intersection points, ridges and valleys, skeletons, convex hulls, Euler numbers, texture features, area, perimeter, and compactness, aspect ratio, orientation, circularity, and eccentricity, which will be discussed in detail below.

[0047] Edges are the boundaries or outlines of objects in an image. Edge detection is advantageous for distinguishing different regions of an image based on sudden changes in intensity. Corner and intersection points are points in a CAD file where two or more edges intersect. These features are often critical to understanding the geometry of an object in an image. Ridges are curved structures where the intensity of an image increases in multiple directions, while valleys are areas of decreasing intensity, which can be useful for terrain modeling. The skeleton of a shape is a simplified version of that shape that is equidistant from its boundaries. Skeletonization processes are helpful for reducing image detail to preserve the general shape of an object.

[0048] For example, some embodiments include a specific input configuration that parses out the skeleton and contour features, and then parses the contour features to display curvature of flat faces or surface shading. This is similar to a form of style transfer. Style transfer is used to modify the overall visual appearance of a texture. In essence, this task allows the content of one image to be recombined in the style of another image. Some embodiments include custom segmentation to distinguish specific parts or elements of an image. Specifically, custom segmentation is performed at the pixel level - the goal is to assign each pixel in an image to the object it belongs to (i.e. class label). Thus, the structural components or contour information should be classified and segmented, and then the contour information can be converted from line-frame contour data to surface-shading style data through a style transformation. In some embodiments, the frame and contour are custom segmented, and then shading is overlaid on each surface with an independent plane. In some embodiments, the data can be pre-processed by converting the CAD model to a compatible format such as a voxel grid or point cloud, and resizing the output image to be uniform in size.

[0049] A convex hull is the smallest convex shape that can contain an object in an image. The convex hull is advantageous for object recognition and shape analysis by providing a boundary within which all other points of an object lie. The Euler number is a scalar value that describes the topological structure of an image. The Euler number provides the number of objects in an image minus the number of holes in those objects, which is advantageous for characterizing the connectivity properties of an image.

[0050] Texture features describe the arrangement and distribution of pixels in an image region. Texture analysis can include measures of smoothness, coarseness, and regularity. Area, perimeter, and compactness features are basic geometric features that describe the size and shape of an object. For example, compactness compares the area of an object to the area of a circle with the same perimeter, giving an idea of how round the object is. Aspect ratio is the ratio of the width to the height of the bounding box of an object. Aspect ratio is useful for distinguishing objects that are similar in shape but different in orientation.

[0051] Orientation refers to the angle of an object in an image. This feature is useful for understanding the position of an object relative to other objects or the image frame. Roundness and eccentricity features describe how much an object deviates from being circular. Eccentricity measures the ratio of the distance between the foci of an ellipse to the length of its major axis, which is useful for understanding the elongation of an object.

[0052] In some embodiments, these morphological features can be extracted using specific techniques in image processing, such as thresholding, edge detection algorithms (e.g., Sobel, Canny), and morphological operations such as dilation, erosion, opening, and closing. By analyzing these features, the training module 110 can accurately interpret and classify objects in images, enhancing applications in various fields, such as medical imaging, automated quality control, and even complex tasks such as converting CAD models into detailed hatching renderings.

[0053] Predictive model architecture

[0054] As mentioned above, in some embodiments, the training module 110 includes one or more Convolutional Neural Networks (CNNs). Given the proficiency of CNNs in handling image data, the trainer 110 includes a specially designed CNN that is not only able to recognize different parts of a 3D CAD file but also to understand the underlying 3D structure implied by a 2D projection. In some embodiments, the training 110 includes a generative model, including a GAN, which is advantageous for generating high-fidelity hatching renderings from processed CAD data. Some embodiments can include a Conditional Generative Adversarial Network (cGAN) to generate images that correctly apply learned hatching techniques to new drawings. For example, the “condition” can be the surface type and the desired hatching, which influences the output of the generator. In some embodiments, autoencoders and / or U-Net can be employed to perform tasks such as denoising or segmentation before the final illustration is generated, if needed, which will be discussed in detail below.

[0055] In some embodiments, the training module 110 can split the global dataset into a training set, a validation set, and a test set. The training set is used to train the CNN, the validation set is used to tune hyperparameters and evaluate model performance during training, and the test set is used to evaluate the model’s performance on unseen data.

[0056] In some embodiments, the training module 110 includes a classification layer for distinguishing different types of surface treatment— determining where hatching should be applied according to surface type. This is advantageous for training the model to replicate the annotations in the annotated examples. In some embodiments, the training module 110 includes a regression / segmentation layer. For example, based on the complexity of the object, additional layers can be used to predict the precise location of the hatching, which can include a regression or segmentation task.

[0057] In some embodiments, the training module 110 includes loss functions. These loss functions can utilize a combination of classification loss (such as cross-entropy) for surface type identification and pixel-level loss (such as mean squared error) for precise placement of hatching. In some embodiments, the training module 110 includes validation. For example, by using a split of data that is unseen during the training phase to validate the accuracy of the model and the model’s ability to generalize to new unseen CAD models.

[0058] In some embodiments, the training module 110 includes a feedback loop that incorporates user feedback on the initial predictions to optimize the model training, with a particular focus on areas where the model misclassified the surface type or placed the hatching incorrectly. In some embodiments, the training module 110 includes continuous learning. As more CAD models and annotations become available, the model is continuously retrained and fine-tuned to improve its accuracy and adapt to changes in new types of drawings or design standards. Thus, the training module 110 provides a machine learning model that not only accurately classifies different surfaces in a CAD model but also applies the appropriate hatching technique.

[0059] Model evaluation

[0060] In some embodiments, the system 10 advantageously provides evaluation and performance metrics on unseen data before deploying the model. This helps ensure that the model has good generalization capabilities and generates hatched outline renderings that meet the required standards. For example, the performance metrics can include visual quality that evaluates the clarity, accuracy, and adherence to industry standards for designing hatched outline renderings. In some embodiments, the performance metrics can include accuracy metrics (including Structural Similarity Index (SSIM), Mean Squared Error (MSE), or other metrics suitable for evaluating the fidelity of the generated images).

[0061] After the model is trained and validated, it is deployed into production where it can receive new CAD model inputs and generate corresponding shadow outline renderings through feature extraction 112, image generation 114, and output and delivery 116. The system 10 can be deployed as part of a Software as a Service (SaaS) platform where users can upload CAD files and receive back shadow outline renderings. Such a deployment requires a robust backend architecture to handle potentially large amounts of computation, especially when dealing with complex models and large files.

[0062] When a user uploads a new CAD file to the system 10 through a device 140, the process of generating a new rendering begins. The GUI 142 is designed to facilitate easy navigation and file management, ensuring a seamless user experience. After the CAD file is uploaded, the feature extraction 112 processes the input 3D CAD file to identify and classify key features and geometric data. The feature extraction 112 uses advanced algorithms to analyze and extract the salient details necessary for accurate rendering of the illustration.

[0063] The extracted features are then forwarded to the image generation 114. This module utilizes the trained machine learning models (like the training module 110) to convert the extracted data into detailed shadow outline renderings. For example, such renderings can be used for the submission of design patents. For example, the module 110 applies the learned techniques to replicate the necessary style and details required for the patent documentation.

[0064] The output and delivery 116 performs compliance processing and format adjustments on the generated shadow outline renderings. The output and delivery 116 ensures that these shadow outline renderings conform to the required specifications and formats of industry standards, for example, the required specifications and formats for design patent illustration submissions. These illustrations can then be downloaded or viewed directly on the system 10, which the user can easily access and review the generated output results through the device 140.

[0065] The system 10 is continuously improved through a user feedback loop. User feedback is an integral part of the continuous improvement process. Users can provide feedback directly on the platform through the GUI 142 after viewing the generated renderings. In some embodiments, this feedback, along with any manual corrections submitted by the user, will be collected and analyzed by the feature extraction 112 and image generation 114 to optimize and improve the accuracy and output quality of the model.

[0066] In some embodiments, the training 110 can include model updates to accommodate changes in CAD technology and standards, and to make periodic updates to the models in the feature extraction and image generation modules. These updates ensure that the system is able to keep up with the latest technological advancements and continue to meet the changing needs of users. By integrating the above embodiments, the AI-driven system 10 effectively bridges the gap between the complex data contained in CAD models and the clear, standardized visual format required for shadow contour feature rendering.

[0067] In some embodiments, the training 110, feature extraction 112, image generation 114 can include one or more CNNs, U-Nets, and / or GANs, which will be briefly reviewed below. CNNs are specifically designed for processing images and are particularly effective at identifying patterns and features in images. CNNs are composed of convolutional layers, pooling layers, and fully connected layers that help extract and learn important features from images. CNNs are primarily used for tasks such as image classification, where they identify and classify elements in an image. Because of the ability of CNNs' architecture to effectively capture spatial hierarchies, CNNs are effective at identifying patterns, textures, and other features in images. U-Nets are a type of CNN specifically designed for image segmentation tasks. They have a unique architecture that enables precise localization, making them ideal for tasks that require classifying every pixel in an image, such as medical image segmentation. Like standard CNNs, U-Nets do not generate new images themselves, but rather segment existing images into different class parts.

[0068] GANs consist of a generator and a discriminator, where the generator learns to create synthetic data that resembles real data, while the discriminator learns to distinguish between real and generated data. GANs are particularly popular for tasks such as image-to-image translation, style transfer, and data augmentation. Autoencoders are unsupervised neural networks used for tasks such as dimensionality reduction, feature extraction, and image conversion. Autoencoders learn to compress input data and then reconstruct it back to its original form. Variational autoencoders (VAEs) and denoising autoencoders can be used for image conversion tasks. Transformers, originally designed for natural language processing tasks, have also shown great potential in computer vision tasks. Vision transformers (ViTs) divide images into smaller blocks, linearly embed these blocks, and process them through a self-attention mechanism. They have been successfully applied to various image-related tasks, including image classification and image conversion.

[0069] Accordingly, in system 10, generating images by converting CAD models to silhouette renderings can in some embodiments involve integrating CNNs and / or U-Nets with other image generation networks. For example, in one embodiment, an autoencoder can be employed. For example, a VAE can generate new images by learning to encode inputs into a low-dimensional space and then decode back to output images. A CNN serves as the encoder, capturing important features from the CAD model.

[0070] In some embodiments, training model 108 and / or image generator 114 can include one or more GANs configured for image generation. Such GANs can include a generator network that generates images from random noise or direct feature conversion, aiming to generate outputs as close to real data (i.e., silhouette renderings) as possible. The GAN includes a discriminator network that attempts to distinguish between real silhouette renderings and those generated by the generator. In some embodiments, the GAN training process involves improving the generator based on feedback from the discriminator until the output of the generator is indistinguishable from actual silhouette renderings.

[0071] Accordingly, in some embodiments, feature extraction 112 uses a CNN or U-Net to analyze a CAD model and extract important features (e.g., edges, shapes, and related geometry and morphology). Image generation 114 can input these features into a generative model (e.g., a GAN or the decoder portion of an autoencoder) that is trained to create high-fidelity silhouette renderings from these features. System 10 effectively combines the analytical capabilities of CNNs and U-Nets with the creative generation capabilities of networks including GANs and / or autoencoders to generate new images that meet the standards required for silhouette renderings.

[0072] Some embodiments include training the generative portion of system 10, such as the decoder of a GAN and / or autoencoder, to generate silhouette renderings from extracted features. Such training can include training a GAN to generate silhouette renderings. For example, a dataset of CAD models and corresponding silhouette renderings would train a GAN to understand what an accurate silhouette rendering includes given certain features of a CAD model. In some embodiments, the generator learns to directly extract and use features during training. In other embodiments, a CNN is utilized to pre-process a CAD model to extract and compress features, which are then input into the generator, as described in detail above.

[0073] In some embodiments, the generator first generates images from a random noise vector (initially) or from a set of features extracted from CAD models (via a CNN). The goal is to generate images that look like corresponding silhouette renderings. The discriminator examines both real silhouette renderings in the dataset and the illustrations generated by the generator. The discriminator learns to distinguish between real and fake, and provides feedback to the generator. In some embodiments, the generator and discriminator are trained simultaneously. The goal of the generator is to fool the discriminator by improving its output, while the discriminator becomes better and better at recognizing fake images. Over time, the output of the generator should become increasingly close to real silhouette renderings.

[0074] In some embodiments, an autoencoder is trained to encode an input into a compressed latent space, and then decode that representation, converting it back into an output that matches the original input. To generate silhouette renderings, some embodiments employ the following training protocol. As with GANs, a dataset of CAD models and corresponding silhouette renderings is used initially. The encoder portion of the network compresses the CAD model into a latent representation. While it is typical for an autoencoder to learn to reconstruct the input image, in embodiments herein the encoder learns from the CAD model features. The decoder can be trained to accept the latent representation (derived from the CAD model) and generate a corresponding silhouette rendering.

[0075] During training, the encoder-decoder network treats the CAD model as input and the silhouette rendering as the target output. The loss is computed based on how close the output of the decoder is to the actual silhouette rendering, rather than how close it is to the original CAD model. Some embodiments employ various loss functions for visual tasks. For example, a loss function that preserves content and style, such as a combination of perceptual loss or mean squared error with adversarial loss (when combined with class-GAN features).

[0076] Such training processes are iterative, requiring multiple training cycles, where a cycle represents one complete pass through the entire dataset. To optimize performance, it is advantageous to tune hyperparameters, adjust learning rates, adjust the architecture of the neural network (number of layers, layer types, etc.), and other parameters. Some embodiments include validating the model using an independent dataset that did not appear during training, to ensure that the model generalizes well to new CAD models and is able to generate high-quality silhouette renderings.

[0077] Some embodiments include strategies to mitigate the large data requirements through data augmentation, transfer learning, synthetic data generation, and / or active learning. Data augmentation increases the effective size of the dataset by applying transformations such as rotation, scaling, and cropping to the training images. These transformations help the model to generalize better from fewer samples. Transfer learning utilizes a pre-trained model that has already been trained on a large dataset for a similar task and fine-tuned on the CAD dataset. This can significantly reduce the amount of data required. Active learning implements a strategy where the model identifies which new data points are most beneficial to learn from, thereby enabling the trainer 108 to selectively annotate more data.

[0078] Integrating feature vectors into GANs

[0079] In some embodiments, the system 10 integrates the outputs of the CNN and U-Net into a Generative Adversarial Network (GAN) to generate shadow outline renderings from CAD models. As mentioned above, the CNN and U-Net are initially trained to perform tasks such as identifying key features, segmenting components, or detecting outlines in 3D CAD models, often referred to as feature vectors. The CNN can identify specific feature vectors such as edges, textures, and shapes, while the U-Net can be used for more detailed segmentation tasks where model components need to be precisely delineated.

[0080] The output of the CNN / U-Net includes feature maps or segmentation images, which will be discussed further below. Feature maps are detailed representations that highlight important features detected in the input image. The segmentation images of the U-Net classify each pixel as belonging to a specific part or feature of the input CAD model.

[0081] In preparing the input for the GAN, the feature maps or segmentation outputs of the CNN or U-Net can need to be appropriately formatted for use as input to the GAN. For example, the data can be normalized, the feature maps flattened into a format that can be input into the GAN, or the maps can be encoded into a more compact form. The generator of the GAN must be designed to accept these processed feature maps as input. This design can include adjusting the input layer of the generator to match the size and structure of the feature maps. Adding additional dense or convolutional layers can generate more detailed outputs.

[0082] During the training phase, the generator uses the feature maps to generate images that resemble the target shadow outline renderings. The role of the discriminator is to distinguish between the generated images and the real shadow outline renderings. The generator learns to improve its output based on the feedback from the discriminator, with the goal of generating images that the discriminator cannot easily distinguish from real illustrations.

[0083] Optimization and fine-tuning

[0084] In some embodiments, the CNN and / or U-Net and GAN can be jointly fine-tuned in an end-to-end manner after initial separate training. This means adjusting the weights of both the feature extractor (CNN or U-Net) and the generator based on the overall performance of the generated shadow outline renderings. This joint training can help better integrate feature extraction directly with generation, enabling the network to adapt more holistically. In some embodiments, the loss function can be customized to consider both the accuracy of feature extraction and the quality of image generation. For example, by using a combination of classification loss (from the CNN / U-Net output) and adversarial loss (from the GAN).

[0085] In some embodiments, the system 10 includes integration of 3D CAD models and 2D engineering views of 3D CAD models to facilitate the generation of additional data points for training machine learning models. For example, preparing a 3D CAD model of a physical object can provide a comprehensive spatial structure of the physical object, providing insight into the depth, perspective, and complex geometry of the physical object. As a complement to the 3D CAD file, the system 10 can generate six 2D engineering views, covering six standard perspectives (front, back, top, bottom, left, right). These views help understand how to represent different features of the object in planar format from multiple angles, providing advantageous visual information for accurate modeling.

[0086] In some embodiments, the method of training the prediction model includes a 3D / 2D data correlation phase, which involves developing complex algorithms designed to map identifiable features in the 3D CAD model to corresponding elements in the 2D engineering views. Such a feature mapping process can focus on precisely locating key points, edges, and surfaces in the 3D model and aligning these key points with their counterparts in the 2D views. In addition, projection mapping techniques are also employed to correlate depth information in the 3D model with the contours and features visible in the 2D views, ensuring that the AI model can accurately interpret which elements in the 2D drawings correspond to raised or recessed areas in the 3D object.

[0087] In some embodiments, the training phase described above can integrate 3D model data and related 2D views as paired training data into the AI model. This approach enables the model to learn how depth and surface variations are depicted between different views, thereby improving its prediction accuracy for new illustrations. The model architecture is designed as a hybrid neural network, utilizing CNNs to extract features from 2D views and 3D convolutional networks to manage spatial data in 3D models. The training goal is to generate final 2D shadow outline feature illustrations that accurately reflect depth and design details through detailed contour lines and surface shading. Such shadow outline feature illustrations can be used, for example, for the submission of design patent applications.

[0088] In some embodiments, image generation post-training includes applying learned correlations to new CAD files, incorporating both 3D views and their respective 2D views simultaneously to generate detailed shadow outline feature illustrations. This stage demonstrates the application of model training, using established feature mappings and depth correlations to accurately predict and render features and depths that conform to shadow outline feature rendering standards.

[0089] The optimization process involves applying additional algorithms post-image generation to adjust line weights, enhance shading, and ensure all illustrations conform to required industry drawing standards. The verification process employs a series of tools or manual checks to verify that generated illustrations are both formally and functionally faithful representations of the original CAD model.

[0090] Modules of system 10 can be adapted to additional training correlations. For example, data management 108 can be extended to efficiently handle and store 3D models and multiple 2D engineering views. Feature extraction 112 can be enhanced to perform complex mappings between 3D features and their 2D representations. Training 110, adjusted, can handle multi-input training sessions involving both types of data. Image generation modules are configured to leverage sophisticated algorithms that incorporate depth and outline information derived from 3D / 2D data correlations.

[0091] Through this integrated approach, Al systems are able to create highly accurate and detailed shadow outline feature renderings that are technically informative and compliant with patent application requirements. This approach significantly enhances the model's ability to generate reliable drawings, which are critical for protecting intellectual property, while simplifying a process that has traditionally been manual and labor-intensive.

[0092] Data structure for training predictive models

[0093] Reference is now made to Figure 4 and in conjunction with Figure 1 To FIG. 3, in some embodiments, integrated data correlation 119 includes generating Al data structure 400. In some embodiments, data structure 400 is configured for efficiently correlating 3D CAD with 2D engineering views, thereby improving the efficiency and accuracy of the trained and predictive models of the above-described embodiments. In some embodiments, system 10 includes an efficient and accurate data structure 400 for correlating 3D CAD models with 2D engineering views, which involves designing a structure capable of efficiently handling and integrating complex spatial and visual data. Data structure 400 facilitates fast access and manipulation of data for both training machine learning models and generating output illustrations. In some embodiments, integrated data correlation 119 includes generating and outputting data structure 400, as described below:

[0094] In some embodiments, the data structure 400 can include contour feature data 402 that maintains a direct association between 3D objects and their respective 2D views in a manner that is easily accessible and manipulatable by the AI model and system 10 components (e.g., 108, 110, 114, 116, 118, and / or 119). In some embodiments, the contour feature data 402 includes one or more bits corresponding to a 3D model representation that store a pointer to the complete 3D CAD model data of the corresponding physical object. In some embodiments, such data can include a format that includes vertices, edges, faces, and metadata about the material and texture of the object (e.g., 200).

[0095] In some embodiments, the data structure can include a 2D view array 404. The 2D view array 404 can include an array or list of 2D view objects, where each object represents a standard engineering view (front view, back view, top view, bottom view, left view, and / or right view). These lists and arrays can store pointers to locate more data. The data structure 400 can include a feature linking table (FLT) 406 that maps specific features in the 3D model of the object 200 to their representation in each 2D view (e.g., 202, 204, 206, 208, 210, and / or 212). Figures 3A-3G ).

[0096] In some embodiments, the data structure 400 can include structure values 408. The structure values 408 can include one or more of the following: a feature ID that corresponds to a unique identifier for each feature in the 3D model; 3D coordinates that correspond to the spatial coordinates of the feature in the 3D model; and a 2D coordinate mapping that maps the 3D feature to respective coordinates in the individual 2D views. In some embodiments, this mapping can be structured as a dictionary or a set of key-value pairs, where the key is the view identifier and the value is the coordinate pair or bounding box. For example, implemented through linked lookup tables. In some embodiments, these lookup tables can be linked remotely through the resources 120 or stored by the server 102 and / or data management 108.

[0097] In some embodiments, the data structure 400 can include depth data 410. The depth data 410 can include a depth information layer or depth map to enhance the 2D views with depth information taken from the 3D model, which is advantageous for accurate shading and contouring. In some embodiments, the depth data 410 can include a pointer to a depth map that can include a grid or matrix associated with each 2D view. Such a grid or matrix indicates the depth of the feature at individual points, thereby facilitating the application of the correct shading techniques in the output illustrations.

[0098] In some embodiments, the data structure 400 is operable to integrate with neural networks used by the system 10, such as CNNs, U-Nets, and / or GANs. The data structure 400 can be input directly into the neural networks of the system 10, for example, through the data management 108 and / or the integrated data association 119. The neural networks of the system 10 are trained to understand and process the relationship between the 3D features described above and their 2D projections. In some embodiments, with the depth data 410 and the data structure 400, a CNN and / or U-Net can be adjusted to process 2D views with associated depth maps as multi-channel input images, where one channel represents visual data and another channel represents depth data. Such functionality can be controlled through the multi-channel bit 412.

[0099] As will be discussed in further detail below, in some embodiments, the data structure 400 is operable to optimize efficiency and accuracy through indexing, normalization, and / or batching. For example, it is advantageous to implement indexing on the FLT 406 to speed up the querying and data retrieval during the training and generation phases. Such indexing creates searchable database indexes for one or more columns (attributes) in the table, such as frequently queried feature identifiers or keys. The searchable attributes enable the database system, such as the server 102, to find data associated with these keys faster than scanning every row in the table, which in many cases can reduce the time complexity from linear to logarithmic.

[0100] In complex queries involving multiple joins, or in cases where a particular subset of data is frequently accessed, such as matching 3D features with 2D features of common shapes, indexing has further advantages. By reducing the time spent searching and retrieving data, indexing minimizes the use of CPU and memory. This optimized resource utilization is advantageous when processing large datasets to train predictive models with CAD models and engineering drawings, as described herein. As the dataset grows with the use of more CAD files and engineering views, the advantage of indexing becomes increasingly significant, enabling the system 10 to scale more efficiently.

[0101] For real-time processing, such as in the image generation process where speed is of the essence, having an indexed table means that the feature data needed to generate an accurate shadow contour feature illustration can be retrieved almost instantaneously. In the training phase, when processing large amounts of data, indexing can significantly reduce the overhead of repeatedly fetching relevant data, thereby speeding up the training process.

[0102] Further, the index helps maintain data integrity by ensuring that each entry is unique according to the index attribute, which is advantageous when entries are frequently updated or modified during the training phase. By facilitating faster and more accurate data retrieval, the index reduces the likelihood of errors that can arise from data mapping errors or long data acquisition operations. The instant feedback enabled by fast data retrieval supports faster adjustments and optimization of algorithm and model parameters.

[0103] In some embodiments, the normalization allows for storing all 3D and 2D coordinates in a normalized format to reduce computational overhead and improve learning efficiency of the model. Further, in some embodiments, the data structure 400 is configured for batch processing, where multiple instances of the data structure 400n are batched together for parallel processing.

[0104] For example, during the training process, the CNN / U-Net / GAN model can use the FLT 406 in the data structure 400n to learn how the features represented in the 3D model appear in different 2D views under various transformations and projections. The depth data 410 with the depth information layer can be used to train the CNN / U-Net / GAN model on how to apply contour feature shading based on the depth of the features, thereby improving the realism and technical accuracy of the generated shaded contour renderings. This structured approach not only improves the accuracy of the AI model in associating 3D models with their corresponding 2D views, but also improves the efficiency of the training process by providing well-organized and easily accessible data. Further, the data structure 400 supports scalability and flexibility, which can accommodate various types of CAD models and / or engineering drawings.

[0105] In some embodiments, the data structure 400 can be used in a GAN network to output predicted images based on new 3D or 2D CAD models input by a user through the user device 140. For example, the data structure 400 of the embodiments herein is structured to associate 3D CAD models with their corresponding 2D engineering views, which not only facilitates training of the neural network but also facilitates integration into a Generative Adversarial Network (GAN) framework. This structure can significantly enhance the ability of the GAN to generate accurate and detailed predicted shaded contour feature illustrations from new CAD models input by a user.

[0106] For example, the system 10 integrates the data structure 400 with a GAN during the pre-processing and input preparation (particularly feature extraction) phase. In some embodiments, when a user uploads a new 3D CAD model or a set of 2D views through the device 140, the system 10 can process the new input using, for example, the FLT 406 and the depth data 412 with the depth information layer. This operation involves identifying salient features in the input CAD model and mapping the identified features to corresponding representations in the 2D engineering views.

[0107] In some embodiments, simultaneously or substantially simultaneously, a depth map 402 is generated or updated for each 2D view based on the 3D model to indicate the relative depth of different features. This is advantageous for realistic shading in the illustrations. Next, the processed features and the depth map 402 are formatted as input to the GAN (e.g., through the modules 108, 112, 114, and / or 119). In some embodiments, this formatting can include structuring the data into channels 404, with one set of channels 404a carrying 2D view information and another set of channels 404b carrying depth information. In some embodiments, normalization and scaling can further ensure that all data input to the GAN matches the input requirements of the network, thereby maximizing the effectiveness of feature learning and image generation through the feature extraction 112 and the image generation 114, respectively.

[0108] In some embodiments, the training module 110 can include a generator modification (GM). For example, the GM can adapt the generator of the GAN to accept such structured data (e.g., 600) as input. The GM can enhance the architecture of the generator to process multiple types of data (e.g., visual features from 2D views and depth data) simultaneously and effectively. Some embodiments can include a discriminator enhancement by modifying the discriminator to not only assess the realism of the generated images but also to assess the technical accuracy of the generated images based on how the generated images integrate the depth and feature information obtained from the 3D model and the 2D views.

[0109] In some embodiments, the training module 110 can train the GAN with training data, leveraging a mix of historical data in the data structure 500 and real-time processed data to train the GAN. This helps the network learn a comprehensive range of feature representations and depth variations, thereby enhancing its ability to generalize from new CAD model inputs. Some embodiments include a feedback loop mechanism via the feedback manager 117, where the output of the GAN is checked against the expected feature and depth specifications in the data structure 400. Any discrepancies can be used to fine-tune the performance of the generator.

[0110] Generating predictive shadow contour renderings

[0111] Once the GAN model is trained, the GAN model can directly generate predictive shadow outline feature illustrations (e.g., FIGS. 3a-3h) from new CAD model inputs and / or data provided by the CNN / U-Net discussed herein. As described above, the generator uses structured inputs to create detailed 2D engineering views that accurately reflect the geometric and depth features of the objects depicted in the 3D CAD models (e.g., object 200) that are mapped and structured in data structure 400.

[0112] Some embodiments can include post-processing by applying additional image processing to optimize the illustrations and continually update the FLT 406 and depth map 410 in data structure 400. In some embodiments, feedback manager 117 manages all user feedback received from device 140. Any discrepancies between the generated images and the actual CAD models are recorded and updates are provided to data structure 400. This post-processing and iterative optimization improves the accuracy and reliability of the generated shadow outline feature renderings.

[0113] This customized approach of integrating specialized data structures into the GAN framework not only simplifies the process of generating shadow outline renderings, but also significantly improves the quality and accuracy of the output. By leveraging the detailed features and depth data in 3D and 2D CAD models, system 10 can output highly detailed and technically accurate shadow outline feature renderings, which are advantageous for documents such as appearance design patent applications. This integration demonstrates the power of implementing advanced data structures within an AI framework to solve complex real-world problems.

[0114] In some embodiments, this data structure 400 can be a Python-based data structure. Python-based data structures can have advantages in handling the complex relationships between 3D CAD models and their corresponding 2D engineering views. In some embodiments, the Python-based data structure in the GAN used to generate shadow outline renderings involves designing classes that can encapsulate all necessary attributes and methods.

[0115] For example:

[0116]

[0117]

[0118]

[0119]

[0120]

[0121] Accordingly, the above-described Python-based data structure and processing facilitates the establishment of a structure for processing and correlating data between 3D and 2D representations, and which is configured to be extended with actual data processing and neural network integration to generate a shadow outline rendering.

[0122] This data structure 400 can be used to manage and correlate 3D CAD models and their corresponding 2D engineering, which has advantages for optimizing the computational efficiency and accuracy of the generated shadow outline rendering. The following is a summary of how this data structure enhances the functionality of a computer system in training a neural network, particularly in terms of the following processing resources, memory capacity, and rendering accuracy:

[0123] As the data structure 400 organizes data in a way that directly correlates features and corresponding views, this organization enables quick access to relevant data when needed, thereby reducing the time spent searching in unstructured data collections. By structuring data to support batch processing, where similar types of data are processed together, as described above, the system 10 can take advantage of vectorization operations and parallel processing capabilities. This is particularly effective in reducing processing time for the training and inference phases of machine learning models. Establishing a structured correlation between 3D features and their 2D representations means that once a certain feature or depth map has been processed, this data can be reused in different tasks without the need for recalculation. This minimizes redundant calculations, thereby saving processing resources.

[0124] The data structure 400 facilitates memory optimization through selective loading and on-demand loading. The data structure 400 allows components of the system (e.g., 108-119) to load only the necessary data blocks into memory when needed. For example, if a certain 2D view or feature is not relevant to the current processing task, that feature can remain unloaded, thereby saving memory. Compact storage: By organizing features and corresponding mappings in a structured format (e.g., FLT 406, array 402), data can be stored more compactly compared to loose or unstructured formats. Efficient data encoding and compression techniques can further reduce the use of memory.

[0125] The data structure 400 improves the accuracy of rendering through precise feature mapping. Detailed mapping of features of a 3D model into its 2D projections ensures that important details are not lost in the conversion process. This has advantages when the rendering must meet strict standards for shadow outline rendering images. The inclusion of depth maps in the data structure enables the rendering process to incorporate accurate shadows and outlines based on the spatial information of the 3D model. This adds realism and technical accuracy to the 2D illustrations, which is advantageous for industrial applications.

[0126] By maintaining consistent format and association between 3D and 2D data, the system 10 ensures that all conversions and renderings are based on the same baseline data. This consistency helps maintain accuracy between different views and renderings. The structured nature of the data facilitates robust checking and verification to ensure that the mapping and rendering are correct. Discrepancies can be quickly identified and corrected, which is advantageous for training machine learning models to produce reliable outputs.

[0127] Furthermore, due to the structured data format, machine learning models can focus on learning the most relevant features and their transformations, while irrelevant data can be easily excluded from the training process. This targeted learning improves the efficiency and effectiveness of the models. For example, by having quick access to structured and relevant data, the models can be iteratively trained and fine-tuned more quickly. This speeds up the model development cycle and allows for quicker adjustments based on performance feedback.

[0128] Thus, employing the data structure 400 not only improves the efficiency and capacity of the computing resources of the system 10, but also significantly enhances the accuracy and reliability of the generated silhouette rendering of the physical object that is output to the user device 140. This dual advantage of operational efficiency and technical precision is particularly valuable in, for example, generating appearance design patent illustrations, where both accuracy and processing efficiency are of utmost importance.

[0129] Reference is now made to the following drawings wherein Figures 5 to 7 and which is incorporated herein by reference Figures 1 to 4 , Figure 5 A flowchart of a method 500 in accordance with one or more of the above-described embodiments is shown. The method 500 includes the following operations: at operation 502, receiving a 3D model input (200) corresponding to a physical object (202); at operation 504, generating, based on the 3D model input, a data structure (400) comprising one or more features (402) of the physical object, and one or more 2D renderings (300) of the physical object; at operation 506, associating the one or more features with the one or more 2D renderings of the physical object; at operation 508, determining a silhouette rendering of the physical object based on the one or more features; and at operation 510, transmitting the silhouette rendering of the physical object to a display device. In some embodiments, operations 502-510 can be performed by modules 108-119.

[0130] Figure 6A flowchart of method 600 according to one or more embodiments described above is shown. Method 600 includes the following operations: in operation 602, receiving a three-dimensional model of a physical object; in operation 604, determining one or more two-dimensional views of the physical object based on the three-dimensional model; in operation 606, associating one or more feature vectors of the three-dimensional object with one or more two-dimensional views; in operation 608, outputting a data structure including one or more features; in operation 610, training a first prediction model using the data structure; and in operation 612, training a second prediction model using the data structure. In some embodiments, operations 602 to 612 may be performed by modules 108 to 119.

[0131] Figure 7 A flowchart of method 700 according to one or more of the above embodiments is shown. Method 700 includes the following operations: in operation 702, providing a computer-implemented system including a memory storing a data structure configured to associate features between a 3D CAD model and corresponding 2D engineering views, and a processor coupled to the memory operation; in operation 704, populating the data structure in the memory with model data including spatial structure and 2D data including standard views depicting the object from different angles; in operation 706, associating feature data to associate features in the 3D model with corresponding 2D views; and in operation 708, outputting the data structure. In some embodiments, operations 702 to 708 may be performed by modules 108 to 119.

[0132] See Figure 8 In some embodiments, system 10 employs a U-Net architecture to train a prediction model to output shadow contour feature rendering. This U-Net architecture may include two main paths: an encoder path 804 and a decoder path 806, both cleverly designed as a feature pyramid network (U-Net) 800 with strategically implemented skip connections 808. Figure 8 As shown, encoder path 804 of U-Net 800 systematically downsamples the image through a series of convolutional and pooling layers. This sequential reduction is used to refine the input into a form in which high-level features are extracted while spatial dimensions are minimized. Encoder path 804 is beneficial for separating important features from CAD model input, such as edges, textures, and different geometric patterns, which are advantageous for subsequent feature extraction stages (e.g., 112).

[0133] In contrast to the encoder, the decoder path 806 reconstructs the segmented output back to the resolution of the original input image. This is achieved by progressively upsampling the compressed feature maps, thereby gradually recovering the detailed structure of the image. The upsampling process is carefully designed to optimize the output, ensuring that the generated shadow outline feature rendering is not only accurate but also meets the stringent details required by industry submissions.

[0134] In one embodiment, the overall feature of the U-Net design is the inclusion of skip connections 808 that span corresponding layers of the encoder path and the decoder path. These connections have an advantage in preserving and passing on the fine-grained details that might otherwise be lost during the downsampling process. In some embodiments, by directly connecting the feature maps of the encoder to the corresponding decoder layers, the skip connections 808 ensure that both local details and global contextual information are preserved, thereby improving the fidelity and accuracy of the reconstructed image.

[0135] In the system 10, the U-Net architecture 800 is integrated to leverage its advanced segmentation capabilities. This integration facilitates the accurate mapping of the complex geometry of 3D CAD models into their corresponding 2D engineering views in the feature extraction module 112 (as described above). The efficient handling of local and global features by the U-Net 800 ensures that the final image output is of high quality, accurately aligned, and clearly presents the complex details of the CAD model, which is crucial for the legal robustness of the output rendering.

[0136] Therefore, the system 10 represents a significant advancement in the field of computer-aided design (CAD) and shadow outline feature map generation. By leveraging advanced machine learning techniques, including CNN / U-Net architecture and GAN, as well as sophisticated data structure techniques such as indexed feature linking tables, the embodiments herein effectively bridge the gap between complex 3D CAD models and their required 2D shadow outline illustrations. This not only eliminates the traditionally manual and labor-intensive process but also improves the accuracy and precision of the final rendering.

[0137] The integrated modules 108-119 work in synergy to ensure a streamlined, efficient, and user-friendly experience. Together, the modules 108-119 manage complex data processing, feature mapping, image synthesis, and final illustration formatting and delivery, making the system highly scalable and adaptable to various industrial needs and evolving technological environments. Furthermore, the system is capable of learning from historical data and continuously improving through user feedback and regular model updates, positioning it at the forefront of AI-driven design technology. This enables the process and methodology to be continually optimized, ensuring that the system remains relevant and effective in meeting the stringent requirements of patent documentation.

[0138] The AI-driven solution of the embodiments herein not only simplifies the creation of renderings applicable to design patent illustrations, but also significantly reduces the time and resources required for their production, providing a competitive advantage to patent applicants and contributing to more robust intellectual property protection. As such, the technology has the potential to revolutionize patent drafting practices and bring far-reaching benefits to industries that rely on patent protection to safeguard their innovations.

[0139] The embodiments described herein can be embodied in systems, apparatus, methods, computer programs, and / or articles depending on the desired configuration. Any method or logic flow illustrated in the figures and / or described herein does not necessarily have to be executed in the particular order shown or sequentially. Implementations presented in the foregoing description are not meant to represent all implementations consistent with the subject matter described herein. Rather, they are merely some examples consistent with aspects related to the described subject matter. Although some variations have been described above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and sub-combinations of the disclosed features and / or combinations and sub-combinations of one or more additional features not expressly described above. In addition, the features and / or variations described above can be directed to any one of the disclosed implementations or any combination of the disclosed implementations. Further, to the extent that any patents or publications are identified herein, the disclosures of these patents and publications in their entireties are hereby incorporated by reference into this description, but only to the extent that the above-described implementations actually conflict or are inconsistent with such patents and publications.

[0140] Furthermore, to the extent that any clause of the claims herein is determined to be invalid or unenforceable under any applicable law in any jurisdiction, such a determination does not impact the validity or enforceability of any clause of the claims in any other jurisdiction. Implementations of the subject matter have been described in terms of particular implementations. Other implementations have been suggested, and these can be made only by persons skilled in the art, using, no more than routine skill and knowledge. The disclosed subject matter can be implemented in any of various forms, including as one or more computer program products, which can contain machine-readable media storing instructions, which, when executed by a machine, can cause the machine to perform any of the operations described herein. Any implementation suggested herein can be implemented only using software or only using hardware, including but not limited to analog and / or digital hardware, or using combinations of software and hardware, some or all of which can be specifically designed to perform the functions described herein. The term "program" as used herein means any type of computer program, software, firmware, or the like, and includes routines, subroutines, functions, methods, modules, programs, or the like.

[0141] In the claims, any reference signs placed between any of the claims presented herein should not be construed as a limitation of the claims. The word comprising or including does not exclude the presence of elements or steps other than those listed in a claim. In a device claim enumerating several means, the enumerated means can be considered selective rather than exhaustive unless the context clearly dictates otherwise. The word "comprise", or variations such as "comprises" or "comprising", will not be taken to exclude the presence of elements other than those listed in the specific claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The word "but" does not exclude the presence of additional such elements or steps. The word "only" does not exclude the presence of additional such elements or steps. The word "each" preceding a list of two or more elements, does not exclude the presence of only one such element. It is further to be understood that different implementations can include different combinations of the disclosed features or steps.

[0142] While the foregoing description provides details in accordance with presently-preferred embodiments, it is to be understood that these details are provided for the purpose of explanation only and are not intended to limit the disclosure in any manner to the expressly disclosed embodiments. Rather, it is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it is to be understood that the disclosure contemplates that one or more features of any embodiment can be combined with one or more features of any other embodiment.

Claims

1. A system (10) for converting 3D models to 2D silhouette renderings in real-time, the system comprising: a processor (104) in communication with a memory (106) storing executable instructions, when the processor executes the executable instructions, the system is configured to: receive a 3D model input (200) corresponding to a physical object (202), generate, based on the 3D model input, a data structure (600) comprising one or more features (402) of the physical object and one or more 2D renderings (300) of the physical object, associate the one or more features with the one or more 2D renderings of the physical object; determine, based on the one or more features, a silhouette rendering of the physical object (301); transmit the silhouette rendering of the physical object to a display device (140).

2. A computer-implemented method comprising: receiving a three-dimensional model of a physical object; determining one or more two-dimensional views of the physical object based on the three-dimensional model; associating one or more feature vectors of the three-dimensional object with the one or more two-dimensional views; outputting a data structure comprising the one or more features; training a first predictive model with the data structure; and training a second predictive model with the data structure.

3. A method for generating a data structure for training predictive models, comprising: providing a computer-implemented system, the system comprising: a memory storing a data structure configured to associate features between 3D CAD models and corresponding 2D engineering views; and a processor operably coupled with the memory; populating the data structure in the memory with model data and 2D data, the model data comprising spatial structures and complex geometries, the 2D data comprising a plurality of standard views depicting an object from a plurality of angles; associating feature data to correlate features in 3D models to corresponding 2D views; and outputting the data structure. ​

Images