Systems, methods, and data structures for mapping 3D objects to 2D shaded contour rendering.

Abstract

Embodiments described herein relate to a system for converting a 3D model to a 2D shaded contour rendering in real time. The system comprises a processor that communicates with memory. The memory receives a 3D model input corresponding to a physical object when an executable instruction is executed by the processor and stores executable instructions that constitute the system for generating a data structure including one or more features of the physical object and one or more 2D renderings of the physical object based on the 3D model input. The processor correlates one or more features with one or more 2D renderings of the physical object and further configures the system for determining a shaded contour rendering of the physical object (301) based on one or more features. The system is configured to transmit the shaded contour rendering of the physical object to a display device.

Description

TECHNICAL FIELD

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63 / 502,416 filed on May 16, 2023, the content of which is incorporated herein by reference in its entirety. BACKGROUND ART

[0002] Contour and surface shading rendering are fundamental components of product design documentation that are necessary for clearly demonstrating design features in a format compliant with industrial and manufacturing standards. Current methods for creating contour and surface shading rendering have challenges. Manual drafting is not only time-consuming but also prone to human error, leading to potential inconsistencies in the interpretation of complex CAD models. Several algorithmic approaches are inaccurate and do not provide predictable results. Such approaches require significant investment of time and resources, which becomes a costly problem for entities that perform frequent rendering. SUMMARY OF THE INVENTION

[0003] Accordingly, some embodiments include a system for converting a 3D model to a 2D shaded contour rendering in real time. The system includes a processor that communicates with memory. The memory stores executable instructions that, when executed by the processor, constitute the system for receiving a 3D model input (200) corresponding to a physical object. In some embodiments, the system is configured to generate a data structure based on the 3D model input that includes one or more features of the physical object and one or more 2D renderings of the physical object. In some embodiments, the system correlates one or more features with one or more 2D renderings of the physical object and determines a shaded contour rendering of the physical object based on one or more features. The system is configured to transmit the shaded contour rendering of the physical object to a display device.

[0004] Some embodiments include a computer implementation method comprising the steps of receiving a three-dimensional model of a physical object and determining one or more two-dimensional views of the physical object based on the three-dimensional model. In some embodiments, the method includes the steps of correlating one or more feature vectors of the three-dimensional object with one or more two-dimensional views and outputting a data structure containing one or more features. In some embodiments, the method includes the steps of training a first predictive model using the data structure and training a second predictive model using the data structure. In some embodiments, the method includes the step of generating a predictive rendering using the data structure.

[0005] To allow for a more detailed understanding of the above-mentioned features of this disclosure, a more detailed description of this disclosure, which is concisely summarized above, can be obtained by referring to embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings illustrate only some aspects of this disclosure, and this disclosure may allow for other equally valid embodiments.

[0006] For ease of understanding, the same reference numerals are used to indicate identical elements common to the figures, where possible. Elements and features of one embodiment may be usefully incorporated into other embodiments without further enumeration. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram of a system for mapping 3D objects to shaded contour rendering, according to several embodiments.

[0008] [Figure 2] This is a schematic diagram of a 3D object correlated with 2D shaded contour rendering, according to several embodiments.

[0009] [Figure 3A] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments. [Figure 3B] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments. [Figure 3C] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments. [Figure 3D] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments. [Figure 3E] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments. [Figure 3F] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments. [Figure 3G] This is a schematic diagram of a 3D object associated with a 2D view, according to several embodiments.

[0010] [Figure 4]This figure shows data structures for training neural networks and generating shaded contour renderings, according to several embodiments.

[0011] [Figure 5] This is a flowchart illustrating exemplary methods for training a neural network and generating shaded contour renderings, according to several embodiments. [Figure 6] This is a flowchart illustrating exemplary methods for training a neural network and generating shaded contour renderings, according to several embodiments. [Figure 7] This is a flowchart illustrating exemplary methods for training a neural network and generating shaded contour renderings, according to several embodiments.

[0012] [Figure 8] This is a schematic diagram of a neural network for 3D object correlation according to several embodiments. [Modes for carrying out the invention]

[0013] Next, the present disclosure will be described in detail with reference to the drawings, which are provided as exemplary examples of the present disclosure so as to enable those skilled in the art to practice the present disclosure. In particular, the following figures and examples are not intended to limit the scope of the present disclosure to a single embodiment, but other embodiments are possible by replacing some or all of the elements described or illustrated. Furthermore, where certain elements of the present disclosure can be implemented in part or in whole using known components, only the portion of such known component necessary for understanding the present disclosure will be described, and detailed descriptions of other portions of such known component will be omitted so as not to obscure the present disclosure.

[0014] Where used herein, the singular forms “a,” “an,” and “the” include plural references unless otherwise explicitly indicated by the context. Where used herein, the statement that two or more parts or components are “coupled” means that the parts are joined or operate together directly or indirectly (i.e., through one or more intermediate parts or components, insofar as a link is created). Where used herein, “directly coupled” means that the two elements are in direct contact with each other. Where used herein, “fixedly coupled” or “fixed” means that the two components are coupled so that they move together as a unit while maintaining a constant orientation relative to each other. Where used herein, “operably coupled” means that the two elements are coupled so that they function together. It should be understood that two elements that are “operably coupled” do not require a direct or permanent connection between the two elements. As used herein, “substantially” means that any difference is negligible, or that such difference is known to those skilled in the art and falls within an operating tolerance that provides the desired performance and results as described in one or more embodiments herein. Numerical ranges are described including their endpoints.

[0015] As used herein, the term “unitary” means that the component is made as a single part or unit. That is, a component that includes parts that are made separately and then joined together as a unit is not a “unitary” component or body. As used herein, the statement that two or more parts or components “engage” with each other means that the parts exert force on each other directly or through one or more intermediate parts or components. As used herein, the term “number” means one or an integer greater than one (i.e., plural). Directional terms as used herein, such as up, down, left, right, superior, inferior, front, back, and their derivatives, relate to the orientation of the elements shown in the drawings and do not limit the claims unless expressly stated in the claims.

[0016] Embodiments described as being implemented in hardware should not be limited to hardware, and may include embodiments implemented in software or in combination of software and hardware, as will be obvious to those skilled in the art, unless otherwise specifically specified herein. In the exemplary embodiments described herein, embodiments showing a single component should not be considered limiting; rather, the present invention is intended to encompass other embodiments including multiple identical components, unless otherwise specifically specified herein, and vice versa. Furthermore, the applicant does not intend that any term in this specification or in the claims be considered non-generic or special unless expressly stated as such. Furthermore, the present invention encompasses current and future known equivalents to known components mentioned herein as examples.

[0017] Embodiments described herein generally relate to real-time rendering of 3D objects via neural networks and prediction models for generating contour and surface shading renderings, systems, components thereof, and methods of using the same. For example, renderings that depict contour features through surface shading are an integral part of many industrial applications in design and manufacturing. For instance, in the case of injection molding manufacturing, such products are rendered in three dimensions (3D) via a CAD program. However, when presented in 2D, such renderings lose depth information and contour information. Accordingly, embodiments herein leverage computer-aided design (CAD) and artificial intelligence (AI) by using machine learning (ML) models to predict, in real time over a network such as the Internet, the conversion of CAD models to 2D renderings that include contour and surface shading renderings.

[0018] Accordingly, some embodiments herein provide systems, methods, and data structures for AI-powered solutions designed to address these and other inefficiencies. By integrating advanced neural networks, including convolutional neural networks (CNN), U-Nets, and generative adversarial networks (GAN), embodiments herein provide real-time conversion of 3D CAD models to 2D contour and surface shading renderings. The AI-driven approaches of embodiments herein ensure high accuracy and consistency, significantly reduce the time and cost associated with manual rendering, and minimize or eliminate errors. In some embodiments, the AI models of embodiments herein are trained on datasets of CAD models and corresponding contour and surface shading renderings, enabling the prediction model to learn and replicate required styles and regulatory requirements in real time over a communications network. As a result, embodiments herein provide a scalable, efficient, and low-effort alternative to other methods of preparing contour and surface shading renderings, thereby improving productivity and reducing operational costs.

[0019] Referring now to Figure 1, Figure 1 illustrates a practical application in a networked arrangement of an exemplary system 10 (hereinafter referred to as "system 10") for mapping three-dimensional (3D) objects to shaded contour rendering. As shown in Figure 1, in some embodiments, the system 10 may include a server 102, a resource 120, and a user device 140 having a GUI 142. In some embodiments, the user device 140 may include a smartphone, a laptop, a desktop, and / or any computing device that functions as an endpoint for a user, such as an engineer or illustrator, interacting with the system 10. The user device 140 facilitates the user uploading an input 3D CAD model and downloading the output generated by shaded contour feature rendering, which may be displayed on the GUI 142. As shown in Figure 1, the server 102, the resource 120, and the user device 140 may communicate with one another via a network 130. The architecture of the system 10 is configured to handle the processing of 3D CAD models of physical objects into shaded contour feature rendering, which is discussed in detail below.

[0020] In some embodiments, the server 102 includes a processor 104 that communicates with memory 106. Memory 106 may contain software code 105. The processor 104 is configured to receive and execute the software code 105 to implement one or more of the embodiments described herein. For example, the server 102 can execute the code 105 and cause the system 10 to output a predictive rendering of shaded contour feature rendering based on an input 3D CAD model file. Such output can be transmitted in real time to a remote client or user via the network 130. In some embodiments, the server 102 includes one or more modules for performing the corresponding functions of the embodiments described herein. For example, in some embodiments, the server 102 includes a data manager 108, training 110, feature extraction 112, image generation 114, output and distribution 116, monitoring and logging 118, and / or integrated data correlation 119, which are described in more detail below.

[0021] In some embodiments, the server 102 can communicate with external resources 120 via the network 130. In some embodiments, the network 130 may include a LAN / WAN connection configured to provide internet connectivity via, for example, a hybrid optical fiber (HFC) transmission network (e.g., Ethernet twisted-pair shielded cable CAT-5, WiFi, a premises coaxial cable network, or other connections that can establish internet connectivity). In some embodiments, the network 130 may include a wireless network that can establish internet connectivity (e.g., 5G, LTE, 4G, CDMA, etc.). The network 130 facilitates the utilization of external resources 120 to facilitate various functionalities, which will be described in more detail below.

[0022] In some embodiments, the external resource 120 may include access to remote databases and / or third-party API services that facilitate integration and interaction between the system server 102 and the remote client user device 140 with external systems and resources to enhance functionality. For example, resource 120 may facilitate connections with various third-party API services, enabling the system to leverage external tools and data sources. Such API services may include, but are not limited to, platforms that provide advanced AI processing capabilities and predictive analytics tools, which will be discussed in more detail below. In some embodiments, the external resource 120 may establish connections with one or more remote databases (not shown), which may be advantageous for enhancing the data processing and processing capabilities of system 10. By implementing the external resource 120, system 10 can advantageously expand the range of functionality, such as real-time data analysis, machine learning processes, and sophisticated predictive modeling, which will be discussed in more detail below. Such external resource 120 not only enriches the user experience by providing more accurate and efficient results, but also improves overall system performance by integrating virtualization techniques to streamline process efficiency, which will be discussed in more detail below.

[0023] One or more components of system 10 (e.g., device 140, processor 104, and / or modules 108, 110, 112, 114, 116, 118, 119) may be implemented as digital electronic circuits, integrated circuits, specially designed application-specific integrated circuits (ASICs), field-programmable gate array (FPGA) computer hardware, firmware, software, and / or combinations thereof. These various embodiments or features may include implementations in one or more computer programs (e.g., code 105) that are executable and / or interpretable on a programmable system including one or more programmable processors (e.g., 104) that may be special-purpose or general-purpose, coupled to receive and transmit data and instructions from a storage system (e.g., memory 106), at least one input device, and at least one output device. The programmable system or computing system may include a client (e.g., user device 140) and a server (e.g., 102). The client and server are generally geographically distant from each other and typically interact through a communication network (e.g., 130). The client-server relationship arises from computer programs running on each other's computers that have a client / server relationship with each other.

[0024] Such computer programs may also be called programs, software, software applications, applications, components, or code, and may include non-temporary machine-readable instructions for a programmable processor, and may be implemented in high-level procedural languages, object-oriented programming languages, functional programming languages, logic programming languages, and / or assembly / machine languages. As used herein, the term “machine-readable medium” (or “computer-readable medium”) means any computer program product, apparatus, and / or device, such as magnetic disks, optical disks, memory, and programmable logic devices (PLDs), used to provide machine instructions and / or data to a programmable processor that includes a machine-readable medium that receives the machine instructions as machine-readable signals. The term “machine-readable signals” (or “computer-readable signals”) means any signals used to provide machine instructions and / or data to a programmable processor. Machine-readable medium may store such machine instructions non-temporarily, such as non-temporary solid-state memory or a magnetic hard drive or any equivalent storage medium. Machine-readable media can, alternatively or additionally, temporarily store such machine instructions, for example, a processor cache or other random-access memory associated with one or more physical processor cores.

[0025] In some embodiments, the server 102 may include a data manager 108, training 110, feature extraction 112, image generation 114, output and distribution 116, monitoring and logging 118, and / or integrated data correlation 119. As will be described in more detail below, such modules 108-119 may function individually and / or in conjunction with each other to carry out the embodiments described herein. Functions described in the embodiments described herein as being performed by one module may, in other embodiments, be performed by different modules or divided among various modules. For example, a function described in one embodiment as being performed by the data manager 108 may, in another embodiment, be performed by the integrated data correlation 119 and / or monitoring and logging 118, and / or vice versa.

[0026] For example, in some embodiments, the data manager 108 manages all data-related operations, including the capture, storage, and preprocessing of CAD files and shaded contour renderings. The data manager 108 ensures data integrity and efficient access to the storage solution. In some embodiments, the data manager 108 manages data-related operations within the system 10, for example, by the capture, secure storage, and systematic preprocessing of CAD files and detailed shaded contour renderings.

[0027] In some embodiments, the data manager 108 automates the acceptance of new CAD files to ensure that such input files are correctly formatted and error-free. In some embodiments, the acceptance process may include validating the data structure of the incoming file against the requirements of system 10 and performing initial checks for data integrity and reliability.

[0028] In some embodiments, leveraging the integrated CNN / U-Net architecture discussed further below, the data manager 108 preprocesses the CAD files to extract features necessary for subsequent image generation. Such preprocessing includes applying morphological operations to enhance feature visibility and preparing the data for efficient feature extraction and segmentation by the feature extraction module 112.

[0029] In some embodiments, the data manager 108 organizes data storage to ensure that both raw and processed data are stored in a structured manner that facilitates rapid retrieval. Such organization can be achieved using optimized data indexing and partitioning strategies that improve data query performance and reduce data access latency. To ensure data security and integrity throughout its entire lifecycle, the data manager 108 implements robust encryption methods for data at rest and in transit. The data manager 108 can also maintain data integrity checks to prevent and correct data corruption or loss.

[0030] In some embodiments, the data manager 108 works in close cooperation with the feature extraction module (112), the image generation module (114), and the output and distribution module (116). The data manager 108 ensures that the data flow across these modules is seamless and that the data format and structure are consistently maintained to support an end-to-end process for generating shaded contour feature renderings. Such integration is advantageous for maintaining high efficiency and accuracy in generating detailed and standardized shaded contour feature renderings. In one embodiment, the data manager 108 can optimize the retrieval process to ensure that data access is fast and does not become a bottleneck, for example, when processing a large number of high-resolution CAD models and their corresponding illustrations.

[0031] In some embodiments, the training module 110 manages the training of a machine learning model using historical data to effectively learn how to create shaded contour renderings. The training module 110 advantageously continuously improves the accuracy and processing efficiency of the model. The training module 110 uses sophisticated training algorithms and cross-validation techniques to optimize model parameters and prevent overfitting. Regular update and retraining cycles ensure that the model conforms to new types of CAD design and evolving illustration standards, thereby improving the adaptability and long-term performance of the system 10.

[0032] In some embodiments, feature extraction 112 extracts essential feature vectors from a CAD model necessary to generate accurate shaded contour renderings. This process involves sophisticated algorithms designed to accurately identify and depict edges, textures, and other geometric features essential for detailed rendering. Advanced image processing techniques such as edge detection, texture analysis, and morphological transformations are utilized to improve the visibility and distinction of features, which is advantageous in subsequent stages of image generation.

[0033] In some embodiments, the image generation module 114 utilizes a trained model to convert processed CAD data into high-quality shaded contour renderings. The image generation module 114 integrates artificial intelligence techniques, including GANs, to synthesize images. Through iterative refinement and learning, the image generation module 114 continuously improves its ability to produce output that accurately represents the intended design attributes and specifications of the original CAD model.

[0034] In some embodiments, the output and delivery module 116 is responsible for the final stage, where the shaded contour rendering is formatted and delivered to the user's device 140. The output and delivery module ensures that each image is rendered in the correct format, conforms to the exact specifications required for industry submissions, and maintains the highest quality standards. The output and delivery module also manages the delivery channel to ensure the safe and efficient delivery of the illustration rendering to the user's device 140, while supporting various output formats and user-specific customization options.

[0035] In some embodiments, monitoring and logging 118 manages the performance and operational integrity of the system 10 by continuously monitoring and recording the activity of the system 10. Monitoring and logging 118 is beneficial for proactive troubleshooting, performance optimization, and maintaining the reliability of the system 10. For example, by collecting and analyzing log data to detect anomalies, predict potential system failures, and initiate preventive measures. Insights gained from this data are beneficial for ongoing system maintenance, audits, and compliance with technical and regulatory standards.

[0036] In some embodiments, the Integrated Data Correlation (IDC) 119 serves as a central point for integrating 3D CAD data with 2D views and depth maps. This ensures that all parts of the system have access to the same unified data format while maintaining consistency across processes. In some embodiments, the IDC 119 manages data preprocessing through normalization and standardization to standardize the format and scale of 3D and 2D data, ensuring that inputs to machine learning models are consistent and optimized for performance.

[0037] In some embodiments, IDC 119 facilitates data caching by implementing a caching mechanism for storing processed data, thereby reducing redundancy in data processing across different modules (e.g., 108, 110, 112, 114, 116, 117, 118, and / or 119) when the same data needs to be accessed by both the training and prediction models. IDC 119 provides resource optimization by ensuring that data is processed and stored in a way that minimizes memory usage and computational load, which is particularly important in cloud-based architectures. In some embodiments, IDC 119 facilitates the sharing of extracted feature and depth information between different machine learning models, improving system efficiency and effectiveness by leveraging learned information across multiple tasks.

[0038] Referring here to Figures 2-3 along with Figure 1, Figure 2 shows a 3D CAD model of the physical object 200 and a corresponding shaded contour rendering 201 of the physical object 200. As shown in Figure 2, the object 200 shown in so-called CAD wireframe does not convey depth information well. Rather, the shaded contour rendering 201 displays such depth information via shade lines 204 that distinguish the contour of the object 200, even though it is shown in 2D. The contour lines 204 in the 3D model can be mapped to surface shading in the output file. Shading is particularly important for displaying three-dimensional articles, and the shade lines 204 are advantageous for drawing the planes, concave, convex, undulating and / or recessed surfaces of the object 200 with lines, and for distinguishing open surfaces from solid surfaces. Transparent and translucent surfaces may be shown with oblique shading (not shown).

[0039] Therefore, in some embodiments, the training module 110 can perform a method for training a machine learning model to generate predictive shaded contour renderings having shade lines 204. Such a method can compare an unshaded CAD model (e.g., object 200) with a corresponding shaded contour feature (e.g., shaded contour rendering 201) annotated with labels of various surface types. In some embodiments, the technique for training the machine learning model may include classification of surface types (e.g., convex sphere, concave, plane, cylindrical interior, etc.) and the application of appropriate shading techniques, which will be discussed in detail below.

[0040] Data preparation and image classification using CNNs In some embodiments, system 10 provides data preparation and labeling. Input data for training the model includes raw images or processed features from a 3D CAD model. For each input image, the corresponding output is a labeled image including all surfaces and shading annotations, as shown in Figure 2, for example. This is achieved by accurately labeling the CAD model by surface type and the respective shading characteristics of that type, as shown in shaded image 201, for example. Each type of surface (e.g., convex sphere, concave, plane, cylindrical interior, etc.) is clearly identified and marked. In some embodiments, data preparation and labeling includes annotation. For example, 2D engineering drawings (Figures 2B-2H) are annotated in a manner similar to how shaded drawings are annotated in this example. Such annotations include the identification of curves, shaded lines, and linear surfaces. Accurate annotations provide ground truth data from which the model can learn.

[0041] For example, object 200 corresponds to a 3D CAD model of a complex object containing detailed geometric data. The output is a 2D shaded contour feature rendering. As shown in Figure 2A, object 201 contains data annotations. Each input / output pair (200, 201) is prepared. In some embodiments, system 10 can utilize thousands of input / output pairs to train the model via resource 120. In some embodiments, tens or hundreds of thousands of input / output pairs are implemented and are not shown here for clarity. In some embodiments, data preparation may include converting the 3D CAD file to a format suitable for processing (e.g., raster image, simplified vector format). The corresponding output file needs to be annotated, for example, by indicating different features that should be recognized and transformed by the model.

[0042] Feature extraction In some embodiments, the method for training a predictive model includes edge detection. For example, this can be achieved by using an algorithm that can detect edges and curves to distinguish different parts of an object. Such detection can be carried out via deeper CNN layers specifically trained to recognize such features in CAD drawings, which will be discussed in more detail below.

[0043] In some embodiments, the method includes surface analysis, for example, by employing feature extraction techniques that can classify different surface types by analyzing texture and geometric features. Such classification may include analyzing gradients in an image, curvature data derived from a CAD model, or other morphological features. Morphological features may refer to properties related to the shape and structure of an object in an image or 3D CAD rendering. Such features may be advantageous for tasks such as image segmentation, object detection, and feature classification in machine learning models. Some embodiments may include one or more edges, corners and intersections, ridges and valleys, skeletons, convex hulls, Euler numbers, texture features, area, perimeter, as well as compactness, aspect ratio, orientation, roundness, and eccentricity, which are described in detail below.

[0044] Edges are the boundaries or contours of objects in an image. Edge detection is useful for separating different areas of an image based on abrupt changes in intensity. Corners and intersections are points in a CAD file where two or more edges meet. Such features are often key to understanding the geometry of objects in an image. Ridges are curved structures in an image where intensity increases in multiple directions, while valleys are areas where intensity decreases, which is useful in topographic modeling. A skeleton of a shape is a thinned-out version of that shape, equidistant from its boundaries. The skeletonization process helps reduce the detail of an image to preserve the general form of the object.

[0045] For example, some embodiments include a specific input configuration that parses out skeleton versus contour features, and then parses the contour features to reveal plane curvature or surface shading. This is analogous to a form of style transfer, which is used to change the overall visual appearance of a texture. Essentially, this task allows the content of one image to be reconstructed in a different style. Some embodiments include custom segmentation used to distinguish specific parts or elements of an image. Specifically, the custom segmentation works at the pixel level, and the goal is to assign each pixel in the image to the object (i.e., class label) to which that pixel belongs. Thus, structural components or contour information should be classified and segmented, in which case the contour information can undergo a style transformation from wireframe contour data to surface shading style data. In some embodiments, the custom segmentation of frame versus contour, followed by shading, may be an overlay on each surface having independent planes. Some embodiments can preprocess the data by converting the CAD model to a common format such as a voxel grid or point cloud and resizing the output image to a consistent size.

[0046] The convex hull is the smallest convex shape that can contain an object within an image. The convex hull is advantageous for object recognition and shape analysis because it provides a boundary within which all other points of the object reside. The Euler number is a scalar value that describes the topology of an image. The Euler number provides the number of objects in an image minus the number of holes within those objects, which can be advantageous for characterizing the connectivity properties of an image.

[0047] Texture features describe the arrangement and distribution of pixels within an image area. Texture analysis can include measures of smoothness, roughness, and regularity. Area, perimeter, and compactness features are fundamental geometric features that describe the size and shape of an object. Compactness, for example, provides insight into how circular an object is by comparing its area to the area of ​​a circle with the same perimeter. The aspect ratio is the ratio of the width of an object's bounding box to its height. The aspect ratio is advantageous for distinguishing objects that are similar in shape but have different orientations.

[0048] Orientation is the angle at which an object is oriented within an image. This feature helps in understanding the object's position relative to other objects or the frame of the image. Roundness and eccentricity describe how much an object deviates from a circle. Eccentricity measures the ratio of the distance between the two foci of an ellipse to the length of the ellipse's major axis, which provides insight into the object's elongation.

[0049] In some embodiments, such morphological features can be extracted using thresholding, edge detection algorithms (e.g., Sobel, Canny), and specific image processing techniques such as morphological operations like expansion, erosion, opening, and closing. By analyzing such features, the training module 110 can accurately interpret and classify objects in an image, enhancing applications across various fields, including medical imaging, automated quality control, and even complex tasks such as converting CAD models into detailed shaded contour renderings.

[0050] Predictive Model Architecture As described above, in some embodiments, the training module 110 includes one or more convolutional neural networks (CNNs). Given the CNN's proficiency in processing image data, the trainer 110 includes a CNN tuned to not only recognize different parts of a 3D CAD file but also to understand the underlying 3D structure suggested by the 2D projection. In some embodiments, the training 110 includes a generative model including a GAN, which is advantageous for generating high-fidelity shaded contour renderings from processed CAD data. Some embodiments may include a conditional generative adversarial network (cGAN) for generating images that correctly apply the learned shading techniques to new drawings. For example, the “conditions” may be surface type and desired shading that affect the generator’s output. In some embodiments, an autoencoder and / or U-Net may be used for tasks such as noise reduction or segmentation when needed before final illustration generation, which will be discussed in more detail below.

[0051] In some embodiments, the training module 110 can divide the global dataset into a training set, a validation set, and a test set. The training set is used to train the CNN, the validation set is used to tune hyperparameters and evaluate model performance during training, and the test set is used to evaluate the model's performance on unseen data.

[0052] In some embodiments, the training module 110 includes a classification layer that distinguishes different types of surface treatments, determining where shade lines should be applied based on the surface type. This is advantageous for training the model to replicate the labeling in annotated examples. In some embodiments, the training module 110 includes a regression / segmentation layer. For example, based on the complexity of an object, an additional layer may be used to predict the precise location of shade lines, which may involve regression or segmentation tasks.

[0053] In some embodiments, the training module 110 includes a loss function. Such a loss function may utilize a combination of a classification loss for surface type identification (such as cross-entropy) and a pixel-by-pixel loss for accurate shade line placement (such as mean squared error). In some embodiments, the training module 110 includes validation. For example, this is done by using data splits not seen during the training phase to validate the accuracy of the model and its ability to generalize to a new, unseen CAD model.

[0054] In some embodiments, the training module 110 includes a feedback loop that incorporates user feedback from initial predictions to improve model training, particularly focusing on areas where the model misclassifies surface types or misplaces shading lines. In some embodiments, the training module 110 includes continuous learning. As more CAD models and annotations become available, the model is continuously retrained and fine-tuned to improve its accuracy and adapt to new types of drawings or changes in design standards. Thus, the training module 110 provides a machine learning model that not only accurately classifies different surfaces within a CAD model but also applies appropriate shading techniques.

[0055] Model Evaluation In some embodiments, System 10 uses unseen data to favorably provide evaluation and performance metrics before deploying the model. This helps ensure that the model generalizes well and produces shaded contour renderings that meet the required criteria. For example, performance metrics may include visual quality, which evaluates clarity, accuracy, and compliance with industry standards, e.g., design shaded contour rendering standards. In some embodiments, performance metrics may include accuracy metrics, which may include SSIM (Structural Similarity Index), MSE (Mean Squared Error), or other suitable for evaluating the fidelity of the generated image.

[0056] Once the model is trained and validated, it is deployed to a production environment where the model can receive new CAD model inputs and generate corresponding shaded contour renderings via feature extraction 112, image generation 114, and output and distribution 116. System 10 may be deployed as part of a Software as a Service (SaaS) platform where users upload CAD files and receive shaded contour renderings. Such deployments require a robust backend architecture to handle potentially large computations, especially when dealing with complex models and large files.

[0057] When a user uploads a new CAD file to system 10 via device 140, the generation of a new rendering can begin. The GUI 142 is designed to facilitate easy navigation and file management, ensuring a seamless user experience. When a CAD file is uploaded, feature extraction 112 processes the input 3D CAD file to identify and classify important features and geometric data. Feature extraction 112 uses advanced algorithms to analyze and extract the advantageous details necessary for accurate illustration rendering.

[0058] Next, the extracted features are transferred to image generation 114. This module uses a trained machine learning model (e.g., training module 110) to convert the extracted data into a detailed shaded contour rendering. For example, such a rendering can be used in a design patent application. For example, module 110 applies learned techniques to replicate the necessary style and details required for patent documentation.

[0059] The output and delivery 116 conforms to and formats the generated shaded contour rendering. The output and delivery 116 ensures that the improperly shaded contour rendering meets the required specifications and format for industry standards, for example, for design patent illustration applications. The illustration is then available for download or direct review on system 10, and the user can easily access and inspect the generated output via device 140.

[0060] System 10 is configured to continuously improve through a user feedback loop. User feedback is an integral part of the continuous improvement process. After reviewing the generated renderings, users can provide feedback directly through the platform via the GUI 142. In some embodiments, such feedback, along with any manual corrections submitted by the user, is collected and analyzed by feature extraction 112 and image generation 114 to improve and enhance the accuracy and output quality of the model.

[0061] In some embodiments, training 110 may include model updates to accommodate changes in CAD technology and standards, and periodic updates are performed on the models within the feature extraction and image generation modules. These updates ensure that the system stays up-to-date with the latest advancements and continues to meet the evolving needs of the user. By integrating the embodiments described above, the AI-driven system 10 effectively bridges the gap between the complex data contained in CAD models and the clear, standardized visual format required for shaded contour feature rendering.

[0062] In some embodiments, training 110, feature extraction 112, and image generation 114 can include one or more CNNs, U-Nets, and / or GANs, followed by a brief review. CNNs are specifically designed to process images and are particularly effective when recognizing patterns and features within images. CNNs consist of convolutional layers, pooling layers, and fully connected layers, which help extract and learn important features from images. CNNs are primarily used for tasks like image classification, where the CNN identifies and classifies elements within an image. CNNs are efficient at recognizing patterns, textures, and other features within images due to their architecture, which effectively incorporates spatial hierarchies. U-Net, a type of CNN, is specifically designed for image segmentation tasks. U-Net has a unique architecture that enables precise localization, making it ideal for tasks where each pixel in an image needs to be classified (such as medical image segmentation). Like standard CNNs, U-Net does not essentially generate new images, but rather divides existing images into differently classified parts.

[0063] GANs consist of a generator and a discriminator; the generator learns to create synthetic data that resembles true data, and the discriminator learns to distinguish between real data and generated data. GANs are particularly popular for tasks such as image transformation, style transfer, and data augmentation. Autoencoders are unsupervised neural networks used for dimensionality reduction, feature extraction, and image transformation tasks. Autoencoders learn to compress input data and then reconstruct it in its original form. Variational autoencoders (VAEs) and denoising autoencoders can be used for image transformation tasks. Transducers, originally designed for natural language processing tasks, also show great potential for computer vision tasks. Visual transformers (ViTs) divide an image into smaller patches, linearly embed these patches, and process those patches using a self-attention mechanism. They have been successfully applied to a variety of image-related tasks, including image classification and image transformation.

[0064] Therefore, in system 10, in some embodiments, it may be necessary to integrate a CNN and / or U-Net with other image generation networks in order to generate images by converting a CAD model into shaded contour rendering. For example, in one embodiment, an autoencoder may be used. For example, a VAE can generate new images by learning to encode the input into a low-dimensional space and then decode it back into an output image. A CNN acts as an encoder, capturing important features from the CAD model.

[0065] In some embodiments, the training module 108 and / or the image generator 114 may include one or more GANs configured for image generation. Such a GAN may include a generator network that generates images from random noise or direct feature transformations, with the aim of producing an output as close as possible to actual data (i.e., shaded contour renderings). The GAN may include a discriminator network that attempts to distinguish between actual shaded contour renderings and those generated by the generator. In some embodiments, the GAN training process may include improving the generator based on feedback from the discriminator until the generator's output is indistinguishable from actual shaded contour renderings.

[0066] Therefore, in some embodiments, feature extraction 112 analyzes the CAD model using a CNN or U-Net to extract important features (e.g., edges, shapes, and associated geometric shapes and forms). Image generation 114 can input these features into a generative model (e.g., the decoder portion of a GAN or autoencoder) that has been trained to create high-fidelity shaded contour renderings from such features. System 10 effectively combines the analytical power of the CNN and U-Net with the creative generative power of a network including a GAN and / or autoencoder to generate a new image that conforms to the required criteria for shaded contour rendering.

[0067] Some embodiments involve training the generating portion of system 10, such as a decoder for a GAN and / or autoencoder, to generate shaded contour renderings from extracted features. Such training may include training a GAN to generate shaded contour renderings. For example, a dataset of CAD models and corresponding shaded contour renderings trains a GAN to understand what an accurate shaded contour rendering will encompass given specific features of the CAD model. In some embodiments, the generator learns to directly extract and use features during training. In other embodiments, the CAD model is preprocessed using a CNN, thereby extracting and condensing features, which are then fed to the generator, as discussed in detail above.

[0068] In some embodiments, the generator starts by generating an image from a random noise vector (initial) or a set of features extracted from a CAD model (via a CNN). The goal is to generate an image that resembles a corresponding shaded contour rendering. The discriminator examines both the actual shaded contour renderings from the dataset and the illustrations generated by the generator. The discriminator learns to distinguish between real and fake and provides feedback to the generator. In some embodiments, the generator and discriminator are trained simultaneously. The generator aims to deceive the discriminator by improving its output, while the discriminator becomes better at identifying fakes. Over time, the generator's output should increasingly resemble the actual shaded contour renderings.

[0069] In some embodiments, the autoencoder is trained to encode the input into a compressed latent space, and then decode this representation back to an output that matches the original input. To generate shaded contour renderings, some embodiments use the training protocol described below, starting with a dataset of CAD models and corresponding shaded contour renderings, similar to GANs. The encoder portion of the network compresses the CAD models into a latent representation. Typically, the autoencoder learns to recreate the input image, but in the embodiments herein, the encoder learns from the CAD model features. The decoder can be trained to take the latent representation (derived from the CAD model) and generate the corresponding shaded contour rendering.

[0070] During training, the encoder-decoder network considers a CAD model as input and a shaded contour rendering as the target output. The loss is calculated based on how well the decoder's output matches the actual shaded contour rendering rather than the original CAD model. Several embodiments use various loss functions for visual tasks. For example, there are content and style-preserving loss functions, such as perceptual loss or a combination of mean squared error and adversarial loss (when combined with GAN-like features).

[0071] Such a training process is iterative, requiring many epochs of training, each representing a complete pass across the entire dataset. Hyperparameter tuning, adjustment of the learning rate, the architecture of the neural network (number of layers, layer types, etc.), and other parameters are advantageous for optimizing performance. Some embodiments involve validating the model using separate datasets not encountered during training to ensure that the model generalizes well to new CAD models and can produce high-quality shaded contour renderings.

[0072] Some embodiments include strategies to alleviate large data requirements through data augmentation, transfer learning, synthetic data generation, and / or active learning. Data augmentation increases the effective size of the dataset by applying transformations such as rotation, scaling, and cropping to training images. Such transformations help the model generalize better from fewer examples. Transfer learning utilizes a pre-trained model that has been trained on a large dataset for a similar task and then fine-tunes it on the CAD dataset. This can significantly reduce the amount of data required. Active learning allows the trainer 108 to selectively annotate more data by implementing a strategy in which the model identifies which new data points are most beneficial to learn from.

[0073] Integrating feature vectors into GANs In some embodiments, system 10 integrates the outputs of the CNN and U-Net into a generative adversarial network (GAN) to generate shaded contour renderings from a CAD model. As described above, the CNN and U-Net are initially trained to perform tasks such as identifying important features in a 3D CAD model (commonly called feature vectors), segmenting parts, or detecting contours. While the CNN can identify certain feature vectors such as edges, textures, and shapes, the U-Net can be used for more detailed segmentation tasks that require an accurate depiction of model parts.

[0074] The output from a CNN / U-Net consists of feature maps or segmented images, which will be discussed further below. Feature maps are detailed representations that highlight important features detected in the input image. Segmented images from a U-Net classify each pixel as belonging to a specific part or feature of the input CAD model.

[0075] In preparing GAN input, feature maps or segmented outputs from a CNN or U-Net can be appropriately formatted for use as input to a GAN. This can be done, for example, by normalizing the data, flattening the feature maps into a format that can be fed into the GAN, or encoding these maps into a more compact form. The GAN generator must be designed to accept these processed feature maps as input. Such a design may include adapting the generator's input layer to match the size and structure of the feature maps. By incorporating additional high-density or convolutional layers, more detailed outputs can be produced.

[0076] During the training phase, the generator uses feature maps to produce images similar to the target shaded contour rendering. The classifier's role is to distinguish the generated image from the actual shaded contour rendering. The generator learns to improve its output based on feedback from the classifier, with the goal of producing images that the classifier cannot easily distinguish from the actual illustration.

[0077] Optimization and fine-tuning In some embodiments, CNNs and / or U-Nets and GANs can be fine-tuned jointly in an end-to-end manner after initial separate training. This means simultaneously adjusting the weights of both the feature extractor (CNN or U-Net) and the generator based on their overall performance in generating shaded contour renderings. Such joint training helps feature extraction integrate more directly with generation, allowing the network to adapt more holistically. In some embodiments, the loss function can be tuned to reflect both the accuracy of feature extraction and the quality of image generation, for example, by using a combination of classification loss (from the CNN / U-Net output) and adversarial loss (from the GAN).

[0078] In some embodiments, system 10 includes integration of 3D CAD models and 2D engineering views of those 3D CAD models to facilitate the generation of additional data points for training machine learning models. For example, preparing a 3D CAD model of a physical object provides a comprehensive spatial structure of the physical object, revealing insights into its depth, viewpoint, and complex geometry. By complementing the 3D CAD file, system 10 can generate six 2D engineering views encompassing six standard viewpoints (front, back, top, bottom, left, and right). These views are advantageous for understanding how different features of an object are represented from multiple angles in a flat format, providing valuable visual information for accurate modeling.

[0079] In some embodiments, the method for training a predictive model includes a 3D / 2D data correlation stage, which involves developing a sophisticated algorithm designed to map identifiable features from a 3D CAD model to corresponding elements of that model in a 2D engineering view. Such a feature mapping process may focus on identifying the locations of keypoints, edges, and surfaces within the 3D model and aligning such keypoints with their equivalents in the 2D view. Furthermore, projection mapping techniques may be used to correlate depth information from the 3D model with contours and features visible in the 2D view, thereby ensuring that the AI ​​model can accurately interpret which elements of the 2D drawing correspond to raised or recessed areas of the 3D object.

[0080] In some embodiments, the training stage described above can integrate both 3D model data and correlated 2D views into the AI ​​model as paired training data. This approach allows the model to learn how depth and surface variations are depicted across different views, improving its predictive accuracy for new illustrations. The model architecture is designed as a hybrid neural network utilizing a CNN to manage spatial data from the 3D model by extracting features from the 2D views and the 3D convolutional network. The training goal is to generate a final 2D shaded contour feature illustration that accurately reflects depth and design details with detailed contours and surface shading. Such a shaded contour feature illustration can be used, for example, for filing a design patent application.

[0081] In some embodiments, post-training image generation involves applying learned correlations to a new CAD file, incorporating both 3D views and their respective 2D views, to generate detailed shaded contour feature illustrations. This stage demonstrates the application of model training, accurately predicting and rendering features and depths according to shaded contour feature rendering standards using established feature mapping and depth associations.

[0082] The refinement process involves applying additional algorithms after image generation to adjust line weights, enhance shading, and ensure all illustrations meet required industry drawing standards. The verification process uses a range of tools or manual checks to verify that the generated illustrations faithfully represent the original CAD model in both form and function.

[0083] The modules of system 10 can be adapted to additional training correlations. For example, data management 108 can be extended to effectively handle and store both 3D models and multiple 2D engineering views. Feature extraction 112 can be enhanced to perform complex mappings between 3D features and their 2D representations. Training 110 is designed to handle multi-input training sessions containing both types of data. The image generation module is configured to utilize sophisticated algorithms that incorporate depth and contour information derived from 3D / 2D data correlations.

[0084] Through this integrated approach, the AI ​​system can create highly accurate and detailed shaded contour feature renderings that are technically beneficial and compliant with patent application requirements. This method significantly improves the model's ability to generate reliable drawings essential for protecting intellectual property, while streamlining traditional manual and labor-intensive processes.

[0085] Data structure for training predictive models Referring here to Figures 1-3 and Figure 4, in some embodiments, the integrated data correlation 119 includes generating an AI data structure 400. In some embodiments, the data structure 400 is configured for efficient correlation of 3D CAD and 2D engineering views, which enhances the efficiency and accuracy of the training and prediction models in the embodiments described above. In some embodiments, the system 10 includes designing an efficient and accurate data structure 400 for correlating 3D CAD models to 2D engineering views, which can effectively process and integrate complex spatial and visual data. The data structure 400 facilitates quick access to and manipulation of data for both training machine learning models and generating output illustrations. In some embodiments, the integrated data correlation 119 includes, generates, and outputs the data structure 400 as follows:

[0086] In some embodiments, the data structure 400 may include contour feature data 402, which maintains a direct correlation between 3D objects and their respective 2D views in a manner that is easily accessible and manipulable by the AI ​​model and components of the system 10 (e.g., 108, 110, 114, 116, 118, and / or 119). In some embodiments, the contour feature data 402 includes one or more bits corresponding to a 3D model representation that store a pointer to the complete 3D CAD model data of the corresponding physical object. In some embodiments, such data may include a format that includes metadata relating to vertices, edges, faces, and the material and texture of the object (e.g., 200).

[0087] In some embodiments, the data structure may include a 2D view array 404. The 2D view array 404 may contain an array or list of 2D view objects, each object representing one of the standard engineering views (front, back, top, bottom, left, and / or right). Such lists and arrays may store pointers to the location of further data. The data structure 400 may include a feature linking table (FLT) 406 that maps specific features in the 3D model of object 200 to their manifestations in each 2D view (e.g., Figures 3A to 3G).

[0088] In some embodiments, the data structure 400 may include structure values ​​408. The structure values ​​408 may include one or more of the following: a feature ID corresponding to a unique identifier for each feature in the 3D model; 3D coordinates corresponding to the spatial coordinates of the feature in the 3D model; and a 2D coordinate map mapping of the 3D feature to its respective coordinates in individual 2D views. In some embodiments, such mappings may be structured as a dictionary, or as a set of key-value pairs where the key is a view identifier and the value is a coordinate pair or bounding box, for example, as a linked lookup table. In some embodiments, such a lookup table may be remotely linked via resource 120 or stored by server 102 and / or data manager 108.

[0089] In some embodiments, the data structure 400 may include depth data 410. The depth data 410 may include a depth information layer or depth map to enhance the 2D view with depth information derived from the 3D model, which is advantageous for accurate shading and contouring. In some embodiments, the depth data 410 may include pointers to a depth map, which may include a grid or matrix associated with each 2D view. Such a grid or matrix indicates the depth of features at various points, facilitating the application of correct shading techniques in the output illustration.

[0090] In some embodiments, the data structure 400 is operable to integrate with a neural network (e.g., CNNS, U-Net, and / or GAN) utilized by system 10. The data structure 400 may be fed directly to the neural network of system 10, for example, via a data manager 108 and / or integrated data correlation 119. The neural network of system 10 is trained to understand and process the relationships between 3D features and their 2D projections as described above. In some embodiments, using the depth data 410 and the data structure 400, the CNN and / or U-Net may be adapted to process a 2D view with associated depth maps as a multi-channel input image, where one channel represents visual data and the other represents depth data. Such functionality may be indicated by multi-channel bits 412.

[0091] As will be discussed in more detail below, in some embodiments, the data structure 400 may be operable for optimization efficiency and accuracy through indexing, normalization, and / or batch processing. For example, performing indexing on the FLT 406 to speed up queries and data retrieval is advantageous during the training and generation phases. Such indexing creates a searchable database index on one or more columns (attributes) in the table, such as frequently queried feature identifiers or keys. Searchable attributes allow the database system (e.g., server 102) to find data associated with these keys much faster than scanning all rows in the table, often resulting in a linear-to-logarithmic reduction in time complexity.

[0092] Indexing is even more advantageous during complex queries involving multiple junction joins or locations where specific subsets of data are frequently accessed, such as when matching frequently encountered 3D geometries to 2D features. By reducing the time spent searching for and ingesting data, indexing minimizes CPU and memory usage. This optimized resource utilization is advantageous when dealing with large datasets for training predictive models using CAD models and engineering drawings, as described herein. As the dataset grows with more CAD files and engineering views, the benefits of indexing become increasingly significant, enabling System 10 to scale more efficiently.

[0093] In real-time processing, such as image generation processes where speed is advantageous, having an indexed table means that the feature data needed to generate accurate shaded contour feature illustrations can be retrieved almost instantly. During the training phase, when large amounts of data are processed, indexing significantly reduces the overhead of repeatedly fetching relevant data, thereby speeding up the training process.

[0094] Furthermore, indexing helps maintain data integrity by ensuring that each entry is unique according to the indexed attribute, which is advantageous when entries are frequently updated or modified during the training phase. By facilitating faster and more accurate data retrieval, indexing reduces the potential for errors that can arise from incorrect data mapping behavior or long data fetching operations. The immediate feedback enabled by rapid data retrieval supports faster tuning and optimization of algorithm and model parameters.

[0095] In some embodiments, normalization allows all 3D and 2D coordinates to be stored in a normalized format to reduce computational overhead and improve the training efficiency of the model. Also in some embodiments, the data structure 400 is configured for batch processing, and multiple instances of the data structure 400n are batched together for parallel processing.

[0096] For example, during training, a CNN / U-Net / GAN model can use the FLT 406 of data structure 400n to learn how features represented in a 3D model appear in different 2D views under various transformations and projections. Depth data 410 with a depth information layer can be used to train the CNN / U-Net / GAN model on how to apply contour feature shading based on the depth of the features, improving the realism and technical accuracy of the generated shaded contour rendering. Such a structured approach not only increases the accuracy of the AI ​​model in correlating a 3D model to its respective 2D view, but also improves the efficiency of the training process by providing well-organized and easily accessible data. Furthermore, data structure 400 supports scalability and flexibility to accommodate various types of CAD models and / or engineering drawings.

[0097] In some embodiments, the data structure 400 can be used in a GAN network for output prediction images based on a new 3D or 2D CAD model input by the user via a user device 140. For example, the data structure 400 in the embodiments herein is structured to correlate 3D CAD models with their corresponding 2D engineering views, which is advantageous not only for training the neural network but also for integrating it into a generative adversarial network (GAN) framework. Such a structure can greatly improve the GAN's ability to generate accurate and detailed predictive shaded contour feature illustrations from a new CAD model input by the user.

[0098] For example, system 10 integrates the data structure 400 with the GAN in preprocessing and input preparation, particularly in feature extraction. In some embodiments, when a new set of 3D CAD models or 2D views is uploaded by the user via device 140, system 10 can process the new input using, for example, FLT 406 and depth data 412 having a depth information layer. This operation includes identifying favorable features in the input CAD model and mapping the identified features to corresponding representations in the 2D engineering view.

[0099] In some embodiments, simultaneously or substantially simultaneously, depth maps 402 for each 2D view are generated or updated based on a 3D model to show the relative depth of different features. This is advantageous for realistic shading in illustrations. The processed features and depth maps 402 are then formatted as input to a GAN (e.g., via modules 108, 112, 114 and / or 119). In some embodiments, such formatting may involve structuring the data within channels 404, with one set of channels 404a conveying 2D view information and another set of channels 404b conveying depth information. In some embodiments, normalization and scaling can be performed to further ensure that all data supplied to the GAN matches the network's input requirements, maximizing the effectiveness of feature learning and image generation by feature extraction 112 and image generation 114, respectively.

[0100] In some embodiments, the training module 110 may include a Generator Modification (GM). For example, the GM can adapt the GAN's generator to take such structured data (e.g., 600) as input. The GM can enhance the generator's architecture to process multiple types of data (e.g., visual features from 2D views and depth data) simultaneously and effectively. Some embodiments may include classifier enhancement by modifying the classifier to evaluate not only realism but also the technical accuracy of the generated image based on how well the generated image incorporates depth and feature information derived from the 3D model and 2D view.

[0101] In some embodiments, the training module 110 can train the GAN using a training data utilization of a mixture of historical and real-time processed data from the data structure 500 for training the GAN. This helps the network learn a comprehensive range of feature representations and depth variations, improving the network's ability to generalize from new CAD model inputs. Some embodiments include a feedback loop mechanism via a feedback manager 117, where the output from the GAN is checked against expected feature and depth specifications from the data structure 400. Mismatches may be used to fine-tune the performance of the generator.

[0102] Generation of Predictive Shaded Contour Rendering Once the GAN model is trained, it can directly generate predicted shaded contour feature illustrations (e.g., Figures 3a–3h) from new CAD model inputs and / or data supplied from CNNs / U-Nets discussed herein. As described above, the generator uses the structured input to create a detailed 2D engineering view that accurately reflects both the geometry and depth features of objects (e.g., object 200) drawn in the 3D CAD model, so that they are mapped and structured into data structure 400.

[0103] Some embodiments may include post-processing by applying additional image processing to refine the illustration and continuously updating the FLT 406 and depth map 410 within the data structure 400. In some embodiments, a feedback manager 117 manages all user feedback received from the device 140. Discrepancies between the generated image and the actual CAD model are noted, and updates are provided to the data structure 400. Such post-processing and iterative refinement improve the accuracy and reliability of the generated shaded contour feature rendering.

[0104] Such a tailored approach to integrating specialized data structures into a GAN framework not only simplifies the process of generating shaded contour renderings but also significantly improves the quality and accuracy of the output. By leveraging detailed feature and depth data from both 3D and 2D CAD models, system 10 can output highly detailed and technically accurate shaded contour feature renderings, which are advantageous for documentation such as design patent applications. Such integration demonstrates a powerful example of implementing advanced data structures within an AI framework to solve complex real-world problems.

[0105] In some embodiments, such a data structure 400 can be a Python-based data structure. A Python-based data structure may be advantageous for handling the complex relationships between 3D CAD models and their corresponding 2D engineering views. In some embodiments, the Python-based data structure used by the GAN to generate shaded contour renderings involves designing a class that can encapsulate all the necessary attributes and methods.

[0106] For example, the following class CADModel: def __init__(self,model_id,model_data): self.model_id=model_id self.model_data=model_data # This could be a file path or a data blob self.features=[] def add_feature(self,feature): self.features.append(feature) class Feature: def __init__(self,feature_id,description,coordinates_3d): self.feature_id=feature_id description=description self.coordinates_3d=coordinates_3d # (x,y,z) coordinates in the 3D model self.views={} def add_view(self,view_name,coordinates_2d): self.views[view_name] = coordinates_2d # (x,y) coordinates in the 2D view class EngineeringView: def __init__(self,view_name,image_data): objective.view_name = view_name self.image_data = image_data # This can be a file path or an image data blob. self.depth_map=None def generate_depth_map(self,depth_data): self.depth_map=depth_data # A matrix or image representing depth values class GANInputData: def __init__(self,features,depth_maps): self.features=features # A list of features formatted for GANs self.depth_maps = depth_maps # Composite depth map from multiple views class GAN: def __init__(self,generator,discriminator): self.generator = generator self.discriminator=discriminator def train(self,training_data): # Implement the training logic here pass def generate_image(self,input_data): # Implement the image generation logic here return “generated_image_path”

[0107] Therefore, the above Python-based data structuring and process is configured to facilitate the structuring of data for processing and correlating between 3D and 2D representations, and to be extended with actual data processing and neural network integration to generate shaded contour renderings.

[0108] The ability of data structure 400 to manage 3D CAD models and correlate them with their corresponding 2D engineering is advantageous for optimizing both computational efficiency and the accuracy of the generated shaded contour renderings. A review of how such a data structure enhances the functionality of a computer system for training neural networks, particularly in terms of processing resources, memory capacity, and rendering accuracy, is as follows:

[0109] The data structure 400 organizes the data so that features and their corresponding views are directly linked, and such organization allows for quick access to relevant data as needed, reducing the time spent searching in unstructured data collections. By structuring the data to support batch processing where similar types of data are processed together, system 10 can effectively utilize vector operations and parallel processing capabilities. This is particularly effective in reducing processing time during both the training and inference phases of machine learning models. Having structured linkages between 3D features and their 2D representations means that once features are processed or depth maps are generated, the data can be reused across different tasks without needing to be recalculated. This minimizes redundant computations, thereby saving processing resources.

[0110] Data structure 400 facilitates memory optimization through selective loading and load-on-demand. Data structure 400 allows system components (e.g., 108-119) to load only the necessary data into memory as needed. For example, if a particular 2D view or feature is not relevant to the current processing task, such features remain unloaded, thereby saving memory. Compact storage methods allow data to be stored more compactly compared to loosely or unstructured formats by organizing features and their corresponding mappings in a structured format (e.g., FLT 406, Array 402). Efficient data encoding and compression techniques can further reduce memory usage.

[0111] Data structure 400 facilitates improved rendering accuracy through precise feature mapping. Detailed mapping of features from 3D models to their 2D projections ensures that important details are not lost during the conversion. This is advantageous when rendering images that must adhere to stringent standards for shaded contour rendering. Including depth maps in the data structure allows the rendering process to incorporate accurate shading and contouring based on spatial information from the 3D models. This adds realism and technical accuracy to 2D illustrations, which is beneficial for industrial applications.

[0112] By maintaining a consistent format and linkage between 3D and 2D data, System 10 ensures that all transformations and renderings are based on the same baseline data. This consistency helps maintain accuracy across different views and renderings. The structured nature of the data facilitates robust checks and validations to ensure that mappings and renderings are correct. Inconsistencies can be quickly identified and corrected, which is advantageous for training machine learning models to produce reliable outputs.

[0113] Furthermore, because irrelevant data can be easily excluded from the training process due to the structured data format, machine learning models can focus on learning the most relevant features and their transformations. Such targeted learning improves the efficiency and effectiveness of the model. For example, rapid access to structured and correlated data allows for iterative training and faster fine-tuning of the model. This accelerates the model development cycle and enables faster adjustments based on performance feedback.

[0114] Therefore, using the data structure 400 not only improves the efficiency and capabilities of the computing resources of system 10, but also significantly improves the accuracy and reliability of the generated shaded contour rendering output to the user device 140. This dual benefit of operational efficiency and technical accuracy is particularly valuable, for example, in situations where both accuracy and processing efficiency are paramount, such as when generating design patent illustrations.

[0115] Referring here to Figures 1 to 4 and Figures 5 to 7, Figure 5 shows a flowchart of Method 500 according to one or more embodiments described above. Method 500 includes, in operation 502, receiving a 3D model input (200) corresponding to a physical object (202). In operation 504, generating a data structure (400) based on the 3D model input, including one or more features (402) of the physical object and one or more 2D renderings (300) of the physical object. In operation 506, correlating one or more features with one or more 2D renderings of the physical object. In operation 508, determining a shaded contour rendering of the physical object based on one or more features. In operation 510, sending the shaded contour rendering of the physical object to a display device. In some embodiments, operations 502 to 510 may be performed by modules 108 to 119.

[0116] Figure 6 shows a flowchart of Method 600 according to one or more embodiments described above. Method 600 includes, in operation 602, receiving a three-dimensional model of a physical object. In operation 604, determining one or more two-dimensional views of the physical object based on the three-dimensional model. In operation 606, correlating one or more feature vectors of the three-dimensional object with one or more two-dimensional views. In operation 608, outputting a data structure containing one or more features. In operation 610, training a first predictive model using the data structure. In operation 612, training a second predictive model using the data structure. In some embodiments, operations 602-612 may be performed by modules 108-119.

[0117] Figure 7 shows a flowchart of Method 700 according to one or more embodiments described above. Method 700 includes, in operation 702, providing a computer implementation system including a memory for storing a data structure configured to correlate features between a 3D CAD model and a corresponding 2D engineering view, and a processor operably coupled to the memory. In operation 704, the data structure in memory is populated with model data including a spatial structure and 2D data including standard views depicting the object from various angles. In operation 706, the feature data is correlated to link features from the 3D model to their respective 2D views. In operation 708, the data structure is output. In some embodiments, operations 702-708 may be performed by modules 108-119.

[0118] Referring here to Figure 8, in some embodiments, the System 10 uses a U-Net architecture to train a predictive model for outputting shaded contour feature renderings. In some embodiments, the U-Net architecture may include two main paths, namely an encoder path 804 and a decoder path 806, both intricately designed as a feature pyramid network (U-Net) 800 with strategically implemented skip connections 808. As shown in Figure 8, the encoder path 804 of the U-Net 800 systematically downsamples the image through a series of convolutional and pooling layers. This successive reduction helps to combine the input into a form in which high-level features are extracted while minimizing spatial dimensions. The encoder path 804 is advantageous for separating important features such as edges, textures, and different geometric patterns from the CAD model input, which is advantageous for subsequent stages of feature extraction (e.g., 112).

[0119] In contrast to the encoder, the decoder pass 806 reconstructs the segmented output back to the resolution of the original input image. This is achieved by progressively upsampling the condensed feature map, allowing for a gradual restoration of the image's finer details. The upsampling process is intricately designed to improve the output, ensuring that the resulting shaded contour feature rendering is not only accurate but also adheres to the stringent detail required for industry patent applications.

[0120] In one embodiment, an essential feature of the U-Net design is the inclusion of skip connections 808 spanning between the corresponding layers of the encoder and decoder paths. These connections are advantageous for preserving and transferring fine details that might otherwise be lost during the downsampling process. In some embodiments, by directly concatenating feature maps from the encoder to the corresponding decoder layer, the skip connections 808 ensure that both local details and global contextual information are preserved, thereby improving the fidelity and accuracy of the reconstructed image.

[0121] Within System 10, the U-Net architecture 800 is integrated to leverage advanced segmentation capabilities. This integration is advantageous in the feature extraction module 112 described above for accurately mapping the complex geometry of 3D CAD models to their respective 2D engineering views. The efficient processing of both local and global features in U-Net 800 ensures that the final image output is of high quality, critical to the legal robustness of output rendering, with accurate alignment and clear representation of the complex details of the CAD model.

[0122] Therefore, System 10 represents a significant advance in the fields of computer-aided design (CAD) and shaded contour feature illustration generation. By leveraging state-of-the-art machine learning techniques, including CNN / U-Net architectures and GANs, along with sophisticated data structuring techniques such as indexed feature linking tables, the embodiments herein effectively bridge the gap between complex 3D CAD models and their required 2D shaded contour illustrations. This not only eliminates traditional manual, labor-intensive processes but also improves the accuracy and precision of the final rendering.

[0123] The integrated modules 108-119 work together to ensure a seamless, efficient, and user-friendly experience. Modules 108-119 manage complex data processing, feature mapping, image synthesis, and final illustration formatting and delivery, making the system highly scalable and adaptable to diverse industrial needs and evolving technological landscapes. Furthermore, the system's ability to learn from historical data and continuously improve through user feedback and periodic model updates places it at the forefront of innovation in AI-driven design technology. This enables continuous refinement of processes and methodologies, ensuring the system remains relevant and effective in meeting stringent patent documentation requirements.

[0124] The AI-powered solutions of the embodiments herein not only simplify the creation of renderings suitable for design patent illustrations, but also significantly reduce the time and resources required for their generation, thereby providing patent applicants with a competitive edge and contributing to more robust intellectual property protection. Therefore, this technology is quite promising in transforming patent illustration practices and will bring significant benefits to industries that rely on patent protection to safeguard their innovations.

[0125] The embodiments described herein may be embodied in systems, apparatus, methods, computer programs and / or articles, depending on the desired configuration. Any methods or logical flows shown in the accompanying drawings and / or described herein do not necessarily require a specific order or sequence shown to achieve the desired result. The embodiments described herein do not represent all embodiments that are consistent with the subject matter described herein. Rather, they are merely some examples that are consistent with aspects relating to the subject matter described herein. While some modifications have been detailed above, other modifications or additions are also possible. In particular, further features and / or variations can be provided in addition to those described herein. The embodiments described herein may cover various combinations and partial combinations of the disclosed features, as well as combinations and partial combinations of the further features described herein. Furthermore, the advantages described herein are not intended to limit the application of the published claims to processes and structures that achieve any or all of the advantages.

[0126] Furthermore, any general reference to this disclosure or use of the word “Embodiment” in the singular form does not imply any limitation to the claims set forth below. Multiple embodiments may be described in accordance with the limitations of the claims arising from this disclosure, and such claims define the embodiments herein and their equivalents that are thereby protected.

[0127] In the claims, no reference numerals placed in parentheses should be construed as limiting the claims. The words “comprising” or “including” do not preclude the existence of elements or steps other than those enumerated in the claims. In a device claim enumerating several means, some of these means may be embodied by the same item of hardware. The words “a” or “an” preceding an element do not preclude the existence of multiple such elements. In a device claim enumerating several means, some of these means may be embodied by the same item of hardware. The mere fact that certain elements are described in different dependent claims does not imply that these elements cannot be used in combination.

[0128] The descriptions provided above provide details for illustrative purposes based on what is currently considered to be the most practical and preferred embodiments; however, such details are for that purpose only, and it should be understood that this disclosure is not limited to the expressly disclosed embodiments, but rather intended to cover modifications and equivalent arrangements that fall within the spirit and scope of the appended claims. For example, it should be understood that this disclosure is intended to allow, wherever possible, one or more features of any embodiment to be combined with one or more features of other embodiments.

Claims

1. A system (10) for converting a 3D model to 2D shaded contour rendering in real time, wherein the system is The system includes a processor (104) that communicates with a memory (106), and when an executable instruction is executed by the processor, the memory It receives a 3D model input (200) corresponding to a physical object (202), Based on the 3D model input, a data structure (600) is generated that includes one or more features (402) of the physical object and one or more 2D renderings (300) of the physical object. The one or more features described above are correlated with the one or more 2D renderings of the physical object, Based on one or more of the above features, the shaded contour rendering of the physical object (301) is determined. The shaded contour rendering of the physical object is transmitted to the display device (140). A system (10) that stores the executable instructions that constitute the system.

2. A computer implementation method, The steps include receiving a 3D model of a physical object, The steps include determining one or more two-dimensional views of the physical object based on the three-dimensional model, The steps include correlating one or more feature vectors of the three-dimensional object with one or more two-dimensional views, The steps include outputting a data structure that includes one or more of the aforementioned features, A step of training a first predictive model using the aforementioned data structure, The steps include: training a second predictive model using the aforementioned data structure; A computer implementation method, including

3. A method for generating a data structure for training a predictive model, A memory for storing a data structure configured to correlate features between a 3D CAD model and a corresponding 2D engineering view, and Processor operably coupled to the aforementioned memory The steps include providing a computer implementation system that includes, The steps include loading model data including spatial structure and complex geometric shapes and 2D data including multiple standard views that depict the object from various angles into the data structure in the memory, The steps include: a step of correlating feature data that links the features from the aforementioned 3D model to their respective 2D views; The step of outputting the aforementioned data structure Methods that include...

Images