Method and system for estimating the computational cost of a simulation - Patents.com

Abstract

Described herein are methods and systems for estimating the computational cost of a simulation using a machine learning model. An exemplary method includes inputting a feature dataset into the machine learning model. The feature dataset includes model geometry metadata and simulation metadata. The method further includes using the machine learning model to predict computational cost characteristics for the simulation process.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 62 / 931,299, filed November 6, 2019, and entitled "Method for the Estimation of the Computational Cost of Simulation," the disclosure of which is expressly incorporated by reference herein in its entirety.

[0002] The present disclosure relates to multiphysics simulation tools, with a particular focus on the ability to use machine learning methodologies to accurately estimate the computational cost of a simulation prior to execution. [Background technology]

[0003] Computer-aided engineering (CAE) is the practice of simulating representations of physical objects using computational methods such as the finite element method (FEM) and finite difference method (FDM). Because accurate estimates of computational costs for systems of diverse complexity, potentially with billions of degrees of freedom, can take as long to compute as the solutions themselves, modern numerical analysis tools typically fail to provide users with feedback on the time a simulation will take to complete. There is a need in the related art for a method to provide fast and accurate cost feedback prior to running a simulation to enable new service-based simulation tools to be deployed effectively. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure relates to a method and system for estimating the computational cost of a numerical simulation. In one implementation, the method includes building a database that contains specific initial settings and obtained performance metrics collected from a large number of simulations of different types and purposes. From this dataset, a machine learning model is trained and tested, so that the trained machine learning model can estimate how much core time will be required to compute the solution of a new simulation based on the same set of settings used to train the model. The obtained estimate is used to inform the user of the approximate cost of the simulation before execution and to provide an opportunity to pre-optimize the model if necessary.

[0005] An exemplary method for estimating computational costs of a simulation is described herein. The method includes inputting a feature dataset into a machine learning model. The feature dataset includes model geometry metadata and simulation metadata. The method further includes predicting computational cost characteristics for the simulation process using the machine learning model.

[0006] Additionally, the model geometry metadata includes at least one of the number of elements in the mesh, the number of vertices in the mesh, the surface area to volume ratio of the model, the minimum element size, a mesh quality metric, element type and element order, the aspect ratio of the model, the number of edges in the model, the number of surfaces in the model, the number of finite volumes in the model, or the number of control points in the model.

[0007] Optionally, the model is a computer-aided design (CAD) model. In some implementations, the method further includes extracting model geometry metadata from the CAD model.

[0008] Alternatively or in addition, the simulation metadata includes the type of analysis, solution domain, type of linear solver, preconditioner, time integration method, spatial integration method, degree of nonlinearity, nonlinear solver options, type and number of levels of adaptive grid refinement, time step size, or number of frequencies, degrees of freedom of boundary conditions, type of boundary conditions, type of material model, material properties, number of cores, or physics-specific parameters.

[0009] Alternatively or in addition, the computational cost characteristic is at least one of memory usage or time for the simulation process. Optionally, the computational cost characteristic is time for a portion of the simulation process. Optionally, the method further includes estimating the number of core hours based at least in part on the time for the simulation process.

[0010] Alternatively, or in addition, the computational cost characteristic is at least one of a time required for mesh generation, a time required for pre-processing, a time required for a solution, a time required for post-processing, or a time required for a simulation process.

[0011] Alternatively, or in addition, the step of predicting the computational cost characteristic for the simulation process using the machine learning model includes a plurality of computational cost characteristics for the simulation process, each respective computational cost characteristic for the simulation process being based on a different set of computational resources and / or solver options.

[0012] Alternatively, or in addition, the machine learning model is a deep neural network, a convolutional neural network, a gradient boosting decision tree, or a Gaussian process regression.

[0013] Alternatively, or in addition, the feature dataset optionally further includes computing environment metadata.

[0014] Alternatively or in addition, the method further includes determining a feature dataset. In some implementations, the step of determining the feature dataset includes performing a plurality of simulations to obtain an initial simulation dataset, selecting one or more types of metadata from the initial simulation dataset to create a model feature dataset, and training and testing a machine learning model using the model feature dataset. The one or more types of metadata include model geometry metadata, simulation metadata, computing environment metadata, or a combination thereof. Optionally, the method further includes performing one or more additional simulations, supplementing the model feature dataset with metadata from the one or more additional simulations, and training and testing a machine learning model using the supplemented model feature dataset. Optionally, the step or steps of training and testing a machine learning model are performed iteratively.

[0015] Optionally, in some implementations, inputting the feature dataset into the machine learning models includes inputting model geometry metadata into a first machine learning model and inputting simulation metadata into a second machine learning model. In these implementations, the method optionally further includes creating a fixed dimensional representation vector from the output of each of the first and second machine learning models and analyzing the fixed dimensional representation vector to predict computational cost characteristics for the simulation process. Optionally, analyzing the fixed dimensional representation vector to predict computational cost characteristics for the simulation process includes performing a regression analysis.

[0016] Also described herein is an exemplary system for estimating computational costs of a simulation. The system includes a machine learning model and a computing device including a processor and a memory. The memory stores computer-executable instructions that, when executed by the processor, cause the processor to input a feature dataset into the machine learning model and use the machine learning model to predict computational cost characteristics for the simulation process. The feature dataset includes model geometry metadata and simulation metadata.

[0017] Also described herein is an exemplary method for estimating computational costs of a finite element analysis (FEA) simulation. The method includes receiving a first data set including a computer-aided design (CAD) model, inputting the first data set into a first machine learning model, and extracting a model geometry data set by processing the first data set with the first machine learning model. The method also includes receiving a second data set including metadata associated with the CAD model, inputting the second data set into a second machine learning model, and extracting a simulation metadata set by processing the second data set with the second machine learning model. The method further includes creating a fixed dimensional representation vector from the model geometry and the simulation metadata set, and analyzing the fixed dimensional representation vector to determine target parameters for the FEA simulation.

[0018] In addition, the step of analyzing the fixed-dimensional representation vector optionally includes inputting the fixed-dimensional representation vector into a head network and extracting target parameters for the FEA simulation by processing the fixed-dimensional representation vector with the head network. Alternatively or in addition, the head network is configured to perform regression to determine the target parameters for the FEA simulation.

[0019] Alternatively or additionally, the target parameter is at least one of memory usage or simulation run time for the FEA simulation. In some implementations, the method further includes estimating the number of core hours based at least in part on the simulation run time.

[0020] Alternatively or in addition, the metadata associated with the CAD model includes at least one of a time iteration, a material property, a number of cores, a physics-specific parameter, or a number of elements in a mesh of the CAD model.

[0021] Alternatively, or in addition, the metadata associated with the CAD model may be discrete or continuous.

[0022] Alternatively, or in addition, the first machine learning model is a convolutional neural network.

[0023] Alternatively, or in addition, the second machine learning model is a combination of a neural forgetting decision tree and a fully connected neural network.

[0024] Alternatively or in addition, the CAD model includes a spatial model geometry. Optionally, the spatial model geometry is divided into a plurality of discrete cells. Optionally, the method further includes extracting the spatial model geometry as a point cloud, and the step of inputting the first data set into the first machine learning model includes inputting the point cloud into the first machine learning model. In addition, the point cloud includes each vertex of the discrete cells. Alternatively or in addition, the point cloud includes at least the interpolated physical values, the resampled physical values, or the boundary conditions.

[0025] Another exemplary method for estimating the computational cost of an FEA simulation is described herein. The method includes receiving a dataset including a computer-aided design (CAD) model and metadata, inputting the dataset into a machine learning model, and extracting a model geometry and a simulation metadata set by processing the dataset with the machine learning model. The method also includes creating a fixed-dimensional representation vector from the feature set and analyzing the fixed-dimensional representation vector to determine target parameters for the FEA simulation.

[0026] It should be understood that the subject matter described above may also be embodied as an article of manufacture, such as a computer-controlled apparatus, a computer process, a computing system, or a computer-readable storage medium. Other systems, methods, features, and / or advantages will be, or may become, apparent to one with skill in the art upon examination of the accompanying drawings and detailed description. All such additional systems, methods, features, and / or advantages are intended to be included within this description and protected by the accompanying claims.

[0027] The components within the drawings are not necessarily to scale relative to each other. Like reference characters indicate corresponding parts throughout the several views. These and other features of the present invention will become more apparent in the detailed description when reference is made to the accompanying drawings. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is a block diagram of a machine learning inference process using separate neural networks for simulation model data and simulation metadata according to an implementation described herein. [Diagram 2] FIG. 1 is a block diagram of another machine learning estimation process according to an implementation described herein. [Diagram 3] FIG. 1 is a diagram of an exemplary computing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The present disclosure can be more readily understood by referring to the following detailed description, examples, drawings, and the above and following descriptions thereof. However, before the present device, system, and / or method are disclosed and described, it should be understood that the present disclosure is not limited to the specific device, system, and / or method disclosed, unless otherwise specified, and therefore may, of course, vary. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.

[0030] The following description is provided as an enabling teaching. To this end, those skilled in the art will recognize and appreciate that many modifications can be made while still obtaining beneficial results. It will also become apparent that some of the desirable benefits can be obtained by selecting some of the features without utilizing other features. Thus, those skilled in the art will recognize that many modifications and adaptations may be possible and may even be desirable in certain circumstances and are contemplated by the present disclosure. Thus, the following description is provided as an illustration of principles, and not by way of limitation thereof.

[0031] As used throughout, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, a reference to "a 3D model" can include two or more such 3D models unless the context dictates otherwise.

[0032] Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when values ​​are expressed as approximations, by use of the antecedent "about," it will be understood that the particular value forms another embodiment. It will be further understood that each of the endpoints of a range is meaningful both in conjunction with the other endpoint, and independently of the other endpoint.

[0033] As used herein, "optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes cases where the event or circumstance occurs as well as cases where it does not occur.

[0034] The present disclosure relates to methods and systems for estimating the computational cost of a simulation (e.g., a finite element analysis (FEA) simulation) using machine learning techniques. The computational cost estimation described herein is performed before the desired simulation begins, i.e., based on a priori knowledge of the desired simulation. By providing a prediction before the desired simulation is undertaken, a stakeholder (e.g., a user, a customer, etc.) is given an understanding of the cost (e.g., time, computing resources, monetary cost, etc.) of the desired simulation. Such information enables a stakeholder to make decisions, including, but not limited to, whether to proceed with the desired simulation, how to proceed with the desired simulation, and / or what computing resources to use. It should be appreciated that such information is needed in the art, but is not provided by conventional systems and methods, particularly using machine learning methods. For example, machine learning techniques are used to estimate costs in the form of memory and core time, where core time is the amount of calculations that can be performed by a single computing core in one hour. In some implementations, the machine learning techniques described herein are used to predict multiple computational cost characteristics for a simulation process. Here, each respective computational cost characteristic for the simulation process is based on a different set of computational resources and / or solver options. It should be understood that this provides further information and options to the participants, and optionally, such information can then be used to select the optimal set of computing resources (e.g., memory, number of cores, etc.) required to complete the simulation given the participants' resources and circumstances. It should be understood that the computer processor may include multiple computational cores. The use of machine learning techniques can greatly improve the efficiency and accuracy of estimation, especially in time domain analysis. Here, in current technology, an initial time step must be solved and then extrapolated, leaving a significant room for error when the time to convergence is unknown.Machine learning is well suited to these types of problems due to the enormous complexity of the potential inputs to the simulation.

[0035] Referring now to FIG. 1, a block diagram of a machine learning estimation process with separate neural networks for simulation model data and simulation metadata is shown. The machine learning estimator function is intended to perform regressions on the amount of memory and total execution time that the simulation will incur. From the execution time and the number of computational cores utilized, an amount of core-time cost of the simulation can be calculated by the machine learning estimator. In other words, the machine learning estimator can output a core-time cost amount. Two main input sources to the estimator are computer-aided design (CAD) geometry and simulation metadata. Optionally, other input sources to the estimator can include, but are not limited to, metadata related to supporting operations such as post-processing of computational hardware data and metrics. The CAD geometry is divided into discrete elements, called meshes, which aid in the construction of finite element equations used to calculate the solution to the problem. The metadata can include, but are not limited to, the following: ● Time iteration (for dynamic solutions) ● Material properties ● Cores used ● Physical phenomenon specific parameters (solution equation, load type, boundary condition type) ● Number of elements for each physical phenomenon ● Total number of elements in the mesh ● Type of analysis (e.g. static, quasi-static, dynamic) ● Solution domain (e.g., time or frequency) ● The type of linear solver (e.g., direct or iterative methods). ● Types of queue preconditioners ● Nonlinearity ● Nonlinear solver options ● Level of adaptive grid refinement It should be understood that the specific metadata outlined above is provided by way of example only, and the present disclosure contemplates that other simulation metadata may be used with the machine learning models described herein. Target parameters to be considered as inputs to the model and estimated for new simulations are: ● Memory usage ● Simulation execution time It should be understood that the specific target parameters outlined above are provided by way of example only, and the present disclosure contemplates that other target parameters may be used with the machine learning models described herein.

[0036] In one implementation of the method, the estimation is achieved using a strategy of splitting the simulation into two sets of data that will be processed separately by different neural networks. The first set of data is related to the spatial model geometry (e.g., a CAD model). The spatial model geometry is contained within a CAD file, such as a STEP file. The STEP file format is well known in the art. The present disclosure contemplates that the spatial model geometry may be contained within other types of CAD files. It is contemplated that the geometry in question may be split into cells during the mesh generation stage of simulation pre-processing, for example, using algorithms such as those described in U.S. Patent Application Publication No. 2020 / 0050722, filed August 9, 2019, and entitled "HYBRID MESHING METHOD FOR FINITE ELEMENT ANALYSIS." The vertices of these cells may be directly exported as a point cloud 101, retaining any interpolated or resampled physical values ​​and boundary conditions from the simulation setup. Resolution adjustments will also be performed during resampling to ensure compatibility with the network architecture, ensuring that any changes in geometry are preserved as metadata in the point cloud 101. Additionally, statistics related to the estimator results (such as model extent, number of cells, etc.) can be preserved as external metadata and used as independent inputs to the final model outside the convolutional network.

[0037] A first machine learning model is created and pre-trained to process the point cloud 101. For example, it is contemplated that a convolutional neural network 102 is created and pre-trained on a 3D data benchmark such as Princeton's ModelNet40 (http: / / modelnet.cs.princeton.edu) or Stanford's ShapeNet (https: / / shapenet.cs.stanford.edu / ). The convolutional neural network 102 includes a processing component for the point cloud 101, and modular components 103. The design (or "architecture") of the convolutional neural network is defined by modular components 103 arranged together to process an input 3D point cloud (e.g., point cloud 101) and generate a representation of a fixed-length vector 105. The modular components 103 as used herein include, but are not necessarily limited to, convolutional filters ("convolutional" layers), affine transformations ("dense" layers), adaptive normalization and scaling transformations, pooling, and various nonlinear functions. It should be understood that a convolutional neural network is provided merely as an exemplary first machine learning model, and the present disclosure contemplates that other machine learning models, including gradient boosting decision trees and Gaussian process regression, may be trained and used with the systems and methods described herein.

[0038] The neural network includes a standardized model architecture with experimentally proven strength on benchmark classification and semantic segmentation tasks and trained to near the error metrics of current technology ("pre-training"). Methods for achieving low error rates on existing tasks are widely known and publicly available through open source and open license channels. Once the standardized model is selected, the neural network is then fine-tuned using proprietary simulation data to regress the desired variables. The fine-tuning process involves removing the final classification and semantic segmentation weight layers of the trained network and creating a new set of weight layers 121 (or "head network 121") designed to output values ​​within the desired function range.

[0039] Previous layers of the neural network 102, sourced from ModelNet40 or ShapeNet, hold pre-trained priors useful for recognizing patterns in 3D geometry, allowing the learning rate to be reduced and training resumed on the desired target variables ("fine-tuning"). The neural network architecture emphasizes a hierarchical representation of the local model topology, which correlates strongly with simulation run time under a variety of different physics analyses. The point cloud 101 is processed by the trained neural network 102 to create a fixed-dimensional vector representation 105 of the geometry.

[0040] The second set of data to be processed includes metadata 111 collected from the geometric model and simulation metadata generated from parsed parameters recorded from the configuration of the simulation run. Some features depend on the type of simulation physics, such as static analysis vs. transient analysis. Examples of these features may include mesh resolution, material properties, CPU cores used, or mesh elements for each physics to be solved. These features are discretized as categorical variables or as continuous floating-point values. The metadata 111 is then processed by a second machine learning model. For example, a combination of a neural oblivious decision tree and a fully connected neural network 106 can be used to process the metadata 111. A conventional wide-deep (https: / / arxiv.org / abs / 1606.07792) model can create a fixed-dimensional embedding of these features. It should be understood that the combination of a neural oblivious decision tree and a fully connected neural network is provided merely as an exemplary second machine learning model. This disclosure contemplates that other machine learning models may be trained and used with the systems and methods described herein.

[0041] Finally, it is contemplated that the fixed dimensional vector representation 105 obtained from the combination of these two neural networks (geometry and metadata) is then applied to a head network 121, each responsible for a particular regression target, typically either total memory usage or total execution time. The head network 121 may also include statistical estimates such as error bars associated with each metric. The resulting output is an estimate of total memory usage or total execution time based on training data collected from other simulations run on the same platform where the geometry vector representation and metadata may be collected.

[0042] Referring now to FIG. 2, an exemplary method for estimating the computational cost of a simulation process is shown. As used herein, a simulation process includes a simulation (e.g., FEA simulation), and optionally, file upload / download, mesh generation of model geometry, pre-processing of simulation data, and / or post-processing of simulation results. An exemplary feature dataset is shown in block 202. As described below, the feature dataset is an input to a machine learning model 204. Thus, the feature dataset is data that is analyzed by a trained machine learning model operating in reference mode to make predictions (also referred to as a "target" or "target set" of the machine learning model). The present disclosure contemplates that the feature dataset may be stored by a computing device (e.g., computing device 300 shown in FIG. 3). In some implementations, the feature dataset is maintained by the computing device or in a cloud storage environment. Alternatively or additionally, in some implementations, the feature dataset is accessed by the computing device, for example, over a network.

[0043] Alternatively, or in addition, depending on the implementation, the feature dataset is determined from a model feature dataset, which can optionally include a larger dataset, e.g., training data, validation data, and / or testing data. As described herein, determining the feature dataset can be accomplished using automated systems and methods. For example, an automated process can be performed to determine the feature dataset (i.e., any particular metadata) on which the machine learning model 204 makes predictions. This process can include, for example, identifying a representative sample of simulations, selecting a range of possible features (e.g., model geometry metadata, simulation metadata, computing environment metadata, etc.), running the simulations and extracting feature values, and using data science methods to determine which features are important. A model feature dataset used for training and testing, and optionally validation, is created using the important features. A large set of data points (simulations) can then be collected, features for those simulations can be extracted, and the data can be analyzed and trained iteratively to reach a desired confidence level and ensure sufficient coverage before adding more data. This allows for the creation of an initial machine learning model. The present disclosure contemplates that automated systems and methods can be implemented to continually collect data from cloud-based simulations and update model feature datasets to maintain and update the initial machine learning model. The automated systems and methods can generate the machine learning model 204. It should be understood that the machine learning model 204 can be updated to improve accuracy with new data points and to handle newly developed simulation capabilities.

[0044] It is also contemplated that the model feature dataset can be optionally updated frequently and the machine learning model can be retrained to accommodate newly developed simulation capabilities and transformations in the computing environment. As described herein, this can be accomplished using automated systems and methods. For example, in a cloud computing environment, the feature set is continuously updated by collecting anonymous CAD and simulation metadata generated by simulations run on the system without collecting actual, possibly proprietary, CAD or simulation files. Metadata from each simulation run on the system is recorded and added to a database. The machine learning model is frequently retrained to take advantage of the growing model feature dataset, improving prediction accuracy and enabling predictions based on new features. Furthermore, as the simulation software is updated and upgraded, simulation time and memory can be affected. Version numbers of various components of the software, including third-party software, are stored and included as part of the input feature set. As the software evolves, the estimations will evolve.

[0045] The feature dataset includes at least two types of data, for example, model geometry metadata 202a and simulation metadata 202b. Optionally, the feature dataset further includes computing environment metadata 202c. It should be understood that the feature dataset is not limited to model geometry, simulation, and computing environment metadata, and may include other features. The model geometry metadata 202a may include at least one of the following: the number of elements in a mesh, the number of vertices in a mesh, the surface area to volume ratio of the model, the minimum element size, the mesh quality metric, the element type and element order, the aspect ratio of the model, the number of edges in the model, the number of surfaces in the model, the number of finite volumes in the model, or the number of control points in the model. As discussed above, the model geometry metadata 202a may include metadata related to the geometry, the mesh, or both. It should be understood that the examples of model geometry metadata provided above are provided merely as exemplary model geometry metadata 202a. This disclosure contemplates other types of model geometry metadata for use with the techniques described herein. Optionally, the model is a computer-aided design (CAD) model, e.g., a model stored in STEP format. It should be understood that STEP files are provided merely as an exemplary CAD file type. This disclosure contemplates that other file types may be used. In some implementations, model geometry metadata is extracted from the CAD model. Metadata can be extracted from the geometry by analyzing NURBS (Non-Uniform Rational B-Spline) curves and surfaces that comprise the geometry, or possibly discrete geometry composed of points, edges, facets, and volumes, using techniques including, but not limited to, geometric feature detection to obtain metrics including smallest feature size, shortest edge length, number of parts, number of faces, and volume of the model.Metadata can also be extracted from meshed models through analyzing the mesh and obtaining metrics including vertex count, element count, histograms of element sizes, and minimum and average element quality. Techniques for extracting metadata from geometry and / or meshes are well known in the art and therefore will not be described in further detail herein.

[0046] As described above, the feature data set also includes simulation metadata 202b. The simulation metadata may include the type of analysis, solution domain, type of linear solver, preconditioner, time integration method, space integration method, degree of nonlinearity, nonlinear solver options, type and number of levels of adaptive mesh refinement, time step size, or number of frequencies, degrees of freedom of boundary conditions, type of boundary conditions, type of material model, material properties, number of cores, and / or physics-specific parameters. It should be understood that the above examples of simulation metadata are provided merely as exemplary simulation metadata 202b. The present disclosure contemplates other types of simulation metadata for use with the techniques described herein.

[0047] As mentioned above, the feature dataset optionally further includes computing environment metadata 202c. The computing environment metadata may include hardware information, number of processors, and / or bandwidth. It should be understood that the above examples of computing environment metadata are provided merely as exemplary computing environment metadata 202c. This disclosure contemplates other types of computing environment metadata for use with the techniques described herein.

[0048] As described above, the feature dataset (e.g., model geometry metadata 202a and simulation metadata 202b, and optionally, computing environment metadata 202c) is input to the machine learning model 204. The machine learning model 204 can be a deep neural network, a convolutional neural network, a gradient boosting decision tree, or a Gaussian process regression. It should be understood that the above examples of machine learning models are provided by way of example only. The present disclosure contemplates that other types of machine learning models can be used with the techniques described herein. The present disclosure contemplates that the machine learning model 204 can be selected, trained, and tested using a large training dataset encompassing the model geometry and simulation metadata, using training methods well known in the art. Once trained, the machine learning model 204 can be used in a reference mode to make predictions based on new feature datasets.

[0049] For example, in some implementations, the machine learning model 204 is a neural network. An artificial neural network (ANN) is a computing system that includes multiple interconnected neurons (e.g., also referred to as "nodes"). The present disclosure contemplates that the nodes can be executed using a computing device (e.g., a processing unit and memory as described herein). The nodes may be optionally arranged in multiple layers, such as an input layer, an output layer, and one or more hidden layers. Each node is connected to one or more other nodes in the ANN. For example, each layer is made of multiple nodes, and each node is connected to all nodes in the previous layer. The nodes in a given layer are not interconnected with each other. That is, the nodes in a given layer function independently of each other. As used herein, the nodes in the input layer receive data from outside the ANN, the nodes in the hidden layer modify the data between the input layer and the output layer, and the nodes in the output layer provide the results. Each node is configured to receive an input, execute an activation function (e.g., a binary step, linear, sigmoid, tanH, or rectified linear unit (ReLU) function), and provide an output according to the activation function. Additionally, each node is associated with a respective weight. The ANN is trained with a data set to minimize a cost function, which is a measure of the performance of the ANN. Training algorithms include, but are not limited to, backpropagation. The training algorithm adjusts the weights and / or biases of the nodes to minimize the cost function. It should be appreciated that any algorithm that finds a minimum of a cost function can be used to train the ANN.

[0050] It should be understood that the neural network is provided merely as an exemplary machine learning model. The present disclosure contemplates that the machine learning model can be any supervised, semi-supervised, or unsupervised learning model. Optionally, the machine learning model is a deep learning model, e.g., a convolutional neural network (CNN). A CNN is a type of deep neural network that has been applied, e.g., to image analysis applications. Unlike traditional neural networks, each layer in a CNN has multiple nodes arranged in three dimensions (width, height, and depth). A CNN can include different types of layers, e.g., convolutional, pooling, and fully connected (also referred to herein as "dense") layers. A convolutional layer includes a set of filters and performs the majority of the computations. A pooling layer is optionally inserted between convolutional layers to reduce computational power and / or control overfitting (e.g., by downsampling). A fully connected layer includes neurons, each neuron connected to all of the neurons in the previous layer. The layers are stacked in a manner similar to traditional neural networks. Machine learning models are well known in the art and therefore will not be described in further detail herein.

[0051] In block 206, a computational cost characteristic 206a is predicted for the simulation process using the machine learning model 204. In other words, the computational cost characteristic 206a is a target of the machine learning model 204. The machine learning model 204 predicts the computational cost characteristic 206a before the simulation process starts, i.e., based on a priori knowledge of the simulation process. In some implementations, the computational cost characteristic 206a is memory usage and / or time for the simulation process. Optionally, a number of core hours is estimated based at least in part on the time for the simulation process. Alternatively or in addition, the computational cost characteristic 206a is a time required for mesh generation, a time required for pre-processing, a time required for solving, a time required for post-processing, and / or a time required for the simulation process. In some implementations, the computational cost characteristic 206a is a time for the entire simulation process, while in other implementations, the computational cost characteristic 206a is a time for a portion of the simulation process. For example, the simulation process may include pre-processing, mesh generation, simulation, and post-processing steps. It should be understood that the pre-processing, mesh generation, simulation, and post-processing steps are provided merely as exemplary portions or steps of a simulation process. The present disclosure contemplates that the simulation process may include other portions or steps. Thus, the computational cost characteristics 206a estimated by the machine learning model 204 may be the time required for a pre-processing operation, the time required for a mesh generation operation, the simulation run time, the time required for a post-processing operation, or the sum of the time for two or more steps of the simulation process. It should be understood that the above characteristics are provided merely as exemplary computational cost characteristics 206a. The present disclosure contemplates predicting other types of computational cost characteristics using the techniques described herein.

[0052] Optionally, the machine learning model 204 predicts a plurality of computational cost characteristics for the simulation process, where each respective computational cost characteristic for the simulation process is based on a different set of computational resources and / or solver options. It should be understood that the computational cost characteristics are expected to vary depending on the computing resources and / or solver options. The plurality of computational cost characteristics predicted by the machine learning model 204 may optionally be transmitted to and / or displayed by a stakeholder (e.g., a user, a customer, etc.). It should also be understood that providing a plurality of predictions provides a stakeholder with useful information, e.g., information used to inform decision making. For example, the prediction in block 206 may indicate that increasing the number of processors initially results in increased benefits, but then the benefits begin to plateau (e.g., the benefit curve is asymptotic). In this example, the stakeholder may select a computing resource for the application that provides a desired benefit.

[0053] Optionally, in some implementations, inputting the feature dataset into the machine learning model includes inputting the model geometry metadata into a first machine learning model (e.g., CNN 102 shown in FIG. 1 ) and inputting the simulation metadata into a second machine learning model (e.g., Neural Forgetting Decision Tree and Fully Connected Neural Network 106 shown in FIG. 1 ). In these implementations, a fixed dimensional representation vector (e.g., vector 105 shown in FIG. 1 ) is created from the output of each of the first and second machine learning models, and then the fixed dimensional representation vector is analyzed and a computational cost characteristic for the simulation is predicted. Optionally, the step of analyzing the fixed dimensional representation vector and predicting a computational cost characteristic for the simulation includes performing a regression analysis.

[0054] It should be understood that the logical operations described herein with respect to the various figures may be implemented as (1) a series of computer-implemented acts or program modules (i.e., software) executing on a computing device (e.g., the computing device illustrated in FIG. 3), (2) interconnected machine logic circuits or circuit modules (i.e., hardware) within a computing device, and / or (3) a combination of software and hardware of a computing device. Thus, the logical operations described herein are not limited to any particular combination of hardware and software. The implementation is a matter of choice depending on the performance and other requirements of the computing device. Thus, the logical operations described herein are variously referred to as operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules may be implemented in software, in firmware, in dedicated digital logic, and any combination thereof. It should also be understood that more or fewer operations may be performed than shown in the figures and described herein. These operations may also be performed in a different order than described herein.

[0055] Referring to FIG. 3, an exemplary computing device 300 is shown on which the methods described herein may be implemented. It should be understood that the exemplary computing device 300 is merely one example of a suitable computing environment on which the methods described herein may be implemented. Optionally, the computing device 300 may be a well-known computing system, including, but not limited to, a personal computer, a server, a handheld or laptop device, a multiprocessor system, a microprocessor-based system, a network personal computer (PC), a minicomputer, a mainframe computer, an embedded system, and / or a distributed computing environment or device including any of the above systems. A distributed computing environment allows remote computing devices connected to a communication network or other data transmission medium to perform various tasks. In a distributed computing environment, program modules, applications, and other data may be stored on local and / or remote computer storage media.

[0056] In its most basic configuration, computing device 300 typically includes at least one processing unit 306 and a system memory 304. Depending on the actual configuration and type of computing device, system memory 304 may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or some combination of both. This most basic configuration is illustrated in FIG. 3 by dashed line 302. Processing unit 306 may be a standard programmable processor that performs arithmetic and logical operations necessary for the operation of computing device 300. Computing device 300 may also include a bus or other communication mechanism for communicating information between various components of computing device 300.

[0057] Computing device 300 may have additional features / functionality. For example, computing device 300 may include additional storage, such as removable storage 308 and non-removable storage 310, including but not limited to magnetic or optical disks or tape. Computing device 300 may also include network connections 316 that allow the device to communicate with other devices. Computing device 300 may also have input devices 314, such as a keyboard, mouse, touch screen, etc. Also included may be output devices 312, such as a display, speakers, printer, etc. Additional devices may be connected to the bus to facilitate communication of data between components of computing device 300. All these devices are well known in the art and need not be described at length here.

[0058] The processing unit 306 may be configured to execute program code encoded in a tangible computer readable medium. Tangible computer readable medium refers to any medium capable of providing data that causes the computing device 300 (i.e., machine) to operate in a particular manner. A variety of computer readable media may be utilized to provide instructions to the processing unit 306 for execution. Exemplary tangible computer readable media may include, but are not limited to, volatile, non-volatile, removable, and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. The system memory 304, removable storage 308, and non-removable storage 310 are all examples of tangible computer storage media. Exemplary tangible computer-readable recording media include, but are not limited to, integrated circuits (e.g., field programmable gate arrays or application specific ICs), hard disks, optical disks, magneto-optical disks, floppy disks, magnetic tape, holographic storage media, solid state devices, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory, or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices.

[0059] In an example implementation, the processing unit 306 may execute program code stored in the system memory 304. For example, a bus may carry data to the system memory 304, and the processing unit 306 receives and executes instructions from the memory 304. Data received by the system memory 304 may optionally be stored on removable storage 308 or non-removable storage 310 before or after execution by the processing unit 306.

[0060] It should be understood that the various techniques described herein may be implemented in connection with hardware or software, or, where appropriate, a combination of both. Thus, the methods and apparatus of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in a tangible medium, such as a floppy diskette, CD-ROM, hard drive, or any other machine-readable storage medium, such that when the program code is loaded into and executed by a machine, such as a computing device, the machine becomes an apparatus for implementing the presently disclosed subject matter. In the case of execution of program code on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. One or more programs may implement or utilize the processes described in connection with the presently disclosed subject matter, for example, through the use of an application programming interface (API), a reusable control, or the like. Such programs may be implemented in a high-level procedural or object-oriented programming language to interact with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language can be a compiled or interpreted language, and combined with hardware implementations.

[0061] Although the present subject matter has been described in specific language of structural features and / or methodological acts, it is to be understood that the present subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. [Explanation of symbols]

[0062] 101 point cloud 102 Convolutional Neural Networks 103 Module Components 105 Fixed-length vectors, fixed-dimension vector representations 106 Neural forgetting decision trees and fully connected neural networks 111 Metadata 121 Weight Layer, Head Network 202a Model Geometry Metadata 202b Simulation Metadata 202c Computing Environment Metadata 204 Machine Learning Models 206a Computational cost characteristics 300 computing devices 304 System Memory 306 Processing Unit 308 Detachable Storage 310 Non-removable Storage 312 Output Devices 314 Input Devices 316 Network Connection

Claims

1. 1. A computer-implemented method for estimating computational costs of a simulation, comprising: conducting a plurality of simulations to obtain an initial simulation data set; selecting one or more types of metadata from the initial simulation data set to create a model features data set; training a machine learning model using the model feature dataset; inputting a feature dataset into the machine learning model, the feature dataset including model geometry metadata and simulation metadata, the model geometry metadata being associated with a model representing a physical system or device; using the machine learning model to predict computational cost characteristics for a simulation process; 23. A computer implemented method comprising:

2. The computer-implemented method of claim 1 , wherein the computational cost characteristics are predicted prior to performing the simulation process.

3. 3. The computer implemented method of claim 1 or 2, wherein the model geometry metadata includes at least one of the number of elements in a mesh, the number of vertices in the mesh, the surface area to volume ratio of a model, a minimum element size, a mesh quality metric, element type and element order, an aspect ratio of the model, the number of edges in the model, the number of surfaces in the model, the number of finite volumes in the model, or the number of control points in the model.

4. The computer-implemented method of claim 3 , wherein the model is a computer-aided design (CAD) model.

5. The computer implemented method of claim 4 , further comprising extracting the model geometry metadata from the CAD model.

6. 6. The computer implemented method of claim 1, wherein the simulation metadata includes at least one of: analysis type, solution domain, linear solver type, preconditioner, time integration method, spatial integration method, nonlinearity, nonlinear solver options, type and number of levels of adaptive grid refinement, time step size, or number of frequencies, degrees of freedom of boundary conditions, type of boundary conditions, type of material model, material properties, number of cores, or physical phenomenon specific parameters.

7. The computer implemented method of claim 1 , wherein the computational cost characteristic is at least one of memory usage or time for the simulation process.

8. The computer implemented method of claim 7 , wherein the computational cost characteristic is a time for a portion of the simulation process.

9. The computer implemented method of claim 7 , further comprising estimating a number of core hours based at least in part on a time for the simulation process.

10. 7. The computer implemented method of claim 1, wherein the computational cost characteristic is at least one of: a time required for mesh generation, a time required for pre-processing, a time required for solution, a time required for post-processing, or a time required for the simulation process.

11. 11. The computer-implemented method of claim 1, wherein using the machine learning model to predict a computational cost characteristic for a simulation process includes a plurality of computational cost characteristics for the simulation process, each respective computational cost characteristic for the simulation process being based on a different set of computational resources and / or solver options.

12. 12. The computer-implemented method of claim 1, wherein the machine learning model is a deep neural network, a convolutional neural network, a gradient boosting decision tree, or a Gaussian process regression.

13. The computer-implemented method of claim 1 , wherein the feature dataset further comprises computing environment metadata.

14. The computer implemented method of claim 1, wherein the one or more types of metadata include model geometry metadata, simulation metadata, computing environment metadata, or a combination thereof.

15. performing one or more additional simulations; and supplementing the model feature dataset with metadata from the one or more additional simulations; and training and testing the machine learning model using the imputed model feature dataset; The computer implemented method of claim 1 further comprising:

16. The computer-implemented method of claim 1 , wherein training the machine learning model is performed iteratively.

17. inputting the feature dataset into a machine learning model; inputting the model geometry metadata into a first machine learning model; and inputting the simulation metadata into a second machine learning model; 17. A computer implemented method according to any one of claims 1 to 16, comprising:

18. creating a fixed dimensional representation vector from the output of each of the first and second machine learning models; analyzing the fixed dimensional representation vector to predict the computational cost characteristics for the simulation process; 20. The computer implemented method of claim 17, further comprising:

19. 20. The computer implemented method of claim 18, wherein analyzing the fixed dimensional representation vector to predict the computational cost characteristic for the simulation process comprises performing a regression analysis.

20. 1. A system for estimating computational costs of a simulation, comprising: a computing device including a processor and a memory, the memory being configured to, when executed by the processor, cause the processor to: conducting a plurality of simulations to obtain an initial simulation data set; selecting one or more types of metadata from the initial simulation data set to create a model features data set; training a machine learning model using the model feature dataset; inputting a feature dataset into the machine learning model, the feature dataset including model geometry metadata and simulation metadata, the model geometry metadata being associated with a model representing a physical system or device; using the machine learning model to predict computational cost characteristics for a simulation process; A system storing computer-executable instructions to cause a

Images