Measurements and simulation correction using deep learning

Abstract

A method for improving the accuracy of a simulation model may include measuring characteristic values ​​at one or more nodes of a physical embodiment corresponding to the simulation model, inputting the measured characteristic values ​​into a trained machine learning function corresponding to the simulation model, and receiving one or more predicted values ​​from the trained machine learning function that are based at least in part on the input measured characteristic values.

Description

[Technical field]

[0001] This disclosure claims the benefit of U.S. Provisional Application No. 62 / 901,754, filed Sep. 17, 2019, entitled “SIMULATION CORRECTION SYSTEM,” which is hereby incorporated by reference in its entirety.

[0002] FIELD OF THE DISCLOSURE This disclosure relates to test and measurement instruments, and more particularly to systems and methods for comparing device measurement data from a test and measurement instrument with simulation data for the device and correcting the simulation data. [Background technology]

[0003] As well as the actual measurements of the device under test (DUT), To The corresponding simulation model is an important design tool for engineers. For a given circuit design, the simulation model produces a mathematically generated prediction of the circuit behavior. Actual measurements from the DUT can be generated by test and measurement equipment such as oscilloscopes. However, in systems currently in use, the simulation model and the actual measurements often disagree with each other. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2005 / 0022141 [Patent Document 2] US Patent Application Publication No. 2018 / 0203956 [Patent Document 3] US Patent Application Publication No. 2010 / 0088657 Summary of the Invention [Problem to be solved by the invention]

[0005] Traditional tools related to circuit simulation and analysis include the Serial Data Link Analysis (SDLA) tool, which can be used to model transmission channels for high-speed serial applications. The SDLA tool can modify waveforms and display waveforms that are not accessible to the user, but errors in the model will degrade the model results. For experienced users, it is often necessary to refine the model to obtain results that are consistent with measurements.

[0006] Other conventional tools include dedicated software that can analyze measured Time Domain Reflectometer (TDR) data to generate models of, for example, transmission line lengths, impedances, and discontinuities. of It applies to circuit configurations and does not address the various issues related to model to measurement inconsistencies.

[0007] Embodiments of the disclosed technology address these and other deficiencies of the prior art.

[0008] Aspects, features and advantages of embodiments of the disclosed technology will become apparent from the following description of the embodiments, taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a flow diagram illustrating an example of a method for training a machine learning function to create an improved simulation model in accordance with certain embodiments of the disclosed technology. [Diagram 2] FIG. 2 is a flow diagram illustrating an example method for predicting or inferring a set of revised model parameters that better match measured results in accordance with certain embodiments of the disclosed technology. [Diagram 3] FIG. 3 is a block diagram illustrating an example test and measurement system in accordance with certain embodiments of the disclosed technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiments of the disclosed technology generally include systems and methods that can improve predictive results of simulation models.

[0011] FIG. 1 is a flow diagram illustrating an example method 100 for training a machine learning facility to create an improved simulation model, in accordance with certain embodiments of the disclosed techniques.

[0012] At 102, information is received relating to one or more parameters of individual components of the simulation model. In some embodiments, all of the individual components are used, while in other embodiments, only a portion of the individual components of the circuit simulation are used. The individual components may include, but are not limited to, resistors, transistors, capacitors, inductors, diodes, op amps, voltage sources, current sources, and transmission lines. The parameters of the individual components may include, for example, resistance, current, impedance, temperature coefficient, parasitic capacitance, transmission line length, transmission line width, material permittivity, and geometry. As will be described below, embodiments of the present invention are applicable to many types of simulation models, and the circuit simulation model described with reference to the figures is only one example.

[0013] At 104, the generated data set of values ​​is combined with simulation result values ​​for each of the component parameters of the simulation model. The inputs to the simulation may include parameter values ​​for individual components in the circuit identified in step 102, and the output of the simulation may include a very large data set of individual instances of the inputs to the simulation along with the output of the simulated circuit. The instances may be likened to the rows of a spreadsheet, while the columns of the spreadsheet would represent the values ​​of the component parameters themselves and the output predicted by the simulation. There may be any number of instances in the data set, and typically the more instances there are the better the machine learning performance. An example data set may include millions of instances. In addition to the component parameters, the data set may include simulated test data for a circuit created from the component values ​​of the particular instances.

[0014] For example, in a greatly simplified example of a resistive network having three resistors, each instance may have simulated (i.e., generated) values ​​of current and voltage at three different nodes in the circuit having the resistive network, in addition to resistance parameter values ​​for each of the three resistors and signals input to the nodes of the circuit. The simulated values ​​of current and voltage at these nodes are generated by the model based on the input signals. In the context of this example, the resistance parameters are the most important parameters of this simple resistive network, so the terms "resistance value" and "resistor value" are used interchangeably with "resistance parameter value" in the following description of this example. The simulated values ​​may include waveforms and need not be static values; that is, in some embodiments, the simulated values ​​may be a collection of values ​​that change over time. In certain embodiments, the waveform may undergo a data reduction procedure, for example, to reduce the waveform to attributes of a smaller waveform. Also, the simulation can predict values ​​for any node in the circuit and is not limited to a particular number. Additionally, for multiple instances of a simulation, a number of different simulation values ​​may be generated, which may reference component parameters of the model and relate to the parameters of the individual components described above.

[0015] For three-dimensional (3D) electromagnetic simulations, model parameters may include, but are not limited to, line widths, material dielectric constants, and other geometries that may vary with manufacturing. The methodology can also be applied to other simulation domains, such as mechanical and thermal, provided that measurement data corresponds to the output from the simulation.

[0016] For serial data link simulation, model parameters may include, but are not limited to, transmitter and receiver equalization, S-parameters, impulse responses, other filter responses, inter-symbol interference, analog model characteristics, etc.

[0017] Other models that may benefit from embodiments of the present invention include those in the mechanical and thermal simulation domains, correlating measurement data with simulated output data and modifying such models to better predict actual measurements.

[0018] In certain embodiments, the input parameters of a data set may be constrained to fall within a range of values, for example, the value of the resistance parameter for each of the three resistors in the example above may be constrained to be between 100 ohms and 1000 ohms when generating the training data set.

[0019] During the simulation (i.e., during the generation of the data set generated in step 104), the simulation may simulate and store output values ​​for various test nodes in the resistor network (or at the endpoints of the resistor network) in addition to creating instances by varying the resistance value for each of the three resistors individually. Each particular instance of the generated data set may have at least one different component parameter value from all other instances. For example, for the first set of instances generated, the resistance values ​​of resistor 1 (R1) and resistor 2 (R2) may be fixed, while the resistance values ​​of resistor 3 (R3) may vary in 10 ohm steps for each instance, between 100 ohms and 1000 ohms. The first four instances of such a data set are shown in the exemplary data set shown in Table 1. [Table 1]

[0020] Inputs to the value simulator may be created manually or generated according to an algorithm, for example. Recall that the inputs in the above example are the resistor values ​​of R1, R2, and R3. As mentioned above, various combinations of these resistor values, set within the limits for the input parameters, may be generated manually or mechanically. For example, a loop program may be set to step each resistor value from a minimum to a maximum value through the progression of the loop, thereby generating all possible combinations as separate instances. In another example, a Monte Carlo process may be used to generate specific component parameter input values ​​for each instance within the value limits. These input values ​​may also be generated randomly.

[0021] The simulator accepts input values ​​and, in step 104, generates simulated output values ​​using the circuit simulator that are stored for the particular instance used to generate the output values. The output values ​​simulated by the simulator and stored in the data set in step 104 may include currents, voltages, etc., as shown in Table 1. In some embodiments, the simulated output values ​​may include waveforms of particular test nodes of the network at particular times. As will be described below, there may be multiple values ​​generated for each instance in the data set, and such values ​​are not limited to only currents and voltages as shown in Table 1. In some embodiments, the output values ​​may be determined in the time domain, while in other embodiments, the output values ​​may be determined in the frequency domain.

[0022] At 106, a training data set including the simulation model component parameters from step 102 and the corresponding simulated values ​​generated at step 104 is provided as training data input to the machine learning function. Table 1 may be considered a greatly simplified training data set for purposes of illustration. In practice, the data set may have thousands or millions of instances (rows) and hundreds or thousands of columns representing the model component parameters and their simulated values. In some embodiments, multiple values ​​are specified for each component in the network, which may include hundreds of components. Each individual component may have values ​​in the data set for, for example, resistance, impedance, temperature coefficient, parasitic capacitance, transmission line length, transmission line width, material permittivity, and geometry. Any desired output that may be generated from the circuit simulator may be stored as a value (column) for each instance.

[0023] After the machine learning function receives its training data in step 106, a trained network is created in step 108. In step 108, for example, the neural network may read input from a first instance and generate a predicted result. The neural network then compares the generated predicted result with the data (also included in the instance) used to create the simulation result and uses back propagation to change the weights or biases in the neural network so that the next prediction is closer to the original data value. In the example of Table 1, the input to the network is the simulation result (node ​​1 predicted voltage, node 2 predicted voltage, etc.) and the predicted output is the predicted value of R1, R2, R3, which is compared during training to the original values ​​of R1, R2, and R3 used to create the simulation result. Training continues to progress through all instances until the network is fully trained. In some embodiments, the training makes multiple passes through the training database as indicated by optional loopback 109. In some embodiments, the neural network may permute the order in which instances appear in the training data set to avoid training bias.

[0024] Step 108 may use supervised or unsupervised machine learning. Supervised machine learning, as used herein, generally refers to machine learning based on a training set that includes labeled data. Unsupervised machine learning generally refers to the "study" of a training set that includes mostly unlabeled data to train a neural network. Machine learning functions may utilize specific techniques such as a Bayesian approach, random forests, regression models, classification models, etc.

[0025] FIG. 2 is a flow diagram illustrating an example method 200 for predicting or inferring a set of revised model parameters that are more consistent with measurements, in accordance with certain embodiments of the disclosed technology.

[0026] A physical embodiment, such as a suitable device under test (DUT), may be measured in step 202. This physical embodiment may correspond to a circuit model previously used for simulation, for example, as described above. In measuring the physical embodiment in step 202, a user may use test probes to actually physically connect to one or more test points (e.g., nodes, lines, traces, pins, or individual components) on the DUT and then obtain actual data, waveforms, or measurements from the DUT.

[0027] Once these measurements are obtained, the measurements obtained in step 202 may be applied to the trained learning network in step 204 to predict or infer a set of modified simulation model parameters that are more consistent with the measurements than the original predictive model. In certain embodiments, the measurement data may be adjusted to account for the uncertainty of the measurements and reflect a set of possible measurements. For example, if the DC gain of a measurement has an accuracy of ±2%, then up to 2% high signals and up to 2% low signals may be stored as additional training sets or additional instances of the original training set. The machine learning network may be improved (i.e., updated or retrained) in step 206 using the additional instances or new training data sets described above. The result of retraining the machine learning network in step 206 is a set of nominal model parameters, each of which will have a range of uncertainty due to measurement error.

[0028] Another aspect of the system is that the trained learning network has the ability to add previously unspecified components to the model and is not limited to simply adjusting the parameters of existing models.

[0029] The original simulation model is then updated with the modified model parameters generated by the updated learning network, and then in step 208, the output of the modified model is compared against the set of measurements taken in step 202. If successful, the updated simulation model will match the actual measurements better than the original simulation model.

[0030] If a physical implementation, such as a circuit, is not behaving as expected, an updated simulation model can provide insight into what may be different from the intended design and what parameters may be contributing to these differences. Adjusted simulation models can also provide a reasonable starting point for revising the design.

[0031] FIG. 3 is a block diagram illustrating a computing system 300 in which certain embodiments of the disclosed technology may be implemented. The computing system 300 may be general purpose or special purpose. In one example, the computing system 300 is a test and measurement system. In this example, the system 300 includes a computing device 320, such as a test and measurement instrument 320. In one embodiment, the test and measurement instrument may be an oscilloscope. The system 300 also includes a machine 310 that hosts a machine learning function 312 configured to use supervised or unsupervised machine learning techniques. In one example embodiment, the machine 310 is a computing device, and the machine learning function 312 may be a combination of computer operations and memory running on the computing device. The machine 310 may be local (within the same device) or remote to the computing device 320. The machine 310 may be a virtualized machine running, for example, in a computer network cloud.

[0032] The computing device 320 has at least one processor 322, at least one input 324, and an output 326 (e.g., a display device or other output). In this example, the computing device 320 also has internal memory 328. In certain embodiments, the system 300 further includes external storage 330, such as, for example, a remote server. The input 324 may be configured to receive measurement data, either directly from measurements or from data extracted from another measurement device. The input 324 may also be configured to receive previously stored data or data from an information cloud.

[0033] Either or both of the internal memory 328 and the external storage 330 may store a simulation model, such as a circuit simulation model. The processor 322 of the computing device 320 is configured to receive information regarding parameters of particular components of the simulation model, simulate values ​​for each of the component parameters of the simulation model, and provide them as inputs to the machine learning function 312.

[0034] The machine learning function 312 applies machine learning to the simulated values ​​to generate predicted values ​​for each input parameter and compares the predicted values ​​to the corresponding simulated values. The machine learning function 312 can adjust the simulation model based on the comparison of the predicted values ​​to the simulated values.

[0035] Embodiments of the present invention can be used for various types of modeling. As mentioned above, embodiments of the present invention are useful for improving simulation models.

[0036] Embodiments of the disclosed technology greatly simplify the design process for common circuits, providing the ability to rapidly improve models and verify how a device is assembled. This model refinement can potentially provide expert insight without the need for an expert user. Various novel aspects of the disclosed technology include automatically training neural networks with simulation data and using measured data to inform simulation models.

[0037] Combining design intent information from simulation with as-assembled performance from measurements provides a more complete and relevant understanding of device performance. Another application is for production test, where a small number of measurements can be used to indirectly measure subcircuit parameters with a certain degree of certainty.

[0038] In certain embodiments, a machine learning network may be trained with data in which a nominal circuit is replaced with a variety of possible manufacturing defects (e.g., missing parts, shorts in the circuitry, incorrect parts, etc.). In such embodiments, parameters may be varied across possible fault values ​​in addition to normal tolerance ranges. The measured data may be fed into a trained neural network to infer malfunctions on the production line, for example, based on what defects are likely to be present. Such implementations facilitate automated or semi-automated troubleshooting and corrective action efforts.

[0039] Aspects of the disclosed technology can operate on specially created hardware, firmware, digital signal processors, or specially programmed general-purpose computers, including processors that operate according to programmed instructions. The term controller or processor in this application contemplates microprocessors, microcomputers, ASICs, dedicated hardware controllers, and the like.

[0040] Aspects of the disclosed technology may be implemented in computer-available data and computer-executable instructions, such as one or more program modules, executed by one or more computers (including a monitoring module) or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., which, when executed by a processor in a computer or other device, perform particular tasks or implement particular abstract data formats. Computer-executable instructions may be stored in computer-readable storage media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, etc. As will be appreciated by those skilled in the art, the functionality of the program modules may be combined or distributed as desired in various embodiments. Furthermore, such functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc.

[0041] Certain data structures may be used to more effectively implement one or more aspects of the disclosed technology, and such data structures are considered to be within the scope of the computer-executable instructions and computer-usable data described herein.

[0042] The disclosed aspects may, in some cases, be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried by or stored on one or more computer-readable media that may be read and executed by one or more processors. Such instructions may be referred to as computer program products. A computer-readable medium as described herein means any medium that can be accessed by a computing device. By way of example, and not limitation, computer-readable media may include computer storage media and communication media.

[0043] Computer storage media means any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital video disc (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any other volatile or non-volatile removable or non-removable media implemented in any technology. Computer storage media excludes signals themselves and transitory forms of signal transmission.

[0044] A communication medium refers to any medium capable of communicating computer readable information. By way of example, and not limitation, communication media may include coaxial cables, fiber optic cables, air, or any other medium suitable for communicating electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals. Working Example

[0045] In the following, examples useful for understanding the technology disclosed in the present application are presented. The embodiment of the technology may include one or more of the examples described below and any combination thereof.

[0046] Example 1 is a method of adjusting a simulation model, comprising steps of measuring characteristic values ​​at one or more nodes of a physical embodiment corresponding to the simulation model, inputting the measured characteristic values ​​to a trained machine learning function corresponding to the simulation model, and receiving one or more prediction values ​​from the trained machine learning function that are based at least in part on the input measured characteristic values.

[0047] Example 2 is the method of example 1, further comprising modifying at least a portion of the simulation model based at least in part on the received one or more predictions to generate a modified simulation model.

[0048] Example 3 is a method of any of Examples 1-2, further including modifying at least a portion of the modified simulation model based at least in part on another prediction value received from the trained machine learning function, to generate a further modified simulation model.

[0049] Example 4 is the method of any one of Examples 1 to 3, further comprising training a machine learning function.

[0050] Example 5 is a method of any of Examples 1 to 4, wherein the process of training the machine learning function includes a process of receiving information regarding at least one parameter of at least one component of a simulation model, and a process of inputting a set of training data to the machine learning function, the set including generated simulated values ​​associated with the at least one parameter of the at least one component of the simulation model.

[0051] Example 6 is the method of any of examples 1-5, wherein the set of training data comprises a set of training data created using a Monte Carlo simulation.

[0052] Example 7 is a method of any of Examples 1 to 6, wherein the simulation model is a circuit simulation model, and the circuit simulation model includes at least one component selected from the group consisting of a resistor, a transistor, a capacitor, an inductor, a diode, an operational amplifier, a voltage source, a current source, and a transmission line.

[0053] Example 8 is a method of any of Examples 1 to 7, wherein the simulation model is a circuit simulation model, and the at least one parameter is selected from the group consisting of resistance, impedance, temperature coefficient, parasitic capacitance, transmission line length, transmission line width, dielectric constant of a material, and geometry.

[0054] Example 9 is a method of any one of Examples 1 to 8, wherein the process of modifying at least a portion of the circuit simulation model includes a process of replacing the simulation model stored in a memory with the modified simulation model.

[0055] Example 10 is the method of any one of Examples 1 to 9, wherein the simulation model is a circuit simulation model, a three-dimensional electromagnetic model, a serial data link simulation model, a thermal simulation model, or a motion simulation model.

[0056] An eleventh embodiment is the method according to any one of the first to ninth embodiments, in which the simulation model is a SPICE circuit model.

[0057] Example 12 is a test and measurement system that includes a machine configured to host a machine learning function and a test and measurement device having one or more processors configured to measure property values ​​at one or more nodes of a physical instance constructed according to a simulation model, input the measured property values ​​to a trained machine learning function corresponding to the simulation model, and receive one or more prediction values ​​from the trained machine learning function that are based at least in part on the input measured property values.

[0058] Example 13 is the test and measurement system of example 12, wherein the one or more processors are configured to improve the simulation model based at least in part on the received prediction values ​​to generate an improved simulation model.

[0059] Example 14 is a test and measurement system of any of Examples 12 to 13, wherein the one or more processors are further configured to receive information of at least one parameter of at least one component of the simulation model and input a set of training data to the machine learning function, the training data including generated simulation values ​​associated with the at least one parameter of the at least one component of the simulation model.

[0060] Example 15 is a test and measurement system of any of Examples 12 to 14, wherein the simulation model is a circuit simulation model, and at least one parameter is selected from the group consisting of resistance value, impedance, temperature coefficient, parasitic capacitance, transmission line length, transmission line width, dielectric constant of material, and geometric arrangement.

[0061] Example 16 is the test and measurement system of any of Examples 12 to 15, further comprising a memory configured to store the simulation model.

[0062] Example 17 is a test and measurement system of any of Examples 12 to 16, wherein the one or more processors are further configured to further improve the simulation model based at least in part on at least one predicted value received for the improved simulation model.

[0063] Example 18 is the test and measurement system of any of Examples 12-17, wherein the machine learning functionality is configured to operate in one of the one or more processors of the test and measurement instrument.

[0064] Example 19 is the test and measurement system of any of Examples 12-18, wherein the machine learning functionality is configured to operate on a processor separate from the one or more processors of the test and measurement instrument.

[0065] Example 20 is a test and measurement system of any of Examples 12 to 19, wherein the simulation model is a circuit simulation model, a three-dimensional electromagnetic model, a serial data link simulation model, a thermal simulation model, or a motion simulation model.

[0066] The above-described versions of the disclosed subject matter have many advantages that have been described or that will be apparent to those of skill in the art. Nevertheless, not all of these advantages or features are required in every version of a disclosed device, system or method.

[0067] In addition, the description of this application refers to specific features. It should be understood that the disclosure herein includes all possible combinations of those specific features. When a specific feature is disclosed in connection with a particular aspect or embodiment, that feature can also be used in connection with other aspects and embodiments, to the extent possible.

[0068] Furthermore, when the application refers to a method having two or more defined steps or processes, those defined steps or processes may be performed in any order or simultaneously, unless the context precludes such a possibility.

[0069] For convenience of illustration, specific embodiments of the invention have been illustrated and described, but it will be understood that various modifications can be made therein without departing from the spirit and scope of the invention. Accordingly, the invention should not be limited, except as by the appended claims.

Claims

1. A method for adjusting a simulation model which receives parameter values ​​of components constituting a corresponding physical embodiment and generates simulation values ​​of characteristic values ​​of said physical embodiment, comprising: measuring property values ​​at one or more nodes of said physical embodiment corresponding to said simulation model; inputting the measured characteristic values ​​into a machine learning function trained to receive the simulation values ​​as inputs and output the parameter values ​​corresponding to the simulation values, and retraining the machine learning function; receiving from the re-trained machine learning function one or more predictions based at least in part on the input measured property values; modifying at least a portion of the simulation model based at least in part on the received one or more forecast values ​​to generate a modified simulation model. A method of adjusting a simulation model comprising:

2. A method for adjusting the simulation model of claim 1, further comprising the step of modifying at least a portion of the modified simulation model based at least in part on another prediction value received from the retrained machine learning function, to generate a further modified simulation model.

3. 1. A test and measurement system comprising: A machine configured to host machine learning functionality; and measuring characteristic values ​​at one or more nodes of a corresponding physical instance constructed according to a simulation model that receives parameter values ​​of components that make up the physical instance and generates simulation values ​​of the characteristic values ​​of the physical instance; retraining a machine learning function that has been trained to receive the simulation values ​​as inputs and output the parameter values ​​corresponding to the simulation values ​​by inputting the measured characteristic values ​​into the machine learning function; receiving one or more predictions from the retrained machine learning function based at least in part on the input measured property values; improving the simulation model based at least in part on the received predictions to generate an improved simulation model. a test and measurement instrument having one or more processors configured to A test and measurement system comprising:

4. the one or more processors 4. The test and measurement system of claim 3, further configured to further improve the simulation model based at least in part on the one or more predictions received for the improved simulation model.

Images