Test measurement system

Abstract

To digitize a signal from a DUT with cost efficiency.SOLUTION: A test measurement system has: a manufacturing (MFG) test device digitizer 10 in a MFG test device for translating signals from one or more DUTs 16 to digital waveforms; a machine learning network 40; and one or more processors. The processor is configured to: perform one or more measurements of the digital waveforms; transmit one or more measurement values of the digital waveforms to the machine learning network 40 as an input; use the machine learning network 40 to translate one or more measurement values to measurement values to be generated by a reference test device, and generate one or more translation values; and determine whether the one or more DUTs 16 meet a performance requirement on the basis of the one or more translation measurement values. The reference test device is more accurate than the MFG test device.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to test and measurement systems, and more particularly to machine learning based measurement translators for test and measurement systems. [Background technology]

[0002] Customers who use test and measurement equipment, such as oscilloscopes, in their manufacturing (MFG) processes are typically looking for ways to make their manufacturing processes cheaper, faster, or both. Some customers who use test equipment in their manufacturing lines may have 100 or more test stations that test, tune, and measure the devices they are making to determine if they meet standards. Higher quality, and therefore more expensive, test and measurement equipment provides more accurate measurements, but at a higher cost. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Publication No. 2022 / 0373598 [Patent Document 2] JP 2022-179459 A [Non-patent literature]

[0004] [Non-Patent Document 1] "Tektronix oscilloscope" introduction site, Tektronix, [online], [searched June 26, 2024], Internet<https: / / www.tek.com / ja / products / oscilloscopes> [Non-Patent Document 2] "DPOJET Data Sheet", Tektronix, [online], [Retrieved June 26, 2024], Internet<https: / / www.tek.com / ja / datasheet / jitter-noise-and-eye-diagram-analysis-solution> [Non-Patent Document 3] "Serial Data Link Analysis Visualizer, DSA8300 JNB02, V2.2.0", Tektronix, [online], [Retrieved June 26, 2024], Internet<https: / / www.tek.com / ja / support / software / application / dsa8300-1-software-4> Summary of the Invention [Problem to be solved by the invention]

[0005] Typically, low-cost measurement equipment will perform poorly in certain measurements, resulting in errors in those measurements. This can cause a device under test (DUT) to pass tests it did not actually pass, which can cause errors when the manufacturer's customers use the device in operation. Errors in measurements can also cause devices to be scrapped when they would have passed the correct measurements.

[0006] For example, a customer may want to make as many high bandwidth measurements as possible using an Equivalent Time (ET) oscilloscope. Equivalent Time oscilloscopes ("scopes") are much cheaper than Real Time (RT) oscilloscopes. A manufacturer will only use a RT oscilloscope in situations where an ET oscilloscope cannot perform the measurement. This can cause many of the problems mentioned above for both the manufacturer and the end customer. [Means for solving the problem]

[0007] Embodiments of the present application allow customers to use lower cost digitizing equipment to make measurements on the production line. However, lower cost typically means lower performance for certain measurements, resulting in errors in those measurements. Thus, embodiments of the present application generally allow a user to make measurements on lower performance production test equipment, such as an ET scope as described above, and convert the measurements using neural networks or other machine learning modules to what would be obtained on a higher performance (and more expensive) reference test equipment, such as an RT scope as described above. For example, in current practice, wide bandwidth oscilloscopes apply a bandwidth enhancement filter (BWE) to expand the bandwidth of the oscilloscope to improve the accuracy of rise time measurements of fast edge waveforms. Embodiments of the present application make these and other enhancements and can convert measurements from production test equipment to match the measurements of a higher performance reference equipment using machine learning.

[0008] The present embodiments differ from the current state of the art in that they incorporate measurements and statistics from production test equipment using neural networks. Some embodiments may optionally also incorporate parameters of the production test equipment that contribute to errors in the measurements and correlate them to what the ideal measurements should be.

[0009] The test equipment may include a digitizer, which typically receives a signal as an input, and a measurement software (SW) application or package generates measurements and statistics on the digitized signal waveform. The test equipment may include, but is not limited to, a real-time oscilloscope, an external tester, a margin tester, a BER test system, a real-time spectrum analyzer, a voltmeter, an ammeter, etc. Thus, the measurement error may be partly due to the difference in the test equipment hardware and partly due to the difference in the measurement SW package algorithm. The embodiments of the present disclosure allow for both domains to be considered in the predictions regarding the conversion from the MFG test equipment / measurement set to the reference test equipment / measurement set. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 shows an embodiment of a learning configuration of a conversion system. [Diagram 2] FIG. 2 illustrates an embodiment of a runtime configuration of a transformation system. [Diagram 3] FIG. 3 shows an embodiment of a learning configuration of a conversion system to DPOJET using JNB. [Figure 4] FIG. 4 shows an example of the runtime configuration of a conversion system to DPOJET using JNB. [Diagram 5] FIG. 5 illustrates another embodiment of the runtime configuration of the DPOJET conversion system using JNB. [Figure 6] FIG. 6 shows an embodiment of a training scheme for the conversion system using Principal Component Analysis (PCA). [Figure 7] FIG. 7 illustrates an embodiment of the run-time configuration of a transformation system using Principal Component Analysis (PCA). [Figure 8] FIG. 8 shows an embodiment of a learning arrangement for the transformation system using measurements and statistics. [Figure 9] FIG. 9 illustrates an embodiment of a run-time configuration of the transformation system using measurements and statistics. [Figure 10] FIG. 10 shows an embodiment of a learning architecture for the transformation system using PCA, measurements and statistics. [Figure 11] FIG. 11 illustrates an embodiment of the run-time implementation of the transformation system using PCA, measurements and statistics. [Figure 12] FIG. 12 illustrates an embodiment of a learning configuration of a measurement package conversion system. [Figure 13] FIG. 13 illustrates an embodiment of a run-time configuration of the measurement package conversion system. [Figure 14] Figure 14 shows an embodiment of a learning configuration of a conversion system for a test device integrating a measurement software package. [Figure 15] Figure 15 shows an embodiment of the run-time configuration of a conversion system for a test device integrating a measurement software package. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The figures of this application use the same reference numbers for different components of a system if the system uses those components in the same environment. Additionally, some figures may reference specific components that are not referenced in subsequent figures. Unless otherwise specified, it is assumed that the non-referenced components are present in similar figures.

[0012] Most, if not all, test and measurement instruments (referred to herein as test instruments) use digitizers to convert analog signals received from a device under test into digital waveforms. Digitizers typically have specific characteristics and operating parameters. Some figures use digitizers to describe embodiments in more general terms, while specific examples may use specific equipment and measurements.

[0013] This description focuses on the test equipment apart from the measurement package used. Different measurement packages may use different measurements for the same or similar measurements. For example, Tektronix's DPOJET measurement SW package can measure jitter, noise, timing and perform eye diagram analysis on a RT oscilloscope (see Non-Patent Document 2). Other packages such as Tektronix's JNB measurement SW package can measure jitter, noise and BER (Bit Error Rate) on an ET oscilloscope (see Non-Patent Document 3). Machine learning systems such as neural networks convert measurements based on test equipment parameters as well as differences between the measurement software used on lower cost, less accurate production test equipment and the software used on higher cost, more accurate reference test equipment. In the figures, the measurement packages (referenced 1 and 2) distinguish the MFG package from the reference package and do not indicate that all embodiments using the package identified by 1 are the same package across all embodiments. Various configurations and various elements may use various packages and are identified by reference numbers. In this application, measurements resulting from conversion to a higher precision device are referred to as converted measurements.

[0014] 1-2 show an embodiment of a manufacturing system that employs a digitizer in an inexpensive MFG (manufacturing) machine 11 to convert one or more measurements produced by the MFG digitizer 10 to those produced by a reference digitizer 20 used in a more sophisticated, more expensive reference test machine 21. FIG. 1 shows a configuration in which a neural network 40 is trained to relate measurements from the MFG digitizer 10 to measurements from a reference test machine digitizer 20. Once trained, the neural network 40 is referred to as a trained or pre-trained neural network and belongs to a user.

[0015] Figure 1 shows three types of waveform input data. The emulated BERT (Bit Error Rate Tester), AWG (Arbitrary Waveform Generator) input 12 represents the signal generator hardware that can generate an emulation of the signal you want to measure. The resulting waveform is used in the neural network training process. The switch 13 indicates that the analog waveform 11 from the BERT is input to both the MFG digitizer 10 and the reference digitizer 20. The neural network can be trained completely using only the BERT or AWG waveforms, and then the training results can be used to measure the user's DUT.

[0016] The simulated input 14 represents an option, enabled by switch 18, that allows offline simulation software to create a waveform on line 15 to input to the measurement block for training. The simulation may include a model of an oscilloscope or it may include a model of the user's DUT 16. Switch 18 indicates that this is an optional input for training the neural network.

[0017] User's DUT 16 represents, in practical terms, where the user's automated test software controls the collection of waveforms from a number of user's DUTs. The resulting waveforms are input to the MFG digitizer and reference digitizer and measured to create the training data required for the neural network. Switch 22 indicates that these waveforms may be optional for the training process. For example, it may be feasible that only the BERT input waveforms need to be trained and then used to measure the DUTs once training is complete.

[0018] After the second measurement package 26 makes measurements on the waveform data, the statistics and histogram block 32 calculates the mean, minimum, maximum, standard deviation and histogram statistics from all of the training waveforms for all of the desired measurements, which are later used at run-time to compare the neural net output measurements and determine a pass or fail result.

[0019] During training, the waveforms created by the MFG digitizer may be converted into a series of short pattern waveforms in a short pattern separator 28 as described in U.S. Patent Publication No. 2022 / 0373598, entitled "Machine Learning of Short Pattern Waveform Databases for Measurements." A tensor builder 30 takes the waveform (i.e., the short pattern representation of the waveform), parameters of the MFG digitizer, measurements from the first measurement package 24, and the like, and builds a tensor. The digitizer parameters include bandwidth, gain, noise waveform, effective number of bits, and the like. The measurements include total jitter, duty cycle jitter, data-dependent jitter, periodic jitter, non-periodic jitter, random jitter, random noise, periodic noise, data-dependent noise, total noise, and non-periodic noise. These are merely some examples and are not intended to limit the embodiments to any particular digitizer parameters or measurements. The data may be prepared to fit into one-dimensional arrays, tensors or vectors, or into 2D or 3D arrays, such as monochrome or RGB images. There are other possibilities that may be incorporated. For example, a multi-branch neural network may be configured to simultaneously receive 1D tensors and 2D or 3D image tensors.

[0020] The tensor builder 30 may create a tensor vector or image. If the tensor builder builds a vector, the vector is normalized at 34 and provided to the neural network in normalized form. To relate the resulting vector to a similar vector from the reference test fixture 20, the reference digitizer parameters and measurements are provided to a reference tensor builder 38, which constructs a reference tensor vector, which is normalized at 36. The normalization process converts the dynamic range of the input data to a range of -1 to 1 before inputting the input data to the neural network. The denormalization process returns the normalized output range to the expected numerical range of the output measurement data. The normalization process may be omitted if it is already present inside the neural network structure. The neural network 40 then relates the measurements represented by the MFG tensors from the tensor builder 30 to the reference tensors in order to "teach" the neural network to convert between different measurements. The user selects the type of tensor before training and before using it on the production line.

[0021] FIG. 2 illustrates the run-time configuration of the trained embodiment of FIG. 1. In the run-time embodiment, the signal generator data waveform source 12 and simulated waveform source 14 are not used. The user DUT 18 generates the waveform on which measurements are made. Generally, measurements are made by the MFG digitizer 10 and the tensor builder 30 builds the MFG tensors. The neural network then outputs the results, which are de-normalized at 42. The reference tester 10 is not involved until the validation process at 44. The validation process 44 uses the statistics from 32 and the results from the neural network 40 that have been de-normalized at 42. The neural network results are used to determine if the DUT has passed or failed. This generally involves determining at 46 if the value of the measurement is within manufacturing tolerances for that measurement. If the measurement is within tolerance, the predicted value is maintained at 48 and the DUT passes. If the measurement is outside the range, switches 23 and 27 are controlled to be on and then the reference digitizer 20 makes a measurement to make the final pass / fail decision.

[0022] Figures 3 and 4 provide a more generalized example of Figures 1 and 2. In these processes, the MFG digitizer resides in the MFG test equipment 11, more specifically, in an ET oscilloscope. Similarly, the reference digitizer resides in the reference test equipment 21, in this case, an RT oscilloscope. The MFG oscilloscope uses a JNB (jitter, noise, BER) measurement package 50, and the reference oscilloscope uses a Tektronix DPOJET measurement package 52. Figure 3 shows a learning configuration similar to Figure 1, with these modifications. Figure 4 shows a run-time configuration for the configuration of Figure 3. Figure 4 may incorporate a validation process to ensure that the first test of this configuration works well. In Figure 5, the run-time configuration is expanded to include multiple instances of the MFG oscilloscope 11. Some manufacturing (MFG) lines may use 100 MFG oscilloscopes.

[0023] In this example, the customer has about 100 ET sampling oscilloscopes on their production line to make measurements. The customer wants to use these because they are much cheaper than RT (real-time) sampling oscilloscopes while still getting the wide bandwidth they need. The customer in this application case would have one RT oscilloscope.

[0024] If only one measurement, such as non-periodic jitter, were input to the neural network for training, the neural network would not have enough information to predict the desired reference value and would not be able to perform the mapping. However, this embodiment relies on the presence of a wide variety of measurements. This forms a large, multi-dimensional input measurement space, which allows one to see the various characteristics of the measured waveforms. The neural network can then see the characterization of this measurement space of the measurement set and relate it to the reference measurement space, mapping or transforming all the measurements to what they should be.

[0025] 6 and 7 show the example embodiment of FIGS. 3 and 4, but with principal component analysis (PCA) added to the system. Adding PCA can be beneficial for training the neural network. PCA receives multiple measurement types and multiple examples of measurements along the length of the input waveform. PCA outputs a data set representing the principal components of these measurements, providing useful information about how the measurements are affected by the parameters of the test equipment. While the internal neural network can extract the component features of the input data, PCA can provide an additional benefit of helping to extract some of the non-linear correlations between all the input measurements. When the neural network outputs a PCA vector, the PCA can be inverse transformed to obtain the desired measurement vector. It will be appreciated that PCA is added in the specific JNB / DPOJET example embodiment, and that the general embodiment of FIGS. 1 and 2 may also provide PCA.

[0026] In Figure 6, the output of the JNB measurements from 50 undergoes PCA at 54 and is used to construct the MFG tensor at 30. Similarly, the output of the DPOJET measurements from 52 undergoes PCA at 56 and is normalized at 36. A neural network is then trained on the tensor containing the PCA. Figure 7 shows the run-time configuration, where inverse PCA is performed at 58 prior to the validation process at 44.

[0027] Returning now to the more general embodiment as shown in Figures 1 and 2, this embodiment provides the user with the ability to control the digitizer. Certain parameters of the test equipment can affect the accuracy of the measurements. These parameters can change when the digitizer settings are changed, such as when the vertical gain or offset, sample rate or bandwidth is changed. For example, the noise of a digitizer can change over a wide range with different vertical gain settings.

[0028] During the training process, these digitizer parameters are input into the neural network along with the measurements, allowing the measurements to be associated with different digitizer settings and even different digitizer models if the training process includes more than one form of digitizer model. The metadata includes an array of input measurements and an array of digitizer parameters associated with each set of measurements. The label data output from the neural network after training is the set of desired measurements after denormalization and, if PCA is used, inverse PCA is performed.

[0029] FIG. 8 shows an embodiment of a measurement conversion system configuration including digitizer parameters for an MFG test instrument. The MFG test instrument may be, for example, an oscilloscope, a real-time spectrum analyzer, or a digital multimeter, among many other examples. User automation software 70 controls the reference digitizer 20 and an array of MFG digitizers (such as 10), which typically collect and measure waveforms from a number of user DUTs 72, for training purposes only in this example. In general, the user automation software controls the DUTs, test equipment, measurement requests, etc. The digitizers provide their parameters and settings to the user automation software 70, which performs measurements and generates measurement statistics at 74. Measurements 94 from the reference digitizers are also subjected to statistical and histogram analysis at 32, and the results are combined at 76. The combination at 76 improves the dynamic range of the measurement results, but it only inputs the average difference of the measurements and the measurements into the tensor builder block. The tensor builder 30 builds a tensor using the measurement statistics from 74, the measurement statistics minus the reference statistics, and the digitizer parameters in the form of metadata from the user automation software, normalized at 34 and 38. The neural network then receives this tensor and the normalized reference measurements as training data. The measurements from the reference digitizer are labeled with metadata.

[0030] Figure 9 shows the run-time configuration. To simplify the diagram, the reference digitizer is not shown in Figure 9. However, as can be seen at 78, if the validation process finds that the measurements are out of range, the DUT may be failed or retested with the reference digitizer. Figure 10 shows the training configuration including the PCA 54, and Figure 11 shows the run-time configuration including the PCA.

[0031] The discussion so far has focused on MFG digitizers and reference digitizers, which are not tied to any particular software measurement package except for the specific examples associated with JNB and DPOJET. We now turn to an embodiment of translating measurements between software packages, as shown in Figures 12 and 13.

[0032] The system consists of a first block labeled MFG Measurement Package 100 and a second block labeled Reference Measurement Package 102. These packages may be the same or different than the other measurement packages mentioned above. Examples of conversions include, but are not limited to, Tektronix JNB to DPOJET conversions, Keysight to Tektronix EZJIT to DPOJET and vice versa conversions, LeCroy or Anritsu to other companies conversions, etc.

[0033] As mentioned above, the system uses a large multi-dimensional input measurement space, which allows it to look at various characteristics of the measured waveforms. The neural network looks at the characterization of this measurement space for a set of measurements, and then relates it to a reference measurement space, allowing it to map all measurements to what they should be.

[0034] FIG. 12 shows a training configuration for a measurement conversion device between different measurement packages. As mentioned above, each measurement package can characterize the same or similar measurements in different ways. The characterization may differ between the two packages. During training, the MFG measurement package 100 builds a tensor, either an image or a vector, and the reference measurement package 102 does the same. Statistics and histograms are generated at 32 for the reference measurements before building the tensor. A neural network is then trained to form an association between the two. FIG. 13 shows an embodiment of a run-time configuration. As with the digitizer example, the converted measurements are verified at 44, and if the DUT fails, a reference device using the reference package may make measurements to confirm pass or fail.

[0035] Having described the translation between the MFG digitizer and the reference digitizer independently of the measurement package used, and the translation between measurement packages independently of the digitizer / test instrument used, we now describe embodiments in which a test instrument is paired with a particular measurement package, in which the neural network translates between measurements based on test instrument parameters and based on measurement package / characterization differences between the MFG test instrument and the reference test instrument.

[0036] In this context, an embodiment trains a neural network to relate measurements and characteristic parameters of the MFG tester to ideal measurements from a reference tester. This embodiment can be used for many classes of testers. This embodiment uses a neural network to capture measurements and statistics from the MFG tester. It may also capture parameters of that device that cause errors in the measurements and relate them to the ideal measurements. The tester may include a digitizer, typically taking a waveform as input, and a measurement SW to generate measurements and statistics of the waveform. Testers include, but are not limited to, RT oscilloscopes, ET oscilloscopes, margin testers, BER test systems, real-time spectrum analyzers, voltmeters, ammeters, etc.

[0037] Measurement errors may be partly due to differences in tester hardware and partly due to differences in SW algorithms of the measurement package. This embodiment allows both areas to be considered in predicting the conversion from the MFG tester / measurement set to the reference tester / measurement set.

[0038] 14 and 15 show block diagrams of a combined oscilloscope and measurement conversion system. This embodiment converts measurements from an MFG test instrument running measurement package 1 to match a reference test instrument running measurement package 2. The test instrument could be an oscilloscope, spectrum analyzer, voltmeter, ammeter or any other type of test instrument where conversion functionality would be useful.

[0039] As shown in FIG. 14, the system consists of a first block named MFG test equipment 110 with digitizer 10 and first measurement package 112, and a second block named reference test equipment 120 with reference digitizer 20 and second measurement package 122. Again, the measurement packages 1 and 2 in these embodiments may differ from those previously described. The oscilloscope and second block are called reference RT oscilloscope. Tensor builder 30 creates tensors using the MFG test equipment digitizer and possibly test equipment information, waveforms (or short waveforms) and measurements, and tensor builder 38 does the same for the reference digitizer, test equipment (if used) and measurements. This trains the system as in the previous embodiment. FIG. 15 shows the run-time configuration.

[0040] These embodiments include training the neural network 40 to accept a set of jitter and noise values ​​for the input waveform. These measurements are obtained from the MFG test equipment 110 and the first measurement package 112, as well as from the reference test equipment 120 and the second measurement package 122. In general, however, the test equipment may include any test equipment having a digitizer. The measurement package may include any software package capable of performing the necessary measurements on digitized input data. This data may be a waveform, a voltage from a voltmeter, a current from an ammeter, etc. The embodiments translate the measurements from one test equipment and SW to those obtained by another equipment and software. The translation takes into account the effect of the particular equipment on the measurements.

[0041] The present embodiment provides a new method that allows customers to use large quantities of a desired test instrument for an MFG test instrument and convert the measurements to those obtained with a higher performance reference test instrument. This allows customers to save a lot of money by having 100 or more relatively inexpensive MFG test instruments, but only one or a few expensive reference test instruments. The present embodiment provides conversion between MFG test instruments and reference test instruments, whether using a measurement package integrated into the test instrument or a separate measurement package. The present embodiment provides conversion between measurement packages, even when the test instrument and digitizer are not an issue.

[0042] Aspects of the disclosed technology can operate on specially created hardware, firmware, digital signal processors, or specially programmed general-purpose computers, including processors that operate according to programmed instructions. The term "controller" or "processor" in this application contemplates microprocessors, microcomputers, ASICs, and dedicated hardware controllers, among others. Aspects of the disclosed technology can be implemented in computer-available data and computer-executable instructions, such as one or more program modules, executed by one or more computers (including a monitoring module) or other devices. Generally, program modules include routines, programs, objects, components, data structures, and the like, which, when executed by a processor in a computer or other device, perform particular tasks or implement particular abstract data formats. The computer-executable instructions may be stored in computer-readable storage media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, and the like. As will be appreciated by those skilled in the art, the functionality of the program modules may be combined or distributed as desired in various embodiments. Furthermore, such functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc. Certain data structures may be used to more effectively implement one or more aspects of the disclosed techniques, and such data structures are considered to be within the scope of the computer-executable instructions and computer-usable data described herein.

[0043] The disclosed aspects may, in some cases, be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried by or stored on one or more computer-readable media that may be read and executed by one or more processors. Such instructions may be referred to as computer program products. A computer-readable medium as described herein means any medium that can be accessed by a computing device. By way of example, and not limitation, computer-readable media may include computer storage media and communication media.

[0044] Computer storage media means any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital video disc (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any other volatile or non-volatile removable or non-removable media implemented in any technology. Computer storage media excludes signals themselves and transitory forms of signal transmission.

[0045] A communication medium refers to any medium capable of communicating computer readable information. By way of example, and not limitation, communication media may include coaxial cables, fiber optic cables, air, or any other medium suitable for communicating electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals. Working Example

[0046] In the following, examples useful for understanding the technology disclosed in the present application are presented. The embodiment of the technology may include one or more of the examples described below and any combination thereof.

[0047] Example 1 is a test and measurement system, a first test and measurement instrument having an input capable of receiving signals from one or more devices under test (DUTs) and one or more digitizers for converting the signals from the one or more DUTs into digital waveforms; Machine learning networks and One or more processors Equipped with said one or more processors: taking one or more measurements of said digital waveform; sending one or more measurements of said digital waveform as an input to said machine learning network; using the machine learning network to transform the one or more measurements into measurements generated by a reference test device that is more accurate than the first test and measurement device to generate one or more transformed measurements; determining whether said DUT meets performance requirements based on said one or more converted measurements; The one or more processors are configured to execute a program (code) that causes the one or more processors to perform the above steps.

[0048] Example 2 is a test and measurement system of Example 1, wherein the program that causes the one or more processors to send the one or more measurements to the machine learning network includes a program that causes the one or more processors to send one or more digitizer parameters along with the one or more measurements to a tensor builder prior to the machine learning network.

[0049] Example 3 is a test and measurement system of either Example 1 or 2, wherein the program that causes the one or more processors to send the one or more measurements to the machine learning network includes a program that causes the one or more processors to send one or more measurement statistics together with the one or more measurements to a tensor builder prior to the machine learning network.

[0050] Example 4 is a test and measurement system of any of Examples 1 to 3, wherein the program that causes the one or more processors to send the one or more measurement statistics includes a program that causes the one or more processors to send a set of principal components along with the one or more measurements to a tensor builder prior to the machine learning network.

[0051] Example 5 is the test and measurement system of Example 2, wherein the one or more processors are further configured to execute a program that causes a process to construct a tensor from the one or more digitizer parameters and the one or more measurements to be sent to the machine learning network.

[0052] Example 6 is a test and measurement system of any of Examples 1 to 5, wherein the program that causes the one or more processors to send the one or more measurements to the machine learning network includes a program that causes the one or more processors to split the digital waveform into one or more short pattern waveforms and send the short pattern waveforms to a tensor builder prior to the machine learning network.

[0053] Example 7 is a test and measurement system of any of Examples 1 to 6, wherein the program that causes the one or more processors to perform a process of determining whether the DUT meets performance requirements based on the one or more converted measurements includes a program that causes the one or more processors to perform a process of determining that the DUT does not meet performance requirements, and the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of digitizing a signal from the DUT and instructing the reference test instrument to perform the same one or more measurements on the digital waveform, and a process of determining whether the DUT meets performance requirements based on the one or more measurements from the reference test instrument.

[0054] Example 8 is a test and measurement system of any of Examples 1 to 7, wherein the program that causes the one or more processors to perform a process of determining whether the DUT meets performance requirements based on the one or more converted measurement values ​​from the machine learning network includes a program that causes the one or more processors to perform a process of determining that the DUT does not meet the performance requirements, and the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of reporting the DUT as a failure.

[0055] Example 9 is a test and measurement system of any of Examples 1 to 8, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process for training the machine learning network.

[0056] Example 10 is a test and measurement system of any of Examples 1 to 9, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of training the machine learning network, and the program that causes the one or more processors to perform a process of training the machine learning network includes a program that causes the one or more processors to perform a process of receiving one or more measurements of the digital waveform from the first test and measurement device, a process of receiving one or more measurements of the digital waveform from a reference test device, and a process of associating the one or more measurements from the first test and measurement device with corresponding measurements among the one or more measurements from the reference test device.

[0057] Example 11 is a test and measurement system of any of Examples 1 to 10, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of calculating at least one statistical value of the measurements, and a process of performing a principal component analysis of the measurements to associate at least one of the statistical values ​​and the principal components with the measurements.

[0058] Example 12 is the test and measurement system of any of Examples 1 to 11, the program causing the one or more processors to perform one or more measurements of the digital waveform includes a program causing the one or more processors to perform the one or more measurements using a first characterization of the one or more measurements; The program for causing the one or more processors to send the one or more measurements of the digital waveform as inputs to the machine learning network includes a program for causing the one or more processors to send the one or more measurements of the digital waveform and the first characteristic estimate as inputs to the machine learning network; The program that causes the one or more processors to use the machine learning network to convert the one or more measurements into measurements generated by the reference test equipment causes the one or more processors to map the one or more measurements made using the first characterization for the one or more measurements to measurements generated by the reference test equipment using a second characterization.

[0059] Example 13 is a test measurement system, comprising: a first test and measurement instrument having an input capable of receiving signals from one or more devices under test (DUTs) and one or more digitizers for converting the signals from the one or more DUTs into digital waveforms; Machine learning networks and One or more processors Equipped with said one or more processors: making one or more measurements of the digital waveform using the first test and measurement instrument using a first characterization of the one or more measurements of the digital waveform; sending the one or more measurements of the digital waveform and the first characterization as inputs to the machine learning network; using the machine learning network to transform the one or more measurements into measurements generated by a reference test instrument that is more accurate than the first test and measurement instrument and that uses a second characterization of the one or more measurements to generate one or more transformed measurements; using the one or more transformed measurements from the machine learning network to determine whether the DUT meets performance requirements; The one or more processors are configured to execute a program that causes the one or more processors to perform the steps of:

[0060] Example 14 is a test and measurement system of Example 13, wherein the program that causes the one or more processors to process the one or more measurements to the machine learning network includes a program that causes the one or more processors to process one or more digitizer parameters along with the one or more measurements to a tensor builder prior to the machine learning network.

[0061] Example 15 is a test and measurement system of any of Examples 13 and 14, wherein the program that causes the one or more processors to send the one or more measurements and the first characteristic estimate to the machine learning network includes a program that causes the one or more processors to send one or more measurement statistics together with the one or more measurements and the first characteristic estimate to a tensor builder prior to the machine learning network.

[0062] Example 16 is a test measurement system of Example 15, comprising a program that causes the one or more processors to process the one or more measurement statistics together with the one or more measurements to a tensor builder prior to the machine learning network, and a program that causes the one or more processors to process the set of principal components together with the one or more measurements to a tensor builder prior to the machine learning network.

[0063] Example 17 is a test and measurement system of any of Examples 13 to 16, wherein the one or more processors are further configured to execute a program to cause a process to construct a tensor from the one or more digitizer parameters and the one or more measurements to be sent to the machine learning network.

[0064] Example 18 is a test and measurement system of any of Examples 13 to 17, wherein the program that causes the one or more processors to send the one or more measurements to the machine learning network includes a program that causes the one or more processors to split the digital waveform into one or more short pattern waveforms and send the short pattern waveforms to a tensor builder prior to the machine learning network.

[0065] Example 19 is the test and measurement system of any of Examples 13 to 18, wherein the program for causing the one or more processors to perform a process of determining using one or more transformed measurements from the machine learning network includes a program for causing the one or more processors to perform a process of determining that the DUT does not meet a performance requirement using the one or more transformed measurements, and the one or more processors: digitizing a signal from the DUT and instructing the reference test instrument to perform the same one or more measurements on the digital waveform; determining whether the DUT meets performance requirements based on one or more measurements from the reference test fixture; The one or more processors may be further configured to execute a program that causes the one or more processors to:

[0066] Example 20 is the test and measurement system of any of Examples 13 to 19, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of training the machine learning network, and the program that causes the one or more processors to perform a process of training the machine learning network includes: receiving the one or more measurements of the digital waveform from the first test and measurement instrument; receiving one or more measurements of the digital waveform from a reference test instrument using a second characterization; correlating the one or more measurements from the first test and measurement device and the first characterization of the one or more measurements with corresponding ones of the one or more measurements from the reference test device; The present invention includes a program for causing one or more processors to perform the above.

[0067] In addition, the present description refers to specific features. It should be understood that the disclosure herein includes all possible combinations of those specific features. When a specific feature is disclosed in connection with a particular aspect or embodiment, that feature can also be used in connection with other aspects and embodiments, to the extent possible.

[0068] Furthermore, when the application refers to a method having two or more defined steps or processes, those defined steps or processes may be performed in any order or simultaneously, unless the context precludes such a possibility.

[0069] All features disclosed in the specification, claims, abstract and drawings, and all steps in any disclosed method or process, may be combined in any combination, except where at least some of such features or steps are mutually exclusive combinations. Each feature disclosed in the specification, abstract, claims and drawings may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0070] For purposes of illustration, specific embodiments of the invention have been illustrated and described, but it will be understood that various modifications can be made without departing from the spirit and scope of the invention. Accordingly, the invention should not be limited, except as by the appended claims. [Explanation of symbols]

[0071] 10 Manufacturing (MFG) Test Equipment Digitizers 11 Manufacturing (MFG) Test Equipment 12 Emulated BERT, AWG input 14 Simulated Inputs 16 User's DUT 20 Reference Test Equipment Digitizer 21 Reference Test Equipment 23 Switch 24 First Measurement Package 26 Second Measurement Package 27 Switch 28 Short Pattern Separator 30 Tensor Builder 32 Statistics / Histogram Analysis Block 40 Neural Networks 44 Verification Process Blocks 50 JNB Measurement Package 52 DPOJET Measurement Package 54 Principal Component Analysis (PCA) Block 56 Principal Component Analysis (PCA) Block 70 User Automation Software 72 Multiple user DUTs 76 Joint 100 MFG Measurement Package 102 Reference Measurement Package 110 Manufacturing (MFG) Test Equipment 112 First Measurement Package 120 Reference Test Equipment Digitizer 122 Second Measurement Package

Claims

1. a first test and measurement instrument having an input capable of receiving signals from one or more devices under test (DUTs) and one or more digitizers for converting the signals from the one or more DUTs into digital waveforms; Machine learning networks and One or more processors Equipped with the one or more processors taking one or more measurements of said digital waveform; sending one or more measurements of said digital waveform as input to said machine learning network; using the machine learning network to transform the one or more measurements into measurements generated by a reference test device that is more accurate than the first test measurement device to generate one or more transformed measurements; determining whether the DUT meets performance requirements based on the one or more converted measurements; a test and measurement system configured to execute a program that causes the one or more processors to perform the steps of:

2. 2. The test and measurement system of claim 1, wherein the program causing the one or more processors to send the one or more measurements to the machine learning network includes a program causing the one or more processors to send one or more digitizer parameters, one or more measurement statistics, or a set of principal components along with the one or more measurements to a tensor builder prior to the machine learning network.

3. 2. The test and measurement system of claim 1, wherein the program causing the one or more processors to send the one or more measurements to the machine learning network includes a program causing the one or more processors to split the digital waveform into one or more short pattern waveforms and send the short pattern waveforms to a tensor builder prior to the machine learning network.

4. The program for causing the one or more processors to perform a process of determining whether the DUT satisfies a performance requirement based on the one or more conversion measurements includes a program for causing the one or more processors to perform a process of determining that the DUT does not satisfy a performance requirement, and the one or more processors digitizing a signal from the DUT and instructing the reference test instrument to perform the same one or more measurements on the digital waveform; determining whether the DUT meets performance requirements based on one or more measurements from the reference test apparatus; 2. The test and measurement system of claim 1, further configured to execute a program that causes the one or more processors to:

5. The one or more processors are further configured to execute a program that causes the one or more processors to perform a process of training the machine learning network, the program that causes the one or more processors to perform a process of training the machine learning network is receiving the one or more measurements of the digital waveform from the first test and measurement instrument; receiving one or more measurements of a digital waveform from a reference test instrument; correlating the one or more measurements from the first test measurement device with corresponding ones of the one or more measurements from the reference test device; 2. The test and measurement system of claim 1, further comprising a program for causing said one or more processors to:

6. the program causing the one or more processors to perform one or more measurements of the digital waveform includes a program causing the one or more processors to perform the one or more measurements using a first characterization of the one or more measurements; The program for causing the one or more processors to send the one or more measurements of the digital waveform as inputs to the machine learning network includes a program for causing the one or more processors to send the one or more measurements of the digital waveform and the first characteristic estimate as inputs to the machine learning network; 2. The test and measurement system of claim 1, wherein the program that causes the one or more processors to use the machine learning network to convert the one or more measurements into measurements generated by the reference test equipment causes the one or more processors to map the one or more measurements made using the first characterization for the one or more measurements to measurements generated by a reference test equipment using a second characterization.

7. a first test and measurement instrument having an input capable of receiving signals from one or more devices under test (DUTs) and one or more digitizers for converting the signals from the one or more DUTs into digital waveforms; Machine learning networks and One or more processors Equipped with the one or more processors making one or more measurements of the digital waveform using the first test and measurement instrument using a first characterization of the one or more measurements of the digital waveform; sending the one or more measurements of the digital waveform and the first characterization as inputs to the machine learning network; using the machine learning network to transform the one or more measurements into measurements generated by a reference test device that is more accurate than the first test and measurement device and that uses a second characterization of the one or more measurements to generate one or more transformed measurements; using the one or more transformed measurements from the machine learning network to determine whether the DUT meets performance requirements; a test and measurement system configured to execute a program that causes the one or more processors to perform the steps of:

8. 8. The test and measurement system of claim 7, wherein the program causing the one or more processors to send the one or more measurements and the first characteristic estimate to the machine learning network comprises a program causing the one or more processors to send a set of one or more measurement statistics or principal components, together with the one or more measurements and the first characteristic estimate, to a tensor builder prior to the machine learning network.

9. The program for causing the one or more processors to perform a process of determining using one or more transformed measurements from the machine learning network includes a program for causing the one or more processors to perform a process of determining using the one or more transformed measurements that the DUT does not meet a performance requirement, and the one or more processors digitizing a signal from the DUT and instructing the reference test instrument to perform the same one or more measurements on the digital waveform; determining whether the DUT meets performance requirements based on one or more measurements from the reference test apparatus; 8. The test and measurement system of claim 7, further configured to execute a program that causes the one or more processors to:

10. The one or more processors are further configured to execute a program that causes the one or more processors to perform a process of training the machine learning network, the program that causes the one or more processors to perform a process of training the machine learning network is receiving the one or more measurements of the digital waveform from the first test and measurement instrument; receiving one or more measurements of the digital waveform from a reference test instrument using a second characterization; correlating the one or more measurements and the first characterization of the one or more measurements from the first test measurement device with corresponding ones of the one or more measurements and the one or more measurements from the reference test device; 8. The test and measurement system of claim 7, further comprising a program for causing said one or more processors to:

Images