Instrument and measurement translator using machine learning

Abstract

A test and measurement system includes a first test and measurement instrument having: an input to allow the test and measurement instrument to receive signals from one or more devices under test (DUTs); and one or more digitizers to convert a signal from the one or more DUTs into a digital waveform; a machine learning network; and one or more processors for: performing one or more measurements of the digital waveform; sending the one or more measurements of the digital waveform to the machine learning network as input; using the machine learning network to translate the one or more measurements into measurements made by a reference instrument to produce one or more translated measurements, the reference instrument being more accurate than the first test and measurement instrument; and determining whether the DUT meets performance requirements based on the one or more translated measurements.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS The present disclosure is a non-provisional application of U.S. Provisional Application No. 63 / 523,621, filed on June 27, 2023, entitled “INSTRUMENT AND MEASUREMENT TRANSLATOR USING MACHINE LEARNING,” and claims the benefit of the U.S. Provisional Application, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The present disclosure relates to test and measurement instruments, and more particularly to a machine learning based measurement translator for test and measurement instruments. Background Art

[0003] Customers who use test and measurement instruments (e.g., oscilloscopes) in their manufacturing ("MFG") processes generally desire ways to make their manufacturing processes less expensive, faster, or both. Some customers who use test equipment on a manufacturing line may have 100 or more test stations that make measurements to test, tune, and qualify the equipment they manufacture. Higher quality, and therefore more expensive, test and measurement instruments provide more accurate measurements, but increase the cost.

[0004] Lower cost instruments generally result in lower performance for certain measurements, resulting in errors in those measurements. This can then result in devices under test (DUTs) passing tests they actually failed, which can result in failures when the manufacturer's customers employ the devices in operations. Measurements with errors can also result in scrapping devices that otherwise passed with correct measurements.

[0005] For example, a customer may want to use an ET equivalent time oscilloscope to complete as many of their high bandwidth measurements as possible. Equivalent time oscilloscopes ("scopes") are much less expensive than RT real-time scopes. Manufacturers will only use RT scopes for situations where an ET scope is not capable of performing measurements. This can lead to many problems, such as those mentioned above for both the manufacturer and the end customer. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An embodiment of a translator system in a training configuration is shown.

[0007] Figure 2 An embodiment of a translator system in a runtime configuration is shown.

[0008] Figure 3 An embodiment of a translator system using a JNB to DPOJET in a training configuration is shown.

[0009] Figure 4-5 An embodiment of a translator system using JNB to DPOJET in a runtime configuration is shown.

[0010] Figure 6 An embodiment of a translator system using principal component analysis (PCA) in a training configuration is shown.

[0011] Figure 7 An embodiment of a translator system using principal component analysis (PCA) in runtime configuration is shown.

[0012] Figure 8 An embodiment of a translator system using measurements and statistics in a training configuration is shown.

[0013] Fig. 9 An embodiment of a translator system using measurements and statistics in runtime configuration is shown.

[0014] Fig.10 An embodiment of a translator system using PCA, measurements and statistics in a training configuration is shown.

[0015] Fig.11 An embodiment of a translator system using PCA, measurements and statistics in a runtime configuration is shown.

[0016] Fig.12 and 13 Embodiments of the measurement result packaging translation system in training configuration and runtime configuration are shown respectively.

[0017] Fig.14 An embodiment of a translator system for a test instrument with an integrated measurement software package in a training configuration is shown.

[0018] Fig.15 An embodiment of a translator system for a test instrument with integrated measurement software packaging in a runtime configuration is shown. DETAILED DESCRIPTION

[0019] Embodiments herein enable customers to use lower cost digitizing instruments to perform measurements on manufacturing lines. However, lower cost generally means lower performance for certain measurements, resulting in errors in those measurements. Therefore, embodiments herein generally allow users to obtain measurements on lower performance manufacturing test instruments (e.g., ET mirrors as mentioned above) and use neural networks or other machine learning modules to translate the measurements into what was originally obtained on higher performance reference test instruments (such as, the RT mirrors mentioned above). For example, currently practiced high bandwidth oscilloscopes will apply bandwidth enhancement filters (BWE) to extend the bandwidth of the mirror and make rise time measurements more accurate for fast edge waveforms. Embodiments herein can perform these and other enhancements and use machine learning to transform measurements from manufacturing test instruments to match measurements for higher performance reference instruments.

[0020] The embodiments herein differ from the prior art in that they use a neural network to take into account measurements and statistics from a manufacturing test instrument. As an option, some embodiments may also take into account parameters of the instrument that are responsible for errors in the measurements, and then correlate them with what an ideal measurement should be.

[0021] The test instrument may contain a digitizer and generally receives a signal as input and then a measurement software (SW) application or wrapper generates measurements and statistics on the digitized signal waveform. The test instrument may include, but is not limited to, RT oscilloscopes, ET oscilloscopes, margin testers, BER test systems, real-time spectrum analyzers, voltmeters, ammeters, etc. Therefore, measurement error may be due in part to differences in the test instrument hardware and also in part to differences in the measurement SW wrapper algorithms. Embodiments of the present disclosure allow both domains to be considered in the prediction for the translation from the MFG test instrument / measurement set to the reference test instrument / measurement set.

[0022] The various figures herein use the same reference numerals for different components of the system, regardless of whether the system uses those parts in the same environment. Further, some figures may mention specific components that are not mentioned in subsequent figures. Unless otherwise specified, unmentioned components are assumed to also exist in similar figures.

[0023] Most (if not all) test and measurement instruments, referred to herein as test instruments, employ digitizers to convert analog signals received from the device under test into digital waveforms. Digitizers generally have specific characteristics and operating parameters. Some figures will use digitizers to more generally discuss embodiments, and specific examples may employ specific instruments and measurements.

[0024] This section of the discussion will describe the test instrument independently of any measurement result package used. Different measurement result packages can use different measures for the same or similar measurement results. For example, the DPOJET measurement SW package from Tektronix can measure jitter, noise, timing, and perform eye movement analysis on the RT mirror. Other packages (such as the JNB measurement SW package from Tektronix) can measure jitter, noise, and BER (bit error rate) on the ET mirror. Machine learning systems (such as neural networks) translate the measurement results based on the parameters of the test instrument and based on the difference between the measurement software used on the lower cost less accurate manufacturing instrument and the software used on the higher cost more accurate reference instrument. In each figure, the measurement result package referred to as 1 and 2 distinguishes the MFG package from the reference package, and does not indicate that all embodiments use the package identified by 1 as the same package across all embodiments. Different configurations and different elements can use different packages and are identified by their reference numerals. The measurement results generated by the translation to the higher accuracy instrument are referred to as translated measurement results here.

[0025] Figure 1-2 An embodiment of a manufacturing system is shown that employs a digitizer in a less expensive manufacturing (MFG) instrument 10 and translates one or more measurements made by the MFG digitizer 10 to a reference digitizer used in a reference instrument 20. Figure 1 A configuration is shown in which neural network 40 undergoes training to correlate measurements from MFG digitizer 10 with measurements from reference digitizer 20. Once trained, neural network 40 may be referred to as a trained or pre-trained neural network that belongs to the user.

[0026] Figure 1 Three types of waveform input data are shown. Figure 1 There are three types of waveform input data shown in . Figure 1 Three types of waveform input data are shown. The analog BERT (Bit Error Rate Tester), AWG (Arbitrary Waveform Generator) input 12 represents the signal generator hardware that can generate an analog of the desired signal to be measured. The resulting waveform should be used in the training process of the neural network. The switch 16 shows that the analog waveform 11 from the BERT is input to both the MFG digitizer 10 and the reference digitizer 20. It is possible that the neural network can be fully trained using only the BERT or AWG waveforms, and then the trained results can be used to measure the DUT from the user.

[0027] The simulation input 14 enabled by switch 18 represents the option that offline simulation software can create waveforms to be input into the measurement block for training on-line 15. The simulation can include a model of the oscilloscope, and it can include a model of the user's DUT 16. Switch 18 shows that this is an optional input for training the neural network(s).

[0028] The actual user DUT 16 shows where the user's automated test software will control the collection of waveforms from many of the user's DUTs. The resulting waveforms are fed into the MFG digitizer and the reference digitizer and then measured to create the training data needed for the neural network. The switch 22 shows that these waveforms can be optional for the training process. For example, it is possible that training using only the BERT input waveform is all that is necessary for it to then be used to measure the DUT after training is complete.

[0029] After the measurement results package 226 takes the measurement results of the waveform data, the statistics and histogram block 32 calculates the statistics of mean, minimum, maximum, standard deviation and histogram from all training waveforms for all expected measurements. These should be used later during runtime to compare the neural network output measurement results to make a decision on the pass or fail of the measured results.

[0030] During training, the waveform made by the MFG digitizer may be converted into a series of short pattern waveforms at the short pattern separator 28, as described in U.S. Patent Publication 20220373598, "SHORT PATTERN WAVEFORM DATABASE MACHINE LEARNING FOR MEASUREMENT". A tensor builder 30 takes the waveform or its short pattern representation parameters of the MFG digitizer, and takes measurements (e.g., from the measurement package 124), and builds a tensor. The digitizer parameters may include bandwidth, gain, noise waveform, effective number of bits, etc. The measurements may include total jitter, duty cycle jitter, data dependent jitter, periodic jitter, non-periodic jitter, random jitter, random noise, periodic noise, data dependent noise, total noise, and non-periodic noise. These are just some examples and are not intended to limit the embodiments to any particular digitizer parameters or measurements. The data may be prepared to fit into a one-dimensional array tensor or vector, or it may be prepared to fit into a 2D or 3D array, such as a monochrome image or an RGB image. There are other possibilities that can be incorporated. For example, a multi-branch neural network can be configured to receive a 1D tensor and a 2D or 3D image tensor simultaneously.

[0031] The tensor builder 30 can build a tensor vector or image. If the tensor builder builds a vector, the vector undergoes normalization at 34 and is provided to the neural network in normalized form. In order to associate the resulting vector with a similar vector from the reference instrument 20, the reference digitizer parameters and the measurement results are provided to the reference tensor builder 38, which will build a reference tensor vector, which is then normalized at 36. Normalization translates the dynamic range of the input data to a range of -1 to 1 before inputting it into the neural network. Denormalization translates the normalized output range back to the expected numerical range of the output measurement result data. If normalization already exists inside the neural network structure, normalization may not be included. The neural network 40 then associates the measurement results represented by the MFG tensor from the tensor builder 30 with the tensor with the reference tensor to "teach" the neural network to translate between different measurement results. The user selects the tensor type before training and before use on the manufacturing line.

[0032] Figure 2 Shown from Figure 1 1 and 2. The runtime configuration of the trained embodiment of FIG. 1 is shown in FIG. 1 . In the runtime embodiment, the generator data waveform source 12 and the simulated waveform source 14 are not used. The user DUT 18 generates the waveform on which the measurement will be made. Generally, the measurement will be made by the MFG digitizer 10 and the tensor builder 30 will build the MFG tensor. The neural network will now output the result which undergoes denormalization at 42. The reference instrument 10 is not involved until the verification process at 44. The verification process 44 uses the statistics from 32 and the results from the neural network 40 which are denormalized at 42. The results from the neural network are used to determine whether the DUT has passed or has failed. This generally involves determining at 46 whether the value of the measurement is within the manufacturing tolerance for the measurement. If the measurement is within tolerance, the prediction is maintained at 48 and the DUT passes. If the measurement is out of bounds, then the reference digitizer 20 makes a measurement to make the final determination of pass or fail.

[0033] Figure 3 and 4 Provides Figure 1 and 2 1. In these processes, the MFG digitizer resides in the MFG test instrument 11, and more specifically resides in the ET oscilloscope. Similarly, the reference digitizer resides in the reference test instrument 21, which in this case is an RT mirror. The MFG mirror will use the JNB measurement result package 50, and the reference mirror will use the DPOJET measurement result package 52 from Tektronix. Figure 3 Shown with Figure 1Similar training configuration with those changes. Figure 4 Shown for Figure 3 Runtime configuration of the configuration. Figure 4 A validation process can be implemented to ensure that initial testing of the configuration works. Figure 5 In , the runtime configuration has been expanded to include multiple instances of the MFG mirror 11. Some manufacturing lines may use 100 MFG mirrors.

[0034] For this example, the customer will have about 100 ET sampling oscilloscopes in their MFG line to take measurements. The customer wants to use these because they have high bandwidth when needed, but the price is much less than the RT real-time sampling oscilloscopes. The customer will have one RT scope in this use case.

[0035] If only one measurement (such as non-periodic jitter) is input to the neural network for training, the neural network will not have enough information to predict the expected reference value. No mapping may be made. However, embodiments rely on the existence of many different measurements. This creates a large multi-dimensional input measurement space that allows various characteristics of the measured waveform to be seen. The neural network can see this measurement space representation of the set of measurements and then associate it with the reference measurement space and make a mapping or translation of all the measurements to what they should be.

[0036] Figure 6 and 7 Shows Figure 3 and 4 An example embodiment of the present invention but with principal component analysis (PCA) added to the system. Adding PCA can provide advantages for training the neural network. The PCA receives multiple measurement result types and multiple instances of measurement results along the length of the input waveform. The PCA outputs a data set representing the principal components in those measurements and provides valuable information about how the measurements may be affected by the parameters of the test instrument. Even though the internal neural network can extract component characteristics of the input data, PCA may have some additional advantages towards helping to extract some of the non-linear correlations between all the input measurements. If the neural network outputs a PCA vector, then the PCA can be inverse transformed to obtain the desired measurement result vector. In understanding that it can also be done in Figure 1 and 2 While PCA is provided in the more general embodiment of , PCA is added in the embodiment of the specific JNB / DPOJET example.

[0037] exist Figure 6, the JNB measurements output from 50 undergo PCA at 54, and this is used to construct the MFG tensor at 30. Similarly, the DPOJET measurements output from 52 undergo PCA at 56 prior to normalization at 36. The neural network is then trained on the tensor including the PCA. Figure 7 A runtime configuration is shown, with inverse PCA performed at 58 prior to validation at 44 .

[0038] Now return to the Figure 1 and 2 In more general embodiments such as those shown in , embodiments provide the user with the ability to control the digitizer. Specific test instrument parameters can affect the accuracy of the measurement results. These parameters can change as the digitizer settings change (such as vertical gain and offset or sampling rate and bandwidth changes). For example, the digitizer noise can change over a wide range with different vertical gain settings.

[0039] Inputting these digitizer parameters along with the measurements to the neural network during the training process makes it possible to associate measurements with different digitizer settings or even different digitizer models if more than one digitizer model type is included in the training process. The metadata will include an array of input measurements and an array of digitizer parameters associated with each set of measurements. The label data output from the neural network after training is the expected set of measurements after it has been denormalized and inverse PCA performed if PCA is used.

[0040] Figure 8An embodiment of a measurement translator system configuration including digitizer parameters for MFG instruments is shown. The MFG instrument may include a mirror, a real-time spectrum analyzer, or a digital multimeter, among many other examples. User automation software 70 controls the reference digitizer 20 and an array of MFG digitizers (such as 10) as they collect and measure waveforms from many user DUTs 72, typically only for training in this example. In general, the user automation software controls the DUTs, test instruments, measurement requests, etc. The digitizers provide their parameters and settings to the user automation software at 70, and measurements are performed and measurement statistics are generated at 74. The measurements 94 from the reference digitizers are also subjected to statistical and / or histogram analysis at 32, and the results are combined at 76. The combination at 76 improves the dynamic range of the measurements by inputting only the difference between the measurement mean and the measured value into the tensor builder block. The tensor builder 30 uses the measurement statistics from 74, the measurement statistics minus the reference statistics, and the digitizer parameters from the user automation software in the form of metadata normalized at 34 and 38 to build a tensor. The neural network then takes the tensor and the normalized reference measurements as training data. The measurements from the reference digitizer are labeled with the metadata.

[0041] Fig. 9 The runtime configuration is shown. Fig. 9 The reference digitizer is not shown to simplify the drawing. However, as can be seen at 78, if the verification finds that the measurement result is out of bounds, the DUT will fail, or may undergo retesting using the reference digitizer. Fig.10 A training configuration including PCA 54 is shown, and Fig.11 A runtime configuration including PCA is shown.

[0042] Up to this point, the discussion has focused on MFG digitizers and reference digitizers that are not tied to specific software measurement packages, except for specific examples related to JNB and DPOJET. The discussion now moves to Fig.12 and 13 An embodiment of measurement result translation between software packages is shown in .

[0043] The system consists of a main block labeled MFG measurement result package 1100 and a second block labeled reference measurement result package 2102. These packages may be the same or different than the other measurement result packages discussed above. Examples of translations include but are not limited to JNB to DPOJET, both owned by Tektronix, EZJIT to DPOJET translators between Keysight and Tektronix, or reverse translation between LeCroy or Anritsu and others, etc.

[0044] As discussed above, the system uses a large multi-dimensional input measurement space that allows various characteristics of the waveform being measured to be seen. The neural network can see this measurement space representation of a set of measurement results, and then associate it with a reference measurement space and make a mapping of all the measurement results to what they should be.

[0045] Fig.12 The runtime configuration of the measurement translator between different measurement packages is shown. As discussed above, each measurement package can characterize the same or similar measurement in a different way. The characterization can vary between the two packages. During training, the MFG package 100 builds a tensor of images or vectors, while the reference measurement package 102 does the same. Statistics and histograms are generated for the reference measurements at 32 before the tensor is built. The neural network then undergoes training to form an association between the two. Fig.13 An embodiment of a runtime configuration is shown.Similar to the digitizer example, the translated measurements undergo validation at 44, and if the DUT fails, a reference instrument using a reference package can take measurements to confirm a pass or fail.

[0046] Having discussed translations between MFG and reference digitizers independent of the measurement package used, and translations between measurement packages separate from the digitizer / instrument used, the discussion now turns to embodiments in which a test instrument is combined with a specific measurement package. In these embodiments, a neural network translates between measurements based on parameters of the test instrument and based on differences in the measurement package / characterization between the MFG instrument and the reference instrument.

[0047] Embodiments train a neural network in this case to correlate the measurements and characteristic parameters of the MFG test instrument with the ideal measurement values ​​from a reference test instrument. Embodiments can be used for many classes of instruments. Embodiments use a neural network to take into account the measurements and statistics from the MFG test instrument. It can also take into account the parameters of the instrument responsible for the errors in the measurements and then correlate them with what the ideal measurements should be. The test instrument may contain a digitizer and generally receives a waveform as input and then the measurement SW generates the measurements and statistics on the waveform. The test instrument may include but is not limited to RT oscilloscopes, ET oscilloscopes, margin testers, BER test systems, real-time spectrum analyzers, voltmeters, ammeters, etc.

[0048] Measurement errors may be due in part to differences in test instrument hardware, and also in part to differences in SW algorithm measurement packaging.Embodiments allow both areas to be considered in the prediction for the translation from MFG test instrument / measurement set to reference test instrument / measurement set.

[0049] Fig.14 and 15 A block diagram for a combined mirror and measurement translation system is shown. This embodiment translates measurements from a MFG test instrument run measurement package 1 to match a reference test instrument run measurement package 2. The test instrument may be an oscilloscope, spectrum analyzer, voltmeter, ammeter, or any other kind of test instrument where a translator may be valuable.

[0050] like Fig.14 As shown in FIG, the system consists of a main block labeled MFG test instrument 110, which contains a digitizer 10 and a measurement package 1112; and a reference test instrument 120, which contains a reference digitizer 20 and a measurement package 122. Again, the measurement packages 1 and 2 in these embodiments may be different from the measurement packages discussed previously. The mirror and the second block are labeled reference RT mirrors. The tensor builder 30 will build a tensor using the digitizer and possibly the instrument information with a waveform or short waveform and the measurements for the MFG instrument, and the tensor builder 38 will do the same for the reference digitizer, instrument (if used) and measurements. This trains the system similarly to the previous embodiments. Fig.15 The runtime configuration is shown.

[0051] The embodiments involve training a neural network 40 so that it can receive a set of jitter and noise values ​​for an input waveform. These measurements are obtained from the MFG instrument 110 and measurement wrapper 1112 and from the reference instrument 120 and measurement wrapper 2122. However, in general, the instrument may include any test instrument with a digitizer. The measurement wrapper may include any software wrapper capable of performing the desired measurements on the input digitized data. The data may be a waveform, or it may be a voltage from a voltmeter or a current from an ammeter, etc. The embodiments transform the measurements from one test instrument and SW into what will be obtained on a different instrument and software. The transformation will take into account any effect that the particular instrument has on the measurements.

[0052] Embodiments herein provide a new method that enables customers to use a preferred test instrument for MFG test equipment in large quantities and translate the measurements to what they would be on a higher performance reference test instrument. This saves customers a lot of money where they could have 100 or more less expensive MFG test instruments and only one or more expensive reference test instruments. Embodiments provide for translating between MFG and reference test instruments regardless of whether they use a measurement package integrated into the instrument or whether the package is separate. Embodiments provide for translating between packages where the instrument and digitizer are not in dispute.

[0053] Aspects of the present disclosure may be operated on specially created hardware, on firmware, digital signal processors, or on a specially programmed general-purpose computer including a processor that operates according to programmed instructions. The term controller or processor as used herein is intended to include microprocessors, microcomputers, application-specific integrated circuits (ASICs), and dedicated hardware controllers. One or more aspects of the present disclosure may be embodied in computer-usable data and computer-executable instructions executed by one or more computers (including monitoring modules) or other devices, such as one or more program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types when executed by processors in computers or other devices. Computer-executable instructions may be stored on non-transient computer-readable media, such as hard disks, optical disks, removable storage media, solid-state memory, random access memory (RAM), etc. As will be appreciated by those skilled in the art, the functionality of program modules may be combined or distributed as desired in various aspects. In addition, functionality may be embodied in whole or in part in firmware or hardware equivalents (such as integrated circuits, FPGAs, etc.). Certain data structures may be used to more efficiently implement one or more aspects of the present disclosure, and such data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein.

[0054] In some cases, the disclosed aspects may be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried by one or more non-transient computer-readable media or stored on one or more non-transient computer-readable media, which may be read and executed by one or more processors. Such instructions may be referred to as computer program products. As discussed herein, computer-readable media means any medium accessible by a computing device. As an example and not limitation, computer-readable media may include computer storage media and communication media.

[0055] Computer storage media means any medium that can be used to store computer-readable information. As an example and not limitation, computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital video disk (DVD), or other optical disk storage, cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, and any other volatile or non-volatile, removable or non-removable media implemented in any technology. Computer storage media excludes the signal itself and the signal transmission in transient form.

[0056] Communication media refers to any medium that can be used for communication of computer-readable information. By way of example and not limitation, communication media can include coaxial cables, fiber optic cables, air, or any other medium suitable for communication of electrical, optical, radio frequency (RF), infrared, acoustic or other types of signals.

[0057] Example Illustrative examples of the disclosed technology are provided below. Embodiments of the technology may include one or more and any combination of the examples described below.

[0058] Example 1 is a test and measurement system comprising: a first test and measurement instrument having: an input to allow the test and measurement instrument to receive signals from one or more devices under test (DUTs); and one or more digitizers to convert signals from the one or more DUTs into digital waveforms; a machine learning network; and one or more processors configured to execute code that causes the one or more processors to: perform one or more measurements of the digital waveform; send the one or more measurements of the digital waveform to the machine learning network as input; use the machine learning network to translate the one or more measurements into measurements made by a reference instrument to produce one or more translated measurements, the reference instrument being more accurate than the first test and measurement instrument; and determine whether the DUT meets performance requirements based on the one or more translated measurements.

[0059] Example 2 is the test and measurement system of Example 1, wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system includes code causing the one or more processors to perform the following operations: sending one or more digitizer parameters with the one or more measurement results to a tensor builder before the machine learning network.

[0060] Example 3 is the test and measurement system of any of Examples 1 or 2, wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system includes code causing the one or more processors to perform the following operations: sending one or more measurement result statistics with the one or more measurement results to a tensor builder before the machine learning network.

[0061] Example 4 is the test and measurement system of any one of Examples 1 to 3, wherein the code causing the one or more processors to send the one or more measurement result statistics includes code for performing the following operations: sending a set of principal components having the one or more measurement results to a tensor builder before the machine learning network.

[0062] Example 5 is the test and measurement system of Example 2, wherein the one or more processors are further configured to execute code for performing the following operations: constructing a tensor from the one or more digitizer parameters and the one or more measurement results to send to the machine learning network.

[0063] Example 6 is the test and measurement system of any one of Examples 1 to 5, wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system includes code causing the one or more processors to perform the following operations: separating the digital waveform into one or more short pattern waveforms; and sending the short pattern waveform to a tensor builder before the machine learning network.

[0064] Example 7 is the test and measurement system of any one of Examples 1 to 6, wherein the code that causes the one or more processors to determine whether the DUT meets the performance requirements based on the one or more translated measurement results includes code that causes the one or more processors to determine that the DUT does not meet the performance requirements, and the one or more processors are further configured to execute code that causes the one or more processors to perform the following operations: directing the reference instrument to digitize the signal from the DUT and perform the same one or more measurement results on the digital waveform; and determining whether the DUT meets the performance requirements based on the one or more measurement results from the reference instrument.

[0065] Example 8 is the test and measurement system of any one of Examples 1 to 7, wherein the code that causes the one or more processors to determine whether the DUT meets the performance requirements based on the one or more translated measurements from the machine learning network includes code that causes the one or more processors to determine that the DUT does not meet the performance requirements, and the one or more processors are further configured to execute code that causes the one or more processors to report the DUT as failed.

[0066] Example 9 is the test and measurement system of any one of Examples 1 to 8, wherein the one or more processors are further configured to execute code that causes the one or more processors to train the machine learning network.

[0067] Example 10 is the test and measurement system of any one of Examples 1 to 9, wherein the one or more processors are further configured to execute code that causes the one or more processors to train the machine learning network, and the code that causes the one or more processors to train the machine learning network includes code that causes the one or more processors to perform the following operations: receiving one or more measurement results on the digital waveform from the first test and measurement instrument; receiving one or more measurement results on the digital waveform from the reference instrument; and associating the one or more measurement results from the first test and measurement instrument with corresponding measurement results in the one or more measurement results from the reference instrument.

[0068] Example 11 is a test and measurement system of any one of Examples 1 to 10, wherein the one or more processors are further configured to execute code that causes the one or more processors to perform the following operations: perform at least one of calculating statistics of the measurement results and performing principal component analysis of the measurement results to associate at least one of the statistics and principal components with the measurement results.

[0069] Example 12 is a test and measurement system of any one of Examples 1 to 11, wherein: the code causing the one or more processors to execute one or more measurement results of the digital waveform includes code causing the one or more processors to execute the one or more measurement results using a first representation of the one or more measurement results; the code causing the one or more processors to send the one or more measurement results of the digital waveform to the machine learning network as input includes code causing the one or more processors to send the one or more measurement results of the digital waveform and the first representation to the machine learning network as input; and the code causing the one or more processors to use the machine learning network to translate the one or more measurement results into measurement results made by the reference instrument causes the one or more processors to map the one or more measurement results made using the first representation of the one or more measurement results to the measurement results made by the reference instrument using the second representation.

[0070] Example 13 is a test and measurement system comprising: a first test and measurement instrument having: an input to allow the test and measurement instrument to receive signals from one or more devices under test (DUTs); and one or more digitizers to convert signals from the DUTs into digital waveforms; a machine learning network; one or more processors configured to execute code that causes the one or more processors to perform the following operations: using the first test and measurement instrument to perform one or more measurement results of the digital waveform, the first test and measurement instrument using a first representation of the one or more measurement results; sending the one or more measurement results and the first representation of the digital waveform to the machine learning network as input; using the machine learning network to translate the one or more measurement results into measurements made by a reference instrument to produce one or more translated measurement results, the reference instrument being more accurate than the first test and measurement instrument and using a second representation of the one or more measurement results; and using one or more translated measurement results from the machine learning network to determine whether the DUT meets performance requirements.

[0071] Example 14 is the test and measurement system of Example 13, wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system includes code causing the one or more processors to perform the following operations: sending one or more digitizer parameters with the one or more measurement results to a tensor builder before the machine learning network.

[0072] Example 15 is the test and measurement system of any one of Examples 13 and 14, wherein the code causing the one or more processors to send the one or more measurements and the first representation to the machine learning system includes code causing the one or more processors to perform the following operations: sending one or more measurement result statistics with the one or more measurements and the first representation to a tensor builder before the machine learning network.

[0073] Example 16 is the test and measurement system of Example 15, wherein the code that causes the one or more processors to send one or more measurement result statistics having the one or more measurement results to a tensor builder prior to the machine learning network includes code for performing the following operations: sending a set of principal components having the one or more measurement results to a tensor builder prior to the machine learning network.

[0074] Example 17 is the test and measurement system of any one of Examples 13 to 16, wherein the one or more processors are further configured to execute code for performing the following operations: constructing a tensor from the one or more digitizer parameters and the one or more measurement results to send to the machine learning network.

[0075] Example 18 is the test and measurement system of any one of Examples 13 to 17, wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system includes code causing the one or more processors to perform the following operations: separating the digital waveform into one or more short pattern waveforms; and sending the short pattern waveform to a tensor builder before the machine learning network.

[0076] Example 19 is the test and measurement system of any one of Examples 13 to 18, wherein the code that causes the one or more processors to determine using one or more translated measurements from the machine learning network includes code that causes the one or more processors to determine using one or more of the translated measurements that the DUT does not meet the performance requirement, and the one or more processors are further configured to execute code that causes the one or more processors to: direct the reference instrument to digitize the signal from the DUT and perform the same one or more measurements on the digital waveform; and determine whether the DUT meets the performance requirement based on the one or more measurements from the reference instrument.

[0077] Example 20 is the test and measurement system of any one of Examples 13 to 19, wherein the one or more processors are further configured to execute code that causes the one or more processors to train the machine learning network, and the code that causes the one or more processors to train the machine learning network causes the one or more processors to: receive one or more measurement results on a digital waveform from the first test and measurement instrument using the first representation; receive one or more measurement results on a digital waveform from the reference test and measurement instrument using the second representation; and associate one or more measurement results from the first test and measurement instrument and the first representation of the one or more measurement results with corresponding measurement results and second representations in one or more measurement results from the reference test and measurement instrument and the second representation of the one or more measurement results.

[0078] In addition, the written description mentions specific features. It should be understood that the disclosure in this specification includes all possible combinations of those specific features. Where a specific feature is disclosed in the context of a particular aspect or example, the feature may also be used in the context of other aspects and examples as much as possible.

[0079] Furthermore, when the present application refers to a method having two or more defined steps or operations, the defined steps or operations may be performed in any order or simultaneously, unless the context excludes those possibilities.

[0080] All features disclosed in the specification including claims, abstract and drawings, and all steps in any disclosed method or process may be combined in any combination except combinations in which at least some of such features and / or steps are mutually exclusive. Each feature disclosed in the specification including claims, abstract and drawings may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0081] Although specific examples of the present invention have been shown and described for purposes of illustration, it should be understood that various modifications can be made without departing from the spirit and scope of the present invention. Accordingly, the present invention should not be limited, except by the appended claims.

Claims

1. A test and measurement system comprising: A first test and measurement instrument having: an input to allow the test and measurement instrument to receive signals from one or more devices under test (DUT); and one or more digitizers to convert signals from the one or more DUTs into digital waveforms; Machine learning networks; as well as One or more processors configured to execute code that causes the one or more processors to perform the following operations: performing one or more measurements of the digital waveform; sending the one or more measurements of the digital waveform to the machine learning network as input; using the machine learning network to translate the one or more measurements into measurements made by a reference instrument to produce one or more translated measurements, the reference instrument being more accurate than the first test and measurement instrument; as well as A determination is made as to whether the DUT meets performance requirements based on the one or more translated measurements.

2. The test and measurement system of claim 1 , wherein the code causing the one or more processors to send the one or more measurements to a machine learning system comprises code causing the one or more processors to send one or more digitizer parameters with the one or more measurements to a tensor builder prior to the machine learning network.

3. The test and measurement system of claim 1 , wherein the code causing the one or more processors to send the one or more measurement results to a machine learning system comprises code causing the one or more processors to send one or more measurement result statistics with the one or more measurement results to a tensor builder prior to the machine learning network.

4. The test and measurement system of claim 1 , wherein the code causing the one or more processors to send the one or more measurement statistics comprises code for sending a set of principal components having the one or more measurement results to a tensor builder prior to the machine learning network.

5. The test and measurement system of claim 2, wherein the one or more processors are further configured to execute code for performing the following operations: constructing a tensor from the one or more digitizer parameters and the one or more measurement results to send to the machine learning network.

6. The test and measurement system of claim 1 , wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system comprises code causing the one or more processors to: separate the digital waveform into one or more short pattern waveforms; and send the short pattern waveform to a tensor builder prior to the machine learning network.

7. The test and measurement system of claim 1 , wherein the code that causes the one or more processors to determine whether the DUT meets the performance requirement based on the one or more translated measurement results comprises code that causes the one or more processors to determine that the DUT does not meet the performance requirement, and the one or more processors are further configured to execute code that causes the one or more processors to: directing the reference instrument to digitize the signal from the DUT and perform the same one or more measurements on the digital waveform; and A determination is made as to whether the DUT meets the performance requirement based on one or more measurement results from the reference instrument.

8. The test and measurement system of claim 1 , wherein the code that causes the one or more processors to determine whether the DUT meets the performance requirement based on the one or more translated measurements from the machine learning network includes code that causes the one or more processors to determine that the DUT does not meet the performance requirement, and the one or more processors are further configured to execute code that causes the one or more processors to report the DUT as failed.

9. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute code that causes the one or more processors to train the machine learning network.

10. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute code that causes the one or more processors to train the machine learning network, and the code that causes the one or more processors to train the machine learning network includes code that causes the one or more processors to: receiving one or more measurements on a digital waveform from said first test and measurement instrument; receiving one or more measurements on the digital waveform from the reference instrument; as well as One or more measurements from the first test and measurement instrument are correlated to corresponding measurements of the one or more measurements from the reference instrument.

11. The test and measurement system of claim 10, wherein the one or more processors are further configured to execute code that causes the one or more processors to perform at least one of calculating statistics of the measurement results and performing principal component analysis of the measurement results to associate at least one of the statistics and principal components with the measurement results.

12. The test and measurement system of claim 1, wherein: The code causing the one or more processors to perform one or more measurements of the digital waveform includes code causing the one or more processors to perform the one or more measurements using a first representation of the one or more measurements; The code causing the one or more processors to send one or more measurements of the digital waveform to the machine learning network as input includes code causing the one or more processors to send one or more measurements of the digital waveform and the first representation to the machine learning network as input; and Code that causes the one or more processors to use the machine learning network to translate the one or more measurements into measurements made by the reference instrument causes the one or more processors to map the one or more measurements made using a first characterization of the one or more measurements to measurements made by the reference instrument using a second characterization.

13. A test and measurement system comprising: A first test and measurement instrument having: an input to allow the test and measurement instrument to receive signals from one or more devices under test (DUT); and one or more digitizers to convert signals from the DUT into digital waveforms; Machine learning networks; One or more processors configured to execute code that causes the one or more processors to perform the following operations: performing one or more measurements of the digital waveform using the first test and measurement instrument, the first test and measurement instrument using a first characterization of the one or more measurements; sending the one or more measurements of the digital waveform and the first representation as input to the machine learning network; using the machine learning network to translate the one or more measurements into measurements made by a reference instrument that is more accurate than the first test and measurement instrument to produce one or more translated measurements and using a second representation of the one or more measurements; as well as One or more translated measurements from the machine learning network are used to determine whether the DUT meets performance requirements.

14. The test and measurement system of claim 13, wherein the code causing the one or more processors to send the one or more measurements to a machine learning system comprises code causing the one or more processors to send one or more digitizer parameters with the one or more measurements to a tensor builder prior to the machine learning network.

15. The test and measurement system of claim 13, wherein the code causing the one or more processors to send the one or more measurements and the first representation to a machine learning system comprises code causing the one or more processors to send one or more measurement statistics with the one or more measurements and the first representation to a tensor builder prior to the machine learning network.

16. The test and measurement system of claim 15, wherein the code that causes the one or more processors to send one or more measurement result statistics having the one or more measurement results to a tensor builder prior to the machine learning network comprises code for sending a set of principal components having the one or more measurement results to a tensor builder prior to the machine learning network.

17. The test and measurement system of claim 13, wherein the one or more processors are further configured to execute code for performing the following operations: constructing a tensor from the one or more digitizer parameters and the one or more measurement results to send to the machine learning network.

18. The test and measurement system of claim 13, wherein the code causing the one or more processors to send the one or more measurement results to the machine learning system comprises code causing the one or more processors to: separate the digital waveform into one or more short pattern waveforms; and send the short pattern waveform to a tensor builder prior to the machine learning network.

19. The test and measurement system of claim 13, wherein the code that causes the one or more processors to determine using one or more translated measurements from the machine learning network comprises code that causes the one or more processors to determine using one or more of the translated measurements that the DUT does not meet the performance requirement, and the one or more processors are further configured to execute code that causes the one or more processors to: directing the reference instrument to digitize the signal from the DUT and perform the same one or more measurements on the digital waveform; and A determination is made as to whether the DUT meets the performance requirement based on one or more measurement results from the reference instrument.

20. The test and measurement system of claim 13, wherein the one or more processors are further configured to execute code that causes the one or more processors to train the machine learning network, and the code that causes the one or more processors to train the machine learning network causes the one or more processors to: receiving one or more measurements on a digital waveform from the first test and measurement instrument using the first characterization; receiving one or more measurements on a digital waveform from the reference test and measurement instrument using the second characterization; as well as One or more measurements and a first representation of the one or more measurements from the first test and measurement instrument are associated with corresponding measurements and a second representation of the one or more measurements and the one or more measurements from the reference test and measurement instrument.

Images