System and method for decimation scan measurements of device under test using machine learning

Abstract

A test and measurement instrument includes: one or more ports for allowing the test and measurement instrument to receive signals from a device under test (DUT); a user interface for allowing a user to send inputs to the test and measurement instrument and receive results; and one or more processors configured to acquire the signal from the DUT, measure the signal to create an extracted set of measurements, convert the extracted set of measurements into a tensor, transmit the tensor to a machine learning network, and receive a pass / failure value from the machine learning network. A method includes acquiring a signal from a device under test (DUT), measuring the signal to create an extracted set of measurements, converting the extracted set of measurements into a tensor, transmitting the tensor to a machine learning network, and receiving a pass / fail value from the machine learning network.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure is a non-provisional application of U.S. Provisional Application No. 63 / 585,204, filed on September 25, 2023, entitled “SYSTEM AND METHOD FOR DECIMATEDSWEEP MEASUREMENTS OF A DEVICE UNDER TEST USING MACHINE LEARNING,” and claims the benefit of that U.S. Provisional Application, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to test and measurement systems and methods, and more particularly to applying machine learning to measure characteristics of signals from a device under test. Background Art

[0004] When measuring the performance of a device under test (DUT), the user typically performs a "full scan" of measurements. Examples of measurements include, but are not limited to, various current and / or voltage measurements. As used herein, the term "full scan" means measurements of all possible measurement points in the data. Performing these many measurements takes a long period of time. Typically, the measurements allow the user of the test device to make a pass or fail determination of the DUT.

[0005] Users will attempt to reduce the number of measurements required in order to speed up the data collection process. However, users need to balance the desire for faster measurements with the need for accurate pass / fail determinations. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An embodiment of a training configuration for a test and measurement system is shown.

[0007] Figure 2 An embodiment of a runtime configuration for a test and measurement system is shown.

[0008] Figure 3 An embodiment of a test and measurement instrument is shown. DETAILED DESCRIPTION

[0009] In general, embodiments herein associate full scan measurements and pass / fail results with a decimated version of the scan measurements. This allows the user to speed up the acquisition of measurement data. Rather than just decimating measurements and basing pass / fail results on the decimated measurements, the system will instead train a neural network that uses input labels of pass / fail based on the slower full scan data set and associates the labels with a decimated set of measurements taken from the full scan set. Allowing a smaller decimated set while maintaining accuracy will speed up the process even more.

[0010] An example use case for some embodiments of the present disclosure includes a manufacturing situation where multiple measurements must be made on a device under test (DUT) and then processed to indicate pass / fail or other failure type classification, etc. For example, one may want to measure the IV - current vs. voltage - curve of a component. This may require many point measurements over a long period of time. In addition, data measurements like transfer functions, such as s21 frequency response type data, may require many points and are also very time consuming.

[0011] A user or programmer of such a test system will naturally attempt to reduce the number of measurements required in order to speed up the data collection process. Embodiments of the present disclosure enable a user to reduce the data measurement set even further by using a neural network. Embodiments involve training a machine learning system on the full data set to create training labels, but associating the measurements with the reduced data set. Discussions herein refer to the reduced data set as an "extracted" data set.

[0012] Figure 1 An embodiment of a system configuration for training a machine learning network to provide a pass / fail result for a device under test (DUT) using extracted measurements is shown. This block diagram represents a system configuration for training a neural network according to some example embodiments. The label on the neural network block during training will be the input and will be the pass / fail prediction value. The label can have a value of 1.0 for pass and 0 for fail. The neural network will be trained to associate this label with other inputs.

[0013] In some embodiments, for example, the DUT can be an integrated circuit die, such as a silicon photonic transceiver wafer die. According to some of these embodiments, other inputs to the neural network can consist of six measurement sets obtained from the die through a test station setup. Other examples of DUTs include electronic devices, optical devices such as optical switches, and many other devices. Examples of measurements include current-voltage (IV) curves, S-parameters (S11, S12, S21, and S22) in photonics, or other characteristics.

[0014] Additionally, some embodiments may employ other parameters, such as ambient temperature or model number of wafer / die type, etc. as inputs to the neural network. Training the neural network requires a sufficient amount of data, meaning there are enough DUT samples to cover the expected distribution of DUT characteristics that will be seen during manufacturing. A sufficient amount of data exists when the neural network can be trained to make predictions within a predetermined accuracy threshold.

[0015] Figure 1An embodiment of a training configuration of a system that can use extracted measurements to obtain a pass / fail of a DUT is shown. The test and measurement instrument forms part of a test station 12. The test station is connected to the DUT 10 to receive a signal from the DUT, from which the test and measurement instrument can obtain measurement data such as a waveform. The test and measurement instrument, referred to herein as an "instrument," performs a full scan measurement on each DUT to produce a full scan measurement set at 14. The instrument then reduces each full scan measurement down to a defined reduced measurement set 16, referred to herein as an "extracted measurement set."

[0016] In one embodiment, the tensor builder 20 converts each extracted measurement into a tensor, or a one-dimensional data vector, or a tensor image such as a red-green-blue (RBG) image. More information about tensor images can be found in U.S. patent application Ser. No. 17 / 701,186, filed on March 22, 2021, “OPTICAL TRANSMITTER TUNING USING MACHINELEARNING AND REFERENCE PARAMETERS” and U.S. patent application Ser. No. 17 / 747,954, filed on May 21, 2021, “SHORT PATTERN WAVEFORM DATABASE MACHINE LEARNINGFORMEASUREMENT”, both of which are incorporated herein in their entirety.

[0017] When the tensor builder 20 produces an image tensor, no external normalization block is needed. When the tensor construction produces a single input vector, then a normalization block 22 will be needed to represent the data in the range of 0 to 1 for input to the first neural network layer. For label inputs, if the label is 1 for pass and 0 for fail, then an external normalization block may not be needed. In this case, the output of the label vector block 24 will go directly into the neural network. However, if it is desirable to have the network predict other numerical values ​​in addition to pass and fail, then a normalization block may be needed depending on the nature of the variable.

[0018] Other information, such as the temperature of the DUT during the test, the model # of the DUT, and any other information may also be added to, incorporated into, or encoded in the tensor produced by the tensor builder 20. The full scan measurements corresponding to the extracted measurements undergo analysis at 18 to see if the DUT will pass or fail. The analysis block 18 applies the algorithms needed to analyze the full scan data set and determine what the pass / fail label should be.

[0019] The pass or fail label will also be converted into a label vector at 24. This may undergo normalization at 26 for input to the neural network 28. As discussed above, depending on the value used for pass / fail, the normalization block 26 will not be needed. The extracted measurement tensor is assigned a pass / fail label. This will allow the machine learning network to associate the extracted measurements with the pass or fail labels. Ultimately, this will result in the machine learning network 28 being able to determine pass or fail from the extracted measurements during runtime.

[0020] In summary, the training process includes characterizing a sufficient number of DUTs, such as semiconductor dies, performing a full scan of data points for each DUT. Each full scan of the data points undergoes analysis to identify a pass or fail. The full scan of the data points is extracted to produce extracted data. The extracted data and any other data will then enter the tensor builder and then normalized if necessary. The resulting tensor and label tensor then enter the neural network for training.

[0021] Figure 2 An embodiment of a runtime configuration of a test system is shown. Figure 2 In the configuration of , the full scan measurement set is an option for failed DUTs. The main process flow involves creating extracted data, building tensors, and sending tensor inputs to the neural network. This configuration does not collect a full scan data set. The TestStation block will only spend time collecting the extracted data measurement set. After the neural network receives the extracted data, it provides a value indicating pass or fail (PF).

[0022] The PF results can take many forms and can be further analyzed to make a more precise definition. As mentioned above, the results can undergo un-normalizing at 30, or not, depending on whether the pass / fail range is between 0 and 1. Further analysis can include determining a threshold indicating a pass or fail, or a threshold indicating a pass and a threshold indicating a fail. The label output from the neural network can be represented from a regression layer in the neural network. When the expected value output of "pass" is 1.0 and the value of "fail" is 0.0, or in any embodiment where the range is from 0 to 1, the unnormalization block 30 will not be needed. The neural network can output a score value anywhere between 0 and 1, indicating the degree of pass or fail.

[0023] exist Figure 2 In the embodiment of FIG. 3 , the pass threshold is whether the PF value is greater than or equal to 0.8, in which case the DUT passes, at 32. If the PF value is less than or equal to 0.2, at 34, the DUT fails. Other options may include a single threshold, where values ​​above the threshold are passes and values ​​below the threshold are fails.

[0024] If this determination uses separate thresholds, and the PF value lies between the separate thresholds, a problem may arise. This results in unclear or "fuzzy" results. For example, the DUT does not pass at 32, and does not fail at 34. At this point, there is an option to perform a full scan measurement set, such as Figure 1 Then the full scan is analyzed at 18 and the final result is obtained.

[0025] The pass / fail output from the neural network then determines whether to package the die or otherwise move the DUT through the manufacturing process. If the result is a fail, the chip or DUT is discarded. If the result is a pass, the die should be placed in a package, or the DUT will be passed into the manufacturing process. In summary, threshold levels can be assigned, such as the label must be greater than 0.8 to pass, and less than 0.2 to fail, and thresholds other than 0.8 and 0.2 can be used. By limiting the use of full scans to only unclear results, the process becomes much faster.

[0026] The device under test (DUT) generates signals that are captured by the test and measurement instrument. Figure 3 An embodiment of such an instrument is shown. Figure 3 An embodiment of a testing setup in an example where a test and measurement instrument measures or tests a DUT 10 is shown. The testing setup may include a test and measurement instrument 40, which may include an oscilloscope. The test and measurement instrument 40 receives a signal from the DUT 10 directly or through an instrument probe 46. In the case of an optical transmitter DUT, the probe will typically include a test optical fiber coupled to an optical-to-electrical converter 46, not shown, which provides a signal to the test and measurement instrument through one or more ports 43. Two ports may be used for differential signaling, and one port for single-ended signaling. The instrument samples and digitizes the signal to become a waveform. For example, if the test and measurement instrument 40 includes a sampling oscilloscope, a clock recovery unit (CRU) 50 may recover a clock signal from the data signal. Software clock recovery may be used on a real-time oscilloscope.

[0027] The test and measurement instrument has one or more processors represented by processor 42, memory 52, and a user interface 56. The memory can store executable instructions in the form of code, which when executed by the processor, causes the processor to perform tasks. The user interface 56 of the test and measurement instrument allows the user to interact with the instrument 40, such as entering settings, configuring tests, etc. The test and measurement instrument may also include a reference equalizer and an analysis module 54.

[0028] Embodiments herein employ machine learning in the form of a machine learning network 28, such as a deep learning network. The machine learning network may include a processor that has been programmed with the machine learning network as part of a test and measurement instrument or to which the test and measurement instrument has access. As test equipment capabilities and processors evolve, one or more processors such as 42 may include both. The machine learning system may take the form of a programming model that operates on one or more of the processors. The processor may also be configured to execute code to perform the method of the embodiment.

[0029] Embodiments provide an instrument and generalized method that uses machine learning to make pass / fail predictions based on extracted measurement sets. The network is trained using pass / fail labels derived from a full scan measurement set. This results in accelerated measurement time by collecting data for only the extracted scan measurement set. If the prediction is ambiguous between pass and fail, a full data scan can be performed to make a final prediction. Alternatively, a DUT that is below the pass threshold can also be discarded.

[0030] Aspects of the present disclosure may be operated on specially created hardware, firmware, a digital signal processor, or a specially programmed general-purpose computer including a processor operating according to programming instructions. The term controller or processor as used herein is intended to include a microprocessor, a microcomputer, an application-specific integrated circuit (ASIC), and a dedicated hardware controller. One or more aspects of the present disclosure may be embodied in computer-usable data and computer-executable instructions, such as in one or more program modules, which are executed by one or more computers (including monitoring modules) or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types when executed by a processor in a computer or other device. Computer executable instructions may be stored on a non-transitory computer-readable medium, such as a hard disk, an optical disk, a removable storage medium, a solid-state memory, a random access memory (RAM), etc. As will be appreciated by those skilled in the art, the functionality of the program modules may be combined or distributed in various aspects as desired. In addition, the functionality may be embodied in firmware or hardware equivalents such as integrated circuits, FPGAs, etc., in whole or in part. Certain data structures may be used to more efficiently implement one or more aspects of the present disclosure, and such data structures are contemplated to be within the scope of the computer-executable instructions and computer-usable data described herein.

[0031] In some cases, the disclosed aspects can be implemented with hardware, firmware, software or any combination thereof. The disclosed aspects can also be implemented as instructions carried or stored thereon by one or more or non-transitory computer-readable media, which can be read and executed by one or more processors. Such instructions can be referred to as computer program products. As discussed herein, computer-readable media means any medium that can be accessed by a computing device. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0032] Computer storage media means any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital video disk (DVD) or other optical disk storage, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any other volatile or non-volatile, removable or non-removable media implemented in any technology. Computer storage media excludes signals themselves and transient forms of signal transmission.

[0033] Communication media means any medium that can be used for communication of computer-readable information. By way of example and not limitation, communication media may include coaxial cables, fiber optic cables, air, or any other medium suitable for communication of electrical, optical, radio frequency (RF), infrared, acoustic or other types of signals.

[0034] Example

[0035] The following provides illustrative examples of the disclosed technology. Embodiments of the technology may include one or more of the following examples and any combination thereof.

[0036] Example 1 is a test and measurement instrument comprising: one or more ports for allowing the test and measurement instrument to receive signals from a device under test (DUT); a user interface for allowing a user to send inputs to the test and measurement instrument and receive results; and one or more processors configured to execute code that causes the one or more processors to: acquire signals from the DUT; measure the signals to create an extracted measurement set; convert the extracted measurement set into a tensor; send the tensor to a machine learning network; and receive a pass / fail value from the machine learning network.

[0037] Example 2 is the test and measurement instrument of Example 1, wherein the code that causes the one or more processors to receive a pass / fail value from the machine learning network causes the one or more processors to receive the pass / fail value as a value between 1.0 and 0.0, inclusive.

[0038] Example 3 is the test and measurement instrument of any of Examples 1 or 2, wherein the one or more processors are further configured to execute code that causes the one or more processors to compare the pass / fail value to a threshold to determine whether the DUT passes or fails.

[0039] Example 4 is the test and measurement instrument of Example 3, wherein the DUT passes when the pass / fail value is greater than or equal to 0.8, and the DUT fails when the value is less than or equal to 0.2.

[0040] Example 5 is the test and measurement instrument of any of Examples 1 to 4, wherein the one or more processors are further configured to execute code to cause the one or more processors to perform a full scan measurement of the DUT when a pass / fail indication is unclear.

[0041] Example 6 is the test and measurement instrument of Example 5, wherein a user provides input through the user interface selecting to perform a full scan measurement.

[0042] Example 7 is the test and measurement instrument of any of Examples 1 to 6, wherein the one or more processors are further configured to execute code that causes the one or more processors to train a machine learning network.

[0043] Example 8 is a test and measurement instrument as claimed in claim 7, wherein the code that causes the one or more processors to train the machine learning network includes code that causes the one or more processors to do the following: characterize a sufficient number of DUTs by performing full scans of data points for each DUT to generate a sufficient amount of data for training; perform analysis of each full scan of the data point to determine whether the full scan of the data point is a pass or fail result; use the pass or fail result as a label for training the machine learning network; extract each full scan of the data point to generate an extracted data set corresponding to each full scan of the data point; and train the machine learning network to associate each extracted data set with a pass or fail result label of the corresponding full scan of the data point.

[0044] Example 9 is the test and measurement instrument of Example 8, wherein the one or more processors are further configured to execute code that causes the one or more processors to convert the extracted data set into tensors before training the machine learning network.

[0045] Example 10 is a method comprising: acquiring a signal from a device under test (DUT); measuring the signal to create an extracted measurement set; converting the extracted measurement set into a tensor; sending the tensor to a machine learning network; and receiving a pass / fail value from the machine learning network.

[0046] Example 11 is the method of Example 10, wherein receiving a pass / fail value from the machine learning network causes the one or more processors to receive the pass / fail value as a value between 1.0 and 0.0, inclusive.

[0047] Example 12 is the method of any of Examples 10 or 11, further comprising comparing the pass / fail value to a threshold value to determine whether the DUT passes or fails.

[0048] Example 13 is the method of Example 12, wherein the DUT passes when the pass / fail value is greater than or equal to 0.8, and the DUT fails when the value is less than or equal to 0.2.

[0049] Example 14 is the method of any of Examples 10 to 13, further comprising performing a full scan measurement of the DUT when the pass / fail indication is unclear.

[0050] Example 15 is the method of any one of Examples 10 to 13, wherein a user provides input through a user interface selecting to perform a full scan measurement.

[0051] Example 16 is the method of any one of Examples 10 to 13, further comprising training a machine learning network.

[0052] Example 17 is the method of Example 16, wherein training the machine learning network includes: characterizing a sufficient number of DUTs by performing full scans of data points for each data set to generate a sufficient amount of data for training; performing analysis of each full scan of the data point to determine whether the full scan of the data point is a pass or fail result; using the pass or fail result as a label for training the machine learning network; extracting each full scan of the data point to generate an extracted data set corresponding to each full scan of the data point; and training the machine learning network to associate each extracted data set with a pass or fail result label of the corresponding full scan of the data point.

[0053] Example 18 is the method of Example 17, further comprising converting the extracted data set into a tensor before training the machine learning network.

[0054] Additionally, this written description makes reference to specific features. It will be understood that the disclosure in this specification includes all possible combinations of those specific features. Where a specific feature is disclosed in the context of a particular aspect or example, the feature may also be used in the context of other aspects and examples as much as possible.

[0055] Furthermore, when reference is made in the present application to a method having two or more defined steps or operations, the defined steps or operations may be performed in any order or simultaneously, unless the context excludes those possibilities.

[0056] All features disclosed in the specification including claims, abstract and drawings, and all steps in any disclosed method or process may be combined in any combination, except combinations in which at least some of such features and / or steps are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in the specification including claims, abstract and drawings may be replaced by an alternative feature serving the same, equivalent or similar purpose.

[0057] Although specific examples of the present invention have been shown and described for purposes of illustration, it will be appreciated that various modifications may be made without departing from the spirit and scope of the present invention. Accordingly, the present invention should not be limited, except as by the appended claims.

Claims

1. A test and measurement instrument comprising: One or more ports that allow a test and measurement instrument to receive signals from a device under test (DUT); A user interface for allowing a user to send input to the test and measurement instrument and receive results; and One or more processors configured to execute code, the code causing the one or more processors to: Get the signal from the DUT; performing measurements on the signal to create an extracted measurement set; Convert the extracted measurement set into a tensor; Send tensors to the machine learning network; and Receive a pass / fail value from a machine learning network.

2. The test and measurement instrument of claim 1, wherein: Code causing the one or more processors to receive a pass / fail value from the machine learning network causes the one or more processors to receive the pass / fail value as a value between 1.0 and 0.0, inclusive.

3. The test and measurement instrument of claim 1, wherein: The one or more processors are further configured to execute code such that the one or more processors compare the pass / fail value to a threshold value to determine whether the DUT passes or fails.

4. The test and measurement instrument of claim 3, wherein: When the pass / fail value is greater than or equal to 0.8, the DUT passes, and when the value is less than or equal to 0.2, the DUT fails.

5. The test and measurement instrument of claim 1, wherein: The one or more processors are further configured to execute code to cause the one or more processors to perform a full scan measurement of the DUT when the pass / fail indication is unclear.

6. The test and measurement instrument of claim 5, wherein: A user provides input through a user interface selecting to perform a full scan measurement.

7. The test and measurement instrument of claim 1, wherein: The one or more processors are further configured to execute code that causes the one or more processors to train a machine learning network.

8. The test and measurement instrument of claim 7, wherein: The code causing the one or more processors to train a machine learning network includes code causing the one or more processors to: Characterizing a sufficient number of DUTs by performing a full scan of data points for each DUT to generate a sufficient amount of data for training; performing an analysis of each full scan of the data point to determine whether the full scan of the data point is a pass or fail result; Use pass or fail results as labels for training machine learning networks; decimating each full scan of data points to produce a decimated data set corresponding to each full scan of data points; and A machine learning network is trained to associate each extracted data set with a pass or fail result label of the full scan of the corresponding data point.

9. The test and measurement instrument of claim 8, wherein: The one or more processors are further configured to execute code that causes the one or more processors to convert the extracted data set into tensors before training the machine learning network.

10. A method comprising: Acquire signals from the device under test (DUT); performing measurements on the signal to create an extracted measurement set; Convert the extracted measurement set into a tensor; Send tensors to the machine learning network; and Receive a pass / fail value from a machine learning network.

11. The method according to claim 10, wherein: Receiving a pass / fail value from the machine learning network causes the one or more processors to receive the pass / fail value as a value between 1.0 and 0.0, inclusive.

12. The method of claim 10, further comprising comparing the pass / fail value to a threshold value to determine whether the DUT passes or fails.

13. The method according to claim 12, wherein: When the pass / fail value is greater than or equal to 0.8, the DUT passes, and when the value is less than or equal to 0.2, the DUT fails.

14. The method of claim 10, further comprising performing a full scan measurement of the DUT when the pass / fail indication is unclear.

15. The method according to claim 10, wherein: A user provides input through a user interface selecting to perform a full scan measurement.

16. The method of claim 10, further comprising training a machine learning network.

17. The method according to claim 16, wherein: Training a machine learning network involves: Characterizing a sufficient number of DUTs by performing a full scan of the data points for each data set to generate a sufficient amount of data for training; performing an analysis of each full scan of the data point to determine whether the full scan of the data point is a pass or fail result; Use pass or fail results as labels for training machine learning networks; decimating each full scan of data points to produce a decimated data set corresponding to each full scan of data points; and A machine learning network is trained to associate each extracted data set with a pass or fail result label of the full scan of the corresponding data point.

18. The method of claim 17, further comprising converting the extracted data set into a tensor before training the machine learning network.

Images